Amazon Elastic Compute Cloud

Amazon Elastic Compute Cloud

Amazon Elastic Compute Cloud
Original author(s) Amazon.com, Inc.
Developer(s) Amazon.com
Initial release August 25, 2006
Development status Active
Operating system
Available in English
Type Virtual Private Server
License Proprietary software
Website aws.amazon.com/ec2/

Amazon Elastic Compute Cloud (EC2) is a central part of Amazon.com's cloud computing platform, Amazon Web Services (AWS). EC2 allows users to rent virtual computers on which to run their own computer applications. EC2 allows scalable deployment of applications by providing a Web service through which a user can boot an Amazon Machine Image to create a virtual machine, which Amazon calls an "instance", containing any software desired. A user can create, launch, and terminate server instances as needed, paying by the hour for active servers, hence the term "elastic". EC2 provides users with control over the geographical location of instances that allows for latency optimization and high levels of redundancy.[1]

In November 2010, Amazon switched its own retail website to EC2 and AWS.[2]

History

Amazon announced a limited public beta test of EC2 on August 25, 2006,[3] offering access on a first-come, first-served basis. Amazon added two new instance types (Large and Extra-Large) on October 16, 2007.[4] On May 29, 2008, two more types were added, High-CPU Medium and High-CPU Extra Large.[5] There are twelve types of instances available.[6]

Amazon added three new features on March 27, 2008:[7] static IP addresses, availability zones, and user selectable kernels. On August 20, 2008, Amazon added Elastic Block Store (EBS)[8] This provides persistent storage, a feature that had been lacking since the service was introduced.

Amazon EC2 went into full production when it dropped the beta label on October 23, 2008. On the same day, Amazon announced the following features:[9] a service level agreement for EC2, Microsoft Windows in beta form on EC2, Microsoft SQL Server in beta form on EC2, plans for an AWS management console, and plans for load balancing, autoscaling, and cloud monitoring services.[9] These features were subsequently added on May 18, 2009.[10]

Amazon EC2 was developed mostly by a team in Cape Town, South Africa led by Chris Pinkham.[11][12][13][14] Pinkham provided the initial architecture guidance for EC2 and then built the team and led the development of the project. Some other members of the early team included Christopher Brown, Quinton Hoole, Roland Paterson-Jones, Willem Van Biljon, Peter DeSantis, Marcin Kowalski, Benjamin Tobler, James Greenfield, Atle Jorgensen, Alan Powell, Eden Adogla, Liehann Loots, David Brown, Anton Eicher, Christopher de Kadt, Greg Kempe, Riel Smit, Jeremy Thurgood and Andries Dippenaar.

Elastic compute units

The elastic compute unit (ECU) was introduced by Amazon EC2 as an abstraction of computer resources. Amazon’s Definition of ECU notes “We use several benchmarks and tests to manage the consistency and predictability of the performance of an EC2 Compute Unit. One EC2 Compute Unit provides the equivalent CPU capacity of a 1.0-1.2 GHz 2007 Opteron or 2007 Xeon processor. This is also the equivalent to an early-2006 1.7 GHz Xeon processor referenced in our original documentation”

Elastic block storage

The Amazon Elastic Block Store (EBS) provides raw block devices that can be attached to Amazon EC2 instances. These block devices can then be used like any raw block device. In a typical use case, this would include formatting the device with a filesystem and mounting it. In addition EBS supports a number of advanced storage features, including snapshotting and cloning. EBS volumes can be up to 1TB in size. EBS volumes are built on replicated storage, so that the failure of a single component will not cause data loss. EBS was introduced to the general public by Amazon in August 2008.[8]

Instance types

EC2 uses Xen virtualization.[15] Each virtual machine, called an "instance", functions as a virtual private server. Amazon sizes instances based on "Elastic Compute Units". The performance of otherwise identical virtual machines may vary.[16]

As of December 2012, the following instance types were offered:[17]

Cost

Amazon CTO Werner Vogels announces price reductions at AWS Summit 2013 NYC

As of July 2014, Amazon charged about $0.013/hour ($9.7/month) for the smallest "Micro Instance" (t2.micro) virtual machine running Linux or Windows. Storage optimized instances cost as much as $6.820/hour (i2.8xlarge). "Reserved" instances can go as low as $4.5/month for a three-year prepaid plan.[note 1][19] The data transfer charge ranges from free to $0.12 per gigabyte, depending on the direction and monthly volume (inbound data transfer is free on all AWS services[20]).

AWS' pricing came under scrutiny due to the complexity of their pricing model.[21] Third party vendors, such as Cloudyn [22] and CloudCheckr[23] offer fee for service analysis to help reduce costs.

Free tier

As of December 2010 Amazon offered a bundle of free resource credits to new account holders. The credits are designed to run a "micro" sized server, storage (EBS), and bandwidth for one year. Charges are applied on demand so the credit need not be used in the first month.[24]

Reserved instances

Reserved instances enable EC2 or RDS service users to reserve an instance for one or three years. The corresponding per hour rate charged by Amazon to operate the instance is much less than the rate charged for on-demand instances.[25]

Reserved Instances can be purchased in three different ways: All Upfront, Partial Upfront and No Upfront. The different purchase options allow for different structuring of payment models.

Comparison table

API name Memory (GiB) Cores/Compute units Instance storage (GB) 32bit/64bit I/O performance EBS-optimizable On-demand cost (Linux: per hour on US East 1)
t1.micro 0.594 Variable/Variable 0 (EBS only) 32/64 Variable No $0.020
t2.micro 1 1/Variable 0 (EBS only) 32/64 Low No $0.013
t2.small 2 1/Variable 0 (EBS only) 32/64 Low No $0.026
t2.medium 4 2/Variable 0 (EBS only) 32/64 Low No $0.052
m1.small 1.7 1/1 160 32/64 Moderate No $0.060
m1.medium 3.75 1/2 410 32/64 Moderate No $0.120
m1.large 7.5 2/4 850 64 Moderate 500 Mbit/s $0.240
m1.xlarge 15 4/8 1600 64 High 1000 Mbit/s $0.480
m2.xlarge 17.1 2/6.5 420 64 Moderate No $0.410
m2.2xlarge 34.2 4/13 850 64 High 500 Mbit/s $0.820
m2.4xlarge 68.4 8/26 1690 64 High 1000 Mbit/s $1.640
m3.xlarge 15 4/13 0 (EBS only) 64 Moderate 500 Mbit/s $0.500
m3.2xlarge 30 8/26 0 (EBS only) 64 High 1000 Mbit/s $1.000
c1.medium 1.7 2/5 350 32/64 Moderate No $0.145
c1.xlarge 7 8/20 1690 64 High 1000 Mbit/s $0.580
c3.8xlarge 60 32 (Xeon E5-2680 v2) 64
c4.8xlarge 60 36 (Xeon E5-2666 v3) 64
cc1.4xlarge 23 2/33.5 (2 Intel Xeon X5570) 1690 64 Very High (10 Gbit) ? $1.300
cc2.8xlarge 60.5 2/88 (2 Intel Xeon E5-2670) 3370 64 Very High (10 Gbit) Not necessary $2.400
cr1.8xlarge 244 2/88 (2 Intel Xeon E5-2670) 240 (SSD) 64 Very High (10 Gbit) Not necessary $3.500
cg1.4xlarge 22 2/33.5 (2 Intel Xeon X5570) + 2 NVIDIA Tesla "Fermi" M2050 GPU 1960 64 Very High (10 Gbit) Not necessary $2.100
g2.2xlarge 15 Xeon E5-2670 + NVIDIA Tesla "Kepler" GPU 60 (SSD) 64
hi1.4xlarge 60.5 16/35 (8 cores + 8 hyperthreads) 2*1024 (SSD) 64 Very High (10 Gbit) Not necessary $3.100
hs1.8xlarge 117 16/35 (8 cores + 8 hyperthreads) 48000 (24 * 2TB drives) 64 Very High (10 Gbit) Not necessary $4.600

Features

Operating systems

When it launched in August 2006, the EC2 service offered Linux and later Sun Microsystems' OpenSolaris and Solaris Express Community Edition. In October 2008, EC2 added the Windows Server 2003 and Windows Server 2008 operating systems to the list of available operating systems.[26][27] In November 2012, Amazon officially supported running FreeBSD in EC2.[28] In March 2011, NetBSD AMIs became available.[29] In November 2012, Windows Server 2012 support was added.[30]

Amazon has their own Linux distribution based on the Fedora and Red Hat Enterprise Linux as a low cost offering known as the Amazon Linux AMI. Version 2013.03 included:[31]

Persistent storage

An EC2 instance may be launched with a choice of two types of storage for its boot disk or "root device." The first option is a local "instance-store" disk as a root device (originally the only choice). The second option is to use an EBS volume as a root device. Instance-store volumes are temporary storage, which survive rebooting an EC2 instance, but when the instance is stopped or terminated (e.g., by an API call, or due to a failure), this store is lost.

EBS volumes provide persistent storage independent of the lifetime of the EC2 instance, and act much like hard drives on a real server. More accurately, they appear as block devices to the operating system that are backed by Amazon's disk arrays. The OS is free to use the device however it wants. In the most common case, a file system is loaded and the volume acts as a hard drive. Another possible use is the creation of RAID arrays by combining two or more EBS volumes. RAID allows increases of speed and/or reliability of EBS. Users can set up and manage storage volumes of sizes from 1GB to 1TB. The volumes support snapshots, which can be taken from a GUI tool or the API. EBS volumes can be attached or detached from instances while they are running, and moved from one instance to another.[32]

Simple Storage Service (S3) is a storage system in which data is accessible to EC2 instances, or directly over the network to suitably authenticated callers (all communication is over HTTP). Amazon does not charge for the bandwidth for communications between EC2 instances and S3 storage "in the same region." Accessing S3 data stored in a different region (for example, data stored in Europe from a US East Coast EC2 instance) will be billed at Amazon's normal rates.

S3-based storage is priced per gigabyte per month. Applications access S3 through an API. For example, Apache Hadoop supports a special s3: filesystem to support reading from and writing to S3 storage during a MapReduce job. There are also S3 filesystems for Linux, which mount a remote S3 filestore on an EC2 image, as if it were local storage. As S3 is not a full POSIX filesystem, things may not behave the same as on a local disk (e.g., no locking support).

Elastic IP addresses

Amazon's elastic IP address feature is similar to static IP address in traditional data centers, with one key difference. A user can programmatically map an elastic IP address to any virtual machine instance without a network administrator's help and without having to wait for DNS to propagate the new binding. In this sense an Elastic IP Address belongs to the account and not to a virtual machine instance. It exists until it is explicitly removed, and remains associated with the account even while it is associated with no instance.

Amazon CloudWatch

Amazon CloudWatch is a Web service that provides real-time monitoring to Amazon's EC2 customers on their resource utilization such as CPU, disk and network. Cloudwatch does not provide any memory, disk space, or load average metrics without running additional software on the instance. Amazon provides example scripts for Linux instances.[33] The data is aggregated and provided through AWS management console. It can also be accessed through command line tools and Web API's, if the customer desires to monitor their EC2 resources through their enterprise monitoring software.

The metrics collected by Amazon CloudWatch enables Auto Scaling feature to dynamically add or remove EC2 instances. The customers are charged by the number of monitoring instances.

Since May 2011, Amazon CloudWatch accepts custom metrics that can be submitted programmatically via Web Services API and then monitored the same way as all other internal metrics, including setting up the alarms for them.

Automated scaling

Further information: Autoscaling

Amazon's auto scaling feature of EC2 allows it to automatically adapt computing capacity to site traffic. The schedule-based (e.g. time-of-the-day) and rule-based (e.g. CPU utilization thresholds) auto scaling mechanisms are easy to use and efficient for simple applications. However, one potential problem is that VMs may take up to several minutes to be ready to use, which are not suitable for time critical applications. The VM startup time are dependent on image size, VM type, data center locations, etc.[34]

Reliability

To make EC2 more fault-tolerant, Amazon engineered Availability Zones that are designed to be insulated from failures in other availability zones. Availability zones do not share the same infrastructure. Applications running in more than one availability zone can achieve higher availability.

EC2 provides users with control over the geographical location of instances that allows for latency optimization and high levels of redundancy. For example, to minimize downtime, a user can set up server instances in multiple zones that are insulated from each other for most causes of failure such that one backs up the other.

High availability database services, like Amazon Relational Database Service run on top of EC2 instances.

Competitors

Since the launch of EC2, several companies have launched competing solutions, including Google Compute Engine, Microsoft's Windows Azure Virtual Machines, Internap's Agile platform, as well as conventional infrastructure providers such as Rackspace.

Issues

In early July 2008 the anti-spam organizations Outblaze and Spamhaus.org began blocking Amazon's EC2 address pool due to problems with the distribution of spam and malware.[35]

On December 1, 2010, Amazon pulled its service to WikiLeaks after coming under political pressure in the US.[36] The Internet group Anonymous attempted to attack EC2 in revenge; however, Amazon was not affected by the attack.[37]

Amazon's websites were temporarily offline on December 12, 2010, although it was initially unclear if this was due to attacks or a hardware failure. An Amazon official later stated that it was due to a hardware failure.[38]

Wikinews has related news: Amazon server outage affects Reddit, other websites

Shortly before 5 am Eastern on April 21, 2011, an outage started at EC2's Northern Virginia data center that brought down several websites, including Foursquare, Springpad, Reddit, Quora, BigDoor and Hootsuite.[39][40][41] Specifically, attempts to use Amazon's elastic-disk and database services hung, failed, or were slow. Service was restored to some parts of the datacenter (three of four "availability zones" in Amazon's terms) by late afternoon Eastern time that day;[42] problems for at least some customers were continuing as of April 25.[43] 0.07% of EBS volumes in one zone have also been lost; EBS failures were a part of normal operation even before this outage and were a risk documented by Amazon,[44] though the number of failures and the number of simultaneous failures may find some EC2 users unprepared.

On Sunday August 6, 2011, Amazon suffered a power outage in one of their Ireland availability zones.[45] Lightning was originally blamed for the outage; however, on August 11, Irish energy supplier ESB Networks dismissed this as a cause, but at time of writing, could not confirm what the cause of the problem was.[46] The power outage raised multiple questions regarding Amazon's EBS infrastructure, which caused several bugs in their software to be exposed. The bugs resulted in some customers' data being deleted when recovering EBS volumes in a mid-write operation during the crash.[47]

August 8, 2011, saw another network connectivity outage of Amazon's Northern Virginia datacenter, knocking out the likes of Reddit, Quora, Netflix and FourSquare.[48] The outage lasted around 25 minutes.

Another Northern Virginia datacenter outage occurred on October 22, 2012, from approximately 10 am to 4 pm PT. Edmodo, Airbnb, Flipboard, Reddit, and other customers were affected. Anonymous claimed responsibility, however Amazon denied this assertion.[49]

See also

Notes

  1. ↑ $109 for a three-year Heavy Utilization Reserved t2.micro Instances reservation amortized over thirty-six months plus one month at $0.002/hour.

References

  1. ↑ LaMonica, Martin (March 27, 2008). "Amazon Web Services adds 'resiliency' to EC2 compute service". CNet News. Retrieved August 1, 2009.
  2. ↑ AWS Cloud Tour 2011 on YouTube
  3. ↑ Barr, Jeff (August 25, 2006). "Amazon EC2 Beta". Amazon Web Services Blog. Retrieved May 31, 2013.
  4. ↑ Jinesh (October 16, 2007). "Amazon EC2 Gets More Muscle".
  5. ↑ Barr, Jeff (May 29, 2008). "More EC2 power". Amazon Web Services Blog. Retrieved August 1, 2009.
  6. ↑ Amazon EC2 Instances. Aws.amazon.com. Retrieved on 2013-08-09.
  7. ↑ Barr, Jeff (March 27, 2008). "New EC2 Features". Amazon Web Services Blog. Retrieved August 1, 2009.
  8. ↑ 8.0 8.1 Barr, Jeff (August 20, 2008). "Amazon EBS (Elastic Block Store) - Bring Us Your Data". Amazon Web Services Blog. Archived from the original on March 28, 2011. Retrieved May 31, 2013.
  9. ↑ 9.0 9.1 Barr, Jeff (August 23, 2008). "Big Day For EC2". Amazon Web Services Blog. Retrieved August 1, 2009.
  10. ↑ Barr, Jeff (May 18, 2009). "New Features for EC2". Amazon Web Services Blog. Retrieved August 1, 2009.
  11. ↑ "Amazon opens Cape software centre". July 19, 2005.
  12. ↑ "Amazon opens dev centre in Cape Town". July 20, 2005.
  13. ↑ "Amazon's Pinkham quits". November 29, 2006.
  14. ↑ "Amazon’s early efforts at cloud computing? Partly accidental". June 17, 2010.
  15. ↑ "Citrix and Amazon Web Services (AWS)". citrix.com. Retrieved October 23, 2013.
  16. ↑ J. Dejun, G. Pierre and C.-H. Chi. EC2 Performance Analysis for Resource Provisioning of Service-Oriented Applications. In Proceedings of the Workshop on Non-Functional Properties and SLA Management in Service-Oriented Computing, November 2009.
  17. ↑ "Amazon EC2 Pricing". amazon.com. Retrieved December 5, 2012.
  18. ↑ "Technical Report CS-2011-09". cs.technion.ac.il. Retrieved December 5, 2012.
  19. ↑ Amazon Elastic Compute Cloud (Amazon EC2), Cloud Computing Servers. Aws.amazon.com (2014-07-01). Retrieved on 2014-07-01.
  20. ↑ "AWS Lowers its Pricing Again! – No Inbound Data Transfer Fees and Lower Outbound Data Transfer for All Services including Amazon CloudFront". typepad.com. June 29, 2011. Archived from the original on July 7, 2011. Retrieved July 6, 2011.
  21. ↑ Klint Finley (August 21, 2012). "Is There a Landmine Hidden in Amazon’s Glacier?". Wired. Retrieved May 31, 2013.
  22. ↑ "Cloudyn.com".
  23. ↑ "CloudCheckr.com".
  24. ↑ "AWS Free Usage Tier".
  25. ↑ Additional Reserved Instance Options for Amazon EC2
  26. ↑ Shankland, Stephen (October 23, 2008). "Amazon's Linux cloud computing out of beta, joined by Windows". CNet News. Retrieved October 24, 2008.
  27. ↑ "Amazon Elastic Compute Cloud (EC2) Running Microsoft Windows Server and SQL Server". Amazon.com. October 23, 2008. Archived from the original on 1 December 2008. Retrieved October 25, 2008.
  28. ↑ AWS Marketplace - Additional EC2 Operating System Support (FreeBSD, Debian, CentOS)
  29. ↑ NetBSD Blog. Blog.netbsd.org (2011-03-13). Retrieved on 2013-08-09.
  30. ↑ "Windows Server 2012 Now Available on AWS". Amazon.com. November 19, 2012. Retrieved March 26, 2014.
  31. ↑ "Amazon Linux AMI 2013.03 Release Notes". AWS web site. Retrieved May 31, 2013.
  32. ↑ "Introduction to EBS Volumes". 1201 Restart.com. December 2010. Retrieved December 18, 2010.
  33. ↑ "Henry". "Cloudwatch for memory usage"
  34. ↑ Mao, Ming; M. Humphrey (2012). "A Performance Study on the VM Startup Time in the Cloud". Proceedings of 2012 IEEE 5th International Conference on Cloud Computing (Cloud2012): 423. doi:10.1109/CLOUD.2012.103. ISBN 978-1-4673-2892-0.
  35. ↑ Krebs, Brian (July 1, 2008). "Amazon: Hey Spammers, Get Off My Cloud!". Washington Post. Retrieved August 1, 2009.
  36. ↑ MacAskill, Ewen (December 2, 2010). "WikiLeaks website pulled by Amazon after US political pressure". The Guardian (London).
  37. ↑ "WikiLeaks cables: Shell, Operation Payback and Assange for the Nobel prize — as it happened". The Guardian. December 9, 2010. Retrieved May 31, 2013.
  38. ↑ "Amazon says outage in Europe due to hardware failure, not hacking attack". The Guardian. December 13, 2010. Retrieved May 31, 2013.
  39. ↑ Pepitone, Julianne (April 21, 2011). "Amazon EC2 outage downs Reddit, Quora". CNN.
  40. ↑ Lohr, Steve (April 22, 2011). "Amazon Malfunction Raises Doubts About Cloud Computing". The New York Times.
  41. ↑ "Cloud storm, EC2 startup takedown". DoesWhat. April 21, 2011.
  42. ↑ 6:18 pm PDT April 21, 2011, entry, says "all Availability Zones except one have been functioning normally for the past 5 hours."
  43. ↑ April 25 entry.
  44. ↑ Amazon Elastic Block Store (EBS) section "Volume Durability"
  45. ↑ "Lightning strikes Amazon's European cloud". August 8, 2011.
  46. ↑ "Amazon web outage not caused by lightning". August 11, 2011.
  47. ↑ "AWS cloud accidentally deletes customer data". August 10, 2011.
  48. ↑ "Amazon cloud outage downs Netflix, Quora". August 9, 2011.
  49. ↑ "Amazon cloud outage impacts Reddit, Airbnb, Flipboard". October 22, 2012.

External links