DevOps Shack

Comprehensive Guide to Amazon EC2

(Elastic Compute Cloud)
Introduction
Amazon Elastic Compute Cloud (EC2) is one of the core services offered by
Amazon Web Services (AWS) and is widely regarded as a foundational
component in cloud computing. Launched in 2006, EC2 provides scalable
computing capacity in the cloud, allowing businesses and developers to run
virtual servers, known as instances, without the need to invest in or maintain
physical hardware. With EC2, users can quickly deploy applications, scale up or
down based on demand, and pay only for the computing resources they use.
This flexibility and scalability make EC2 ideal for a variety of use cases, from
hosting websites and applications to running data-intensive workloads like
machine learning, big data processing, and high-performance computing. By
eliminating the operational overhead of managing physical infrastructure,
Amazon EC2 enables organizations to focus on innovation, cost optimization,
and enhanced operational efficiency.
EC2 Architecture
Understanding EC2’s architecture is critical to leveraging its capabilities
effectively. The key components include:
1. Instances
Instances are virtual servers running on physical machines. Each instance is
created from an Amazon Machine Image (AMI) and configured with specific
instance types to meet performance needs.
2. Amazon Machine Images (AMIs)
AMIs are templates containing the operating system, application server, and
application configurations. AWS offers three types of AMIs:
• AWS-provided: Preconfigured images (e.g., Amazon Linux, Windows
Server).
• Marketplace: Third-party AMIs for specific software or use cases.
• Custom AMIs: User-created images tailored for specific applications.
3. Elastic Block Store (EBS)
EBS provides persistent block storage for EC2 instances. It retains data even
after an instance is terminated. You can attach multiple EBS volumes to a single
instance for scalable storage solutions.
4. Elastic IPs
Elastic IP addresses are static public IPv4 addresses that remain associated with
your account even when the attached instance is stopped or terminated. They
ensure continuous accessibility to your applications.
5. Security Groups
Security groups act as virtual firewalls that control inbound and outbound
traffic to instances. Rules define allowed IP addresses, ports, and protocols.
6. Key Pairs
Key pairs facilitate secure access to EC2 instances using SSH. The private key is
downloaded and stored locally, while the public key is uploaded to the instance
during creation.
EC2 Use Cases
Amazon EC2’s versatility makes it suitable for various industries and
applications. Some key use cases include:
1. Web Hosting
Host dynamic websites and web applications using EC2. Combine EC2 with
Elastic Load Balancing (ELB) and Auto Scaling for high availability and fault
tolerance.
2. Big Data Processing
Use EC2 instances with high compute and memory capacity to process large
datasets. Examples include running Apache Hadoop or Spark clusters.
3. Machine Learning
Train and deploy machine learning models with GPU-optimized EC2 instances
like p3 and g4 families. EC2 offers the flexibility to scale resources based on the
model's complexity.
4. Batch Processing
Run large-scale batch jobs efficiently with EC2 Spot Instances. These jobs can
be scheduled during periods of lower demand to reduce costs.
5. Disaster Recovery
Implement disaster recovery solutions by replicating on-premises
environments in EC2. During outages, failover to EC2 ensures business
continuity.

Advanced EC2 Configurations

EC2 provides several advanced features that enhance performance, reliability,
and usability.
1. Instance Lifecycle
EC2 instances support various lifecycle states, each serving a purpose:
• Running: The instance is active and accessible.
 • Stopped: The instance is halted, but EBS volumes remain attached.
• Hibernation: Preserves the instance’s in-memory state for faster restarts.
• Terminated: The instance is permanently shut down, and resources are
released.
2. Elastic Load Balancing (ELB) Integration
Distribute traffic across multiple EC2 instances using ELB. It improves
application fault tolerance and performance. Choose from:
• Application Load Balancer (ALB): For HTTP/HTTPS traffic with advanced
routing.
• Network Load Balancer (NLB): For TCP/UDP traffic requiring ultra-low
latency.
• Classic Load Balancer (CLB): For legacy applications.
3. Placement Groups
Placement groups control how instances are deployed across AWS
infrastructure:
• Cluster Placement: Instances are grouped in a single Availability Zone for
low-latency communication.
• Spread Placement: Instances are distributed across hardware to reduce
failure risk.
• Partition Placement: Instances are divided into partitions for fault
isolation.
4. Elastic GPUs
Attach GPUs to standard EC2 instances to accelerate graphics-intensive
applications without using specialized GPU instances.
5. Elastic Fabric Adapter (EFA)
Enhance the performance of high-performance computing (HPC) and machine
learning applications with lower network latency and higher bandwidth.
Network ACLs (Access Control Lists)
Network ACLs are an essential part of Amazon VPC (Virtual Private Cloud)
security in AWS. They act as a stateless firewall, controlling traffic entering or
leaving one or more subnets within your VPC. Unlike security groups, which are
applied to instances, Network ACLs are applied at the subnet level and can
control both inbound and outbound traffic.
Key Features of Network ACLs:
• Stateless: Network ACLs do not keep track of traffic flow. If you allow
inbound traffic, you must also explicitly allow the corresponding
outbound traffic.
• Rule-based: You can define both allow and deny rules for both inbound
and outbound traffic. These rules are evaluated in numerical order, and
the first rule that matches the traffic will apply.
• Subnet-Level Control: Network ACLs provide control over traffic entering
or leaving specific subnets within a VPC, providing an additional layer of
security to your architecture.
• Default Network ACL: Every VPC comes with a default network ACL that
allows all inbound and outbound traffic. You can modify or create custom
ACLs for more granular control.
When to Use Network ACLs:
• For controlling traffic at the subnet level.
• When you need to explicitly deny specific traffic (Network ACLs support
"deny" rules, unlike security groups).
• To provide an additional layer of security alongside security groups.

Security Groups
Security Groups are virtual firewalls at the instance level, allowing you to
control inbound and outbound traffic for your EC2 instances. Unlike Network
ACLs, Security Groups are stateful, meaning that if you allow inbound traffic,
the corresponding outbound traffic is automatically allowed, and vice versa.
Key Features of Security Groups:
 • Stateful: Security groups automatically track the state of network
connections. Once you allow inbound traffic, the response traffic is
automatically allowed, even if there's no explicit outbound rule.
• Rule-based: Security groups operate based on allow rules only. You
cannot explicitly deny traffic. All traffic that isn’t explicitly allowed is
denied by default.
• Instance-Level Control: Security groups apply at the EC2 instance level.
Multiple security groups can be associated with an instance, and an
instance can have multiple security groups attached.
• Dynamic Updates: When you modify a security group, changes are
applied immediately to all instances associated with that group, making
it easy to manage and update security rules.
When to Use Security Groups:
• For controlling access to EC2 instances.
• When you need fine-grained control over the types of traffic allowed to
your instances.
• To manage access based on IP addresses, ports, and protocols.

Comparison:
Feature Network ACLs Security Groups

Level of
Subnet Level Instance Level
Application

Statefulness Stateless Stateful

Rules Can allow or deny traffic Only allows traffic (no deny rules)

Controls both inbound and Controls inbound traffic

Traffic Control
outbound traffic (outbound is automatic)

Rules are evaluated in Rules are evaluated based on the

Evaluation
numerical order instance’s associations
Feature Network ACLs Security Groups

Denies all inbound traffic by

Default Allows all inbound and
default, allows all outbound
Behavior outbound traffic by default
traffic

In summary, Network ACLs provide a broader, subnet-level security layer, while

Security Groups provide a more granular, instance-level security control. Using
both in combination enhances the overall security posture of your AWS
infrastructure.

Security Best Practices

Security is paramount when deploying EC2 instances. AWS provides several
tools and practices to safeguard your workloads:
1. Identity and Access Management (IAM)
Use IAM roles to assign granular permissions to EC2 instances. This eliminates
the need for hardcoding credentials in applications.
2. Security Groups and Network ACLs
Define strict inbound and outbound traffic rules. For instance:
• Allow SSH access only from trusted IP ranges.
• Restrict public access to sensitive ports like 3306 (MySQL).
3. Encryption
Encrypt EBS volumes and snapshots to protect sensitive data. EC2 also supports
TLS for secure data transmission.
4. Regular Updates
Keep your instances updated with the latest security patches. Use AWS Systems
Manager to automate patch management.
5. Logging and Monitoring
Enable logging with AWS CloudTrail and Amazon CloudWatch to track user
activity and detect anomalies.
Monitoring and Troubleshooting
Monitoring EC2 instances ensures optimal performance and helps diagnose
issues proactively.
1. Amazon CloudWatch
Use CloudWatch to monitor CPU, memory, and disk utilization. Set up alarms to
notify you of unusual activity.
2. AWS Systems Manager
Centralize operational data and automate maintenance tasks like patching,
compliance checks, and instance inventory collection.
3. EC2 Instance Metadata
Access metadata and user data through the instance’s local IP address.
Example:
bash
Copy code
curl http://169.254.169.254/latest/meta-data/
4. Elastic Load Balancer Health Checks
Configure health checks to detect unhealthy instances and redirect traffic to
healthy ones.

Cost Optimization Strategies

AWS EC2’s flexible pricing model includes several cost-saving options:
1. Reserved Instances
Commit to 1- or 3-year terms for predictable workloads, saving up to 75%
compared to On-Demand pricing.
2. Spot Instances
Leverage unused EC2 capacity for cost savings of up to 90%. Use them for fault-
tolerant workloads like batch processing.
3. Savings Plans
Purchase compute capacity commitments for greater flexibility across instance
families and regions.
4. Right-Sizing Instances
Use AWS Cost Explorer to identify underutilized instances and resize them for
better efficiency.
5. Auto Scaling
Automatically scale resources to match demand, ensuring you pay only for
what you use.
6. Data Transfer Optimization
Use Amazon CloudFront and AWS Direct Connect to minimize data transfer
costs.
EC2 in a Multi-Cloud Strategy
Although EC2 is an AWS service, it can integrate with multi-cloud architectures
using tools like HashiCorp Terraform. This enables organizations to deploy
workloads across AWS, Azure, and Google Cloud, ensuring redundancy and
avoiding vendor lock-in.
HANDS ON
Setting Up an EC2 Instance
Let’s start with the step-by-step process to launch an EC2 instance, including
console instructions and equivalent CLI commands.

1. Launching an EC2 Instance via the AWS Management Console

Step 1: Log in to the AWS Management Console
1. Navigate to the AWS EC2 Console.
2. Select Launch Instance under the "Instances" section.
Step 2: Configure the Instance
1. Choose AMI: Select an Amazon Machine Image (AMI), such as Amazon
Linux, Ubuntu, or Windows Server.
2. Instance Type: Choose an instance type, e.g., t2.micro for free tier-
eligible options.
Step 3: Configure Instance Details
1. Network: Select a Virtual Private Cloud (VPC).
2. Subnet: Choose a public subnet for internet-facing instances.
3. Auto-assign Public IP: Ensure it is enabled for external connectivity.
Step 4: Add Storage
1. Add an EBS volume.
2. Choose the size and volume type (General Purpose SSD, Provisioned
IOPS SSD, or Magnetic).
Step 5: Add Tags
Add key-value pairs to organize and identify resources.
Step 6: Configure Security Group
1. Create a security group to define allowed inbound and outbound traffic.
2. Allow port 22 (SSH) for Linux or port 3389 (RDP) for Windows.
Step 7: Launch and Connect
1. Generate a new key pair or select an existing one.
 2. Use the key pair to connect via SSH or RDP.

2. Launching EC2 Using AWS CLI

Install the AWS CLI, configure it with your credentials, and run the following
commands:
Step 1: Create a Key Pair
aws ec2 create-key-pair --key-name MyKeyPair --query 'KeyMaterial' --output
text > MyKeyPair.pem
chmod 400 MyKeyPair.pem
Step 2: Launch an Instance
aws ec2 run-instances \
--image-id ami-0c02fb55956c7d316 \ # Amazon Linux AMI
--count 1 \
--instance-type t2.micro \
--key-name MyKeyPair \
--security-group-ids sg-0123456789abcdef0 \
--subnet-id subnet-0123456789abcdef0 \
--tag-specifications
'ResourceType=instance,Tags=[{Key=Name,Value=MyInstance}]'
Step 3: Get the Public IP
aws ec2 describe-instances \
--filters "Name=tag:Name,Values=MyInstance" \
--query "Reservations[*].Instances[*].PublicIpAddress" \
--output text
Step 4: Connect to the Instance
ssh -i MyKeyPair.pem ec2-user@<Public-IP>
3. Examples for Different Use Cases
Example 1: Hosting a Web Application
Use an EC2 instance to host a simple web application:
Step 1: Launch an Amazon Linux Instance
Follow the steps mentioned above to launch an instance.
Step 2: Install a Web Server Connect to the instance and install Apache:
sudo yum update -y
sudo yum install httpd -y
sudo systemctl start httpd
sudo systemctl enable httpd
Step 3: Deploy a Website Create an index.html file:
echo "<h1>Welcome to My Web App!</h1>" | sudo tee
/var/www/html/index.html

Step 4: Allow HTTP Traffic

Modify the security group to allow inbound traffic on port 80.
Example 2: Big Data Processing with EC2 and Hadoop
Step 1: Launch Multiple EC2 Instances
• Launch multiple instances with high-memory configurations (m5.large or
above).
Step 2: Install Hadoop
Log in to each instance and install Hadoop:
sudo apt update
sudo apt install openjdk-8-jdk -y
wget https://downloads.apache.org/hadoop/common/stable/hadoop-
3.3.1.tar.gz
tar -xzf hadoop-3.3.1.tar.gz
Step 3: Configure Master and Worker Nodes
Set up the core-site.xml and hdfs-site.xml files to configure the Hadoop cluster.
4. EC2 Spot Instances: Cost-Effective Workloads
Spot instances allow you to utilize unused EC2 capacity at up to 90% savings.
Step 1: Launch a Spot Instance
aws ec2 run-instances \
--image-id ami-0c02fb55956c7d316 \
--instance-type t2.micro \
--key-name MyKeyPair \
--instance-market-options '{"MarketType":"spot"}'
Use Case: Spot instances are ideal for workloads like batch processing and data
analysis.

5. Scaling with Auto Scaling Groups

Automatically scale the number of EC2 instances to match demand:
Step 1: Create a Launch Template
aws ec2 create-launch-template \
--launch-template-name MyLaunchTemplate \
--version-description "Version1" \
--launch-template-data '{"ImageId":"ami-
0c02fb55956c7d316","InstanceType":"t2.micro"}'
Step 2: Create an Auto Scaling Group
aws autoscaling create-auto-scaling-group \
--auto-scaling-group-name MyAutoScalingGroup \
--launch-template "LaunchTemplateName=MyLaunchTemplate,Version=1" \
--min-size 1 \
--max-size 5 \
--desired-capacity 2 \
 --vpc-zone-identifier "subnet-0123456789abcdef0"
Step 3: Attach a Load Balancer Integrate an Elastic Load Balancer for traffic
distribution.

6. Security Practices
IAM Roles for EC2 Instances
Assign IAM roles to instances for secure access to AWS services:
aws iam create-role --role-name MyEC2Role --assume-role-policy-document
file://trust-policy.json
aws iam attach-role-policy --role-name MyEC2Role --policy-arn
arn:aws:iam::aws:policy/AmazonS3ReadOnlyAccess
Example Use Case: Accessing S3
1. Assign the IAM role to the EC2 instance.
2. Run the following command from the instance to access S3:
aws s3 ls

7. Cost Optimization Strategies

1. Reserved Instances: Save up to 75% by committing to a 1- or 3-year
term.
2. Spot Instances: Leverage spot pricing for flexible workloads.
3. Savings Plans: Benefit from usage-based discounts.
Analyze Costs with AWS Cost Explorer:
aws ce get-cost-and-usage \
--time-period Start=2024-12-01,End=2024-12-31 \
--granularity MONTHLY \
--metrics "BlendedCost"
Conclusion
Amazon EC2 remains the cornerstone of AWS's compute services, offering
unparalleled flexibility, scalability, and cost efficiency. By leveraging its diverse
instance types, advanced configurations, and integration capabilities,
businesses can address a wide array of workloads, from small-scale
applications to enterprise-grade deployments. With security best practices,
monitoring tools, and cost optimization strategies in place, EC2 empowers
organizations to innovate and grow in a cloud-first world.