Amazon Virtual Private Cloud

Deep Dive
Randall Hunt – Developer Evangelist, AWS

©2015, Amazon Web Services, Inc. or its affiliates. All rights reserved
Related Presentations – Videos online
https://www.youtube.com/user/AmazonWebServices

• ARC205 – VPC Fundamentals and Connectivity

• ARC401 – Black Belt Networking for Cloud Ninja
– Application centric, network monitoring, management, floating IPs
• ARC403 – From One to Many: Evolving VPC Design
• SDD302 – A Tale of One Thousand Instances
– Example of EC2-Classic customer adopting VPC
• SDD419 – Amazon EC2 Networking Deep Dive
– Network performance, placement groups, enhanced networking
• SDD422 – Amazon VPC Deep Dive (this talk)
Topics today
Virtual networking options

EC2-Classic Default VPC VPC

Simple to get started – The best of both Advanced virtual

all instances have networking services:
Internet connectivity, Get started using the ENIs and multiple IPs
auto-assigned private EC2-Classic routing tables
and public IP addresses experience egress security groups
network ACLs
Inbound security groups If and when needed, private connectivity
begin using any VPC
feature you require Enhanced networking

And more to come...

Virtual networking options

EC2-Classic Default VPC VPC

Simple to get started – The best of both Advanced virtual

all instances have networking services:
Internet connectivity, Get started using the ENIs and multiple IPs
auto-assigned private EC2-Classic routing tables
and public IP addresses experience egress security groups
network ACLs
Inbound security groups
All accounts created after If and when needed, private connectivity
12/4/2013 support VPC begin using any VPC
feature you require Enhanced networking
only and have a default
VPC in each region And more to come...
Confirming your default VPC
describe-account-attributes

VPC only
1. Routing & private connections
Implementing a hybrid architecture

Corporate Data Center

Create VPC

Corporate Data Center

aws ec2 create-vpc --cidr 10.10.0.0/16

aws ec2 create-subnet --vpc vpc-c15180a4 --cidr 10.10.1.0/24 --a us-west-2a
aws ec2 create-subnet --vpc vpc-c15180a4 --cidr 10.10.2.0/24 --a us-west-2b
Create VPN connection

Corporate Data Center

aws ec2 create-vpn-gateway --type ipsec.1

aws ec2 attach-vpn-gateway --vpn vgw-f9da06e7 --vpc vpc-c15180a4
aws ec2 create-customer-gateway --type ipsec.1 --public 54.64.1.2 --bgp 6500
aws ec2 create-vpn-connection --vpn vgw-f9da06e7 --cust cgw-f4d905ea --t ipsec.1
Launch instances

Corporate Data Center

aws ec2 run-instances --image ami-d636bde6 --sub subnet-d83d91bd --count 3

aws ec2 run-instances --image ami-d636bde6 --sub subnet-b734f6c0 --count 3
Using AWS Direct Connect

Corporate Data Center

aws directconnect create-connection --loc EqSE2 --b 1Gbps --conn My_First

aws directconnect create-private-virtual-interface --conn dxcon-fgp13h2s --new
virtualInterfaceName=Foo, vlan=10, asn=60, authKey=testing,
amazonAddress=192.168.0.1/24, customerAddress=192.168.0.2/24,
virtualGatewayId=vgw-f9da06e7
Configuring route table

Corporate Data Center

192.168.0.0/16

Each VPC has a single

routing table at creation time,
used by all subnets

aws ec2 create-route --ro rtb-ef36e58a --dest 0.0.0.0/0 --gateway-id vgw-f9da06e7

Remote connectivity best practices

Availability Zone Availability Zone

Each VPN connection

consists of 2 IPSec
tunnels. Use BGP for
failure recovery.

Corporate Data Center

Remote connectivity best practices

Availability Zone Availability Zone

A pair of VPN
connections (4 IPSec
tunnels total) protects
against failure of your
customer gateway

Corporate Data Center

Remote connectivity best practices

Availability Zone Availability Zone

Redundant AWS Direct

Connect connections
with VPN backup

Corporate Data Center

VPC with private and public connectivity

Corporate Data Center

192.168.0.0/16

aws ec2 create-internet-gateway

aws ec2 attach-internet-gateway --internet igw-5a1ae13f --vpc vpc-c15180a4
aws ec2 delete-route --ro rtb-ef36e58a --dest 0.0.0.0/0
aws ec2 create-route --ro rtb-ef36e58a --dest 0.0.0.0/0 --gateway-id igw-5a1ae13f
aws ec2 create-route --ro rtb-ef36e58a --dest 192.168.0.0/16 --gateway-id vgw-f9da06e7
Automatic route propagation from VGW

Corporate Data Center

192.168.0.0/16

Used to automatically update routing

table(s) with routes present in the VGW

aws ec2 delete-route --ro rtb-ef36e58a --dest 192.168.0.0/16

aws ec2 enable-vgw-route-propagation --ro rtb-ef36e58a --gateway-id vgw-f9da06e7
Isolating connectivity by subnet

Corporate
192.168.0.0/16
Subnet with connectivity only
to other instances and the
Internet via the IGW

aws ec2 create-subnet --vpc vpc-c15180a4 --cidr 10.10.3.0/24 --a us-west-2b

aws ec2 create-route-table --vpc vpc-c15180a4
aws ec2 associate-route-table --ro rtb-fc61b299 --subnet subnet-60975a17
aws ec2 create-route --ro rtb-ef36e58a --dest 0.0.0.0/0 --gateway-id igw-5a1ae13f
Software VPN for VPC-to-VPC connectivity

# VPC A
aws ec2 modify-network-interface-attribute --net eni-f832afcc --no-source-dest-check
aws ec2 create-route --ro rtb-ef36e58a --dest 10.20.0.0/16 --instance-id i-f832afcc
# VPC B
aws ec2 modify-network-interface-attribute --net eni-9c1b693a --no-source-dest-check
aws ec2 create-route --ro rtb-67a2b31c --dest 10.10.0.0/16 –-instance-id i-9c1b693a
Software VPN for VPC-to-VPC connectivity

Software VPN
between these
instances
Software VPN for VPC-to-VPC connectivity

Enabling communication
between instances in these
subnets; adding routes to the
default routing table
Software firewall to the Internet

Routing all traffic from subnets

to the Internet via a firewall is
conceptually similar

# Default routing table directs traffic to the NAT/firewall instance

aws ec2 create-route --ro rtb-ef36e58a --dest 0.0.0.0/0 --instance-id i-f832afcc

# Routing table for 10.10.3.0/24 directs to the Internet

aws ec2 create-route --ro rtb-67a2b31c --dest 0.0.0.0/0 --gateway-id igw-5a1ae13f
2. VPC peering
Shared services VPC using VPC peering

• Common/core services
– Authentication/directory
– Monitoring
– Logging
– Remote administration
– Scanning
Provides infrastructure zoning
• Dev: VPC B
• Test: VPC C
• Production: VPC D
VPC peering for VPC-to-VPC connectivity

VPC A - 10.10.0.0/16 VPC B - 10.20.0.0/16

vpc-c15180a4 vpc-062dfc63

aws ec2 create-vpc-peering-connection --vpc-id vpc-c15180a4 --peer-vpc vpc-062dfc63

aws ec2 accept-vpc-peering-connection --vpc-peer pcx-ee56be87
VPC A> aws ec2 create-route --ro rtb-ef36e58a --des 10.20.0.0/16 --vpc-peer pcx-ee56be87
VPC B> aws ec2 create-route --ro rtb-67a2b31c --des 10.10.0.0/16 --vpc-peer pcx-ee56be87
VPC peering across accounts

VPC B - 10.20.0.0/16
VPC A - 10.10.0.0/16 vpc-062dfc63
vpc-c15180a4 Account ID 472752909333

aws ec2 create-vpc-peering-connection --vpc-id vpc-c15180a4 --peer-vpc vpc-062dfc63

--peer-owner 472752909333
# In owner account 472752909333
aws ec2 accept-vpc-peering-connection --vpc-peer pcx-ee56be87
VPC peering – Additional considerations

• Security groups not supported across peerings

– Workaround: specify rules by IP prefix
• No “transit” capability for VPN, AWS Direct
Connect, or 3rd VPCs
– Example: Cannot access VPC C from VPC A via VPC B
– Workaround: Create a direct peering from VPC A to VPC C
• Peer VPC address ranges cannot overlap
– But, you can peer with 2+ VPCs that themselves overlap
– Use subnets/routing tables to pick the VPC to use
VPC peering with software firewall

VPC A - 10.10.0.0/16 VPC B - 10.20.0.0/16

# Default routing table directs Peer traffic to the NAT/firewall instance

aws ec2 create-route --ro rtb-ef36e58a --dest 10.20.0.0/16 --instance-id i-f832afcc

# Routing table for 10.10.3.0/24 directs to the Peering

aws ec2 create-route --ro rtb-67a2b31c --dest 10.20.0.0/16 --vpc-peer pcx-ee56be87
3. Enhanced networking
Latency: Packets per second

Instance 1 Instance 2
...........
Packet processing in Amazon EC2:
VIF

Instance Virtual NICs

eth1
eth0
Physical NIC

Virtualization layer
Packet processing in Amazon EC2:
SR-IOV

Instance

eth1
eth0
Physical NIC
VF Driver

VF

Virtualization layer
Inter-instance latency
SR-IOV: Is this thing on?
It may already be!

For many newer AMIs, enhanced

networking is already on:
• Newest Amazon Linux AMIs
• Windows Server 2012 R2 AMI
No need to configure
SRIOV: Is this thing on? (Linux)

No Yes!
[ec2-user@ip-10-0-3-70 [ec2-user@ip-10-0-3-70 ~]$
~]$ ethtool -i eth0 ethtool -i eth0

driver: vif driver: ixgbevf

version: version: 2.14.2+amzn
firmware-version: firmware-version: N/A
bus-info: vif-0 bus-info: 0000:00:03.0
… …
SRIOV: Is this thing on? (Windows)
No Yes!
AMI/instance support for SR-IOV
• C3, C4, I2, D2, R3 instance families: 23 types
• HVM virtualization type
• Required kernel version
– Linux: 2.6.32+
– Windows: Server 2008 R2+
• Appropriate VF driver
– Linux: ixgbevf 2.14.2+ module
– Windows: Intel® 82599 Virtual Function driver
 Walkthrough: Enabling enhanced networking
(Amazon Linux)

amzn-ami-hvm-2012.03.1.x86_64-ebs

hvm
Walkthrough: Enabling enhanced networking
(Amazon Linux)

--attribute
sriovNetSupport

Not yet!
InstanceId i-37c5d1d9
Walkthrough: Enabling enhanced networking
(Amazon Linux)

OS update

[ec2-user@ip-10-0-3-125 ~]$ sudo yum update

Walkthrough: Enabling enhanced networking
(Amazon Linux)
reboot-instances

Reboot
(OS update)
Walkthrough: Enabling enhanced networking
(Windows)
Walkthrough: Enabling enhanced networking
(Windows)
Add to Windows driver store
Walkthrough: Enabling enhanced networking
All EBS-backed instances
stop-instances

Stop the instance

Walkthrough: Enabling enhanced networking
All EBS-backed instances
stop-instances

--sriov-net-support
simple
Enable SRIOV
Cannot be undone
Walkthrough: Enabling enhanced networking
All EBS-backed instances
start-instances

Start
Walkthrough: Enabling enhanced networking
All EBS-backed instances
start-instances

--attribute
sriovNetSupport

InstanceId i-37c5d1d9

Value simple
We’re on
 Subnet A Subnet A2 Subnet C
us-east-1a us-east-1a us-east-1c
10.0.1.0/24 10.0.2.0/24 10.0.3.0/24

Instance
1
10.0.2.50
10.0.1.100 Instance
3 10.0.3.99
Instance
4
Instance
2
10.0.2.51
10.0.1.101

Elastic
network
interface
 Subnet A Subnet A2 Subnet C
us-east-1a us-east-1a us-east-1c
10.0.1.0/24 Placement group 10.0.2.0/24 10.0.3.0/24

Instance
1
10.0.2.50
10.0.1.100 Instance
3 10.0.3.99
Instance
4
Instance
2
10.0.2.51
10.0.1.101

elastic
network
interface
Placement Groups

• ~1.5-3x better inter-instance ping (YMMV)

• Cannot span AZs
• Cannot be applied to running instances
• Only available for certain instance types
• Not great for things that scale horizontally
(capacity limited)
4. VPC for EC2-Classic customers
Adopting VPC

• Customers tell us they want to adopt VPC

• Have significant EC2-Classic
infrastructure
• Where do I start?
Start simple

• One subnet per AZ

• Each instance has a public
IP address and Internet
connectivity
• Use security groups to
control access
Add features at your own pace

• Multiple interfaces per

instance
• Multiple IPs per interface
• Enhanced networking
• Private connectivity
• VPC peering
• …
VPC ClassicLink

• Incremental adoption of VPC

• Private IP communication
between EC2-Classic and VPC
instances
• Security groups between EC2-
Classic and VPC instances
• Designed for the largest
deployments
ClassicLink

Route53

ELB

RDS DB
Instance
ClassicLink

Route53

ELB

RDS DB
Instance
ClassicLink

Route53

ELB

RDS DB
Instance
ClassicLink

Route53

ELB

RDS DB
Instance
ClassicLink

Route53

ELB

RDS DB
Instance
ClassicLink

Route53

RDS DB
Instance
ClassicLink

Route53

RDS DB
Instance
ClassicLink
• Preparation: Create VPC and configure for ClassicLink
• Create VPC security groups and deploy VPC components
• Add EC2-Classic instances to your VPC security groups
• Deploy components in stages in VPC
• Clean up un-used EC2-Classic instances
Pros Cons
(Potentially) No disruptive maintenance Additional complexity during migration
Direct private IP connectivity and security group Still need to replace EC2-Classic
integration instances with new VPC instances
Designed for the largest deployments
ClassicLink – Component stages

• Start with AWS-managed

infrastructure
– RDS, ElastiCache, Redshift
RDS DB ElastiCache Elastic Load • Next ELB
EC2-Classic Instance Cache Node Balancer
• Then instances
ClassicLink

RDS DB ElastiCache Elastic Load

Instance Cache Node Balancer
ClassicLink – Additional considerations
• VPC address ranges for use with ClassicLink
– 10.0.0.0/15, or any other range outside 10.0.0.0/8
– Why? EC2-Classic instance private IP addresses are in 10.2.0.0 – 10.255.255.255

• VPC also can’t have extra route table entries to 10.0.0.0/8

• ClassicLink instances use EC2-Classic for all Internet traffic. No
access from VPN/Direct Connect or a VPC peer to a ClassicLink
instance.
• ClassicLink must be enabled after instance launch (Run) or Start
• VPC instance DNS names do not resolve from EC2-Classic, and vice-
versa
ClassicLink APIs & CLI
Enabling ClassicLink
vpc-4325f426

To use ClassicLink the VPC must

have this feature enabled. Can
be restricted with IAM policy.
Attaching a EC2-Classic instance to a VPC

i-2b3ecd1c
vpc-4325f426 sg-da107fbf

Link this specific instance to

the VPC using the specified
VPC security groups
Attaching a EC2-Classic instance to a VPC

i-2b3ecd1c
vpc-4325f426 sg-da107fbf

Link required after Run (new

instance launch) or Start
(stopped instance)
ClassicLink and other services

• Elastic Load Balancing

– EC2-Classic instances can be backends of VPC balancers

• Spot
– Running spot instances can be linked

• Auto Scaling
– Configure to link classic instances following launch