simplr.

At Apigee we’re moving to Amazon VPCs to logically organize the various elements of our API management engine by subnet. Part of this change includes geographically distributed availability for our customers that are multiregional or multinational. For instance, we want replicated Cassandra cluster nodes in multiple regions. Amazon does not offer a native technology to connect VPCs so we are considering our options.

I’ve seen a few forum and mailing lists discussions on using OpenVPN with Amazon but nothing covered what performance could be expected. I did a few simple tests to determine if it would meet our needs.

First I set up a very basic openvpn configuration file.

Client in region us-ea:

remote <elastic.ip.addr.ess>
dev tun
ifconfig 10.8.0.2 10.8.0.1
secret static.key

Server in region us-west-1:

dev tun
ifconfig 10.8.0.1 10.8.0.2
secret static.key

I used this configuration to connect instances in a public subnet in either region, then used iperf to test performance between them. The instances I tested were small, large, and xlarge with high CPU, always on the Amazon Linux AMI. I could use openvpn’s --config flag to point at these files for basic configuration, then tried various tuning parameters on the command line.

I generally followed the advice in this OpenVPN community article. The best performance I found was on a large 64-bit instance using the following OpenVPN configuration

--tun-mtu 48000 --fragment 0 --mssfix 0 -engine aesni

iperf showed an average of 85Mbps with this configuration. Using --cipher none --auth none the speed was around 130Mbps. For reference, iperf averaged 200Mbps when testing between elastic IPs on the Internet without OpenVPN. Strangely, the performance was significantly lower on xl instances with high CPU than on a large instance.

Private-to-private subnet routing

As an aside, I also needed to test routing between private subnets on the OpenVPN-connected VPCs. The CIDR space option is important when creating a new VPC. The space between regions cannot overlap or instances cannot route properly. The steps to route between instances in private subnets are as follows

1. Create a VPC in one region using a specific CIDR block (I used 192.168.0.0/23 in us-east)
2. Repeat step one in another region using another address space (I used 192.168.10.0/23 in us-west)
3. Create an instance to run OpenVPN in a public subnet in both regions.
4. Add the routes to the other region in the OpenVPN configuration. For example, I added route 192.168.0.0 255.255.254.0 for the VPN instance in my us-west VPC.
5. Disable Source/Dest Check on the openvpn host by right-clicking the instance in the AWS management console. The host will not forward packets if the source/dest check is enabled!
6. Enable IP forwarding on the openvpn instances (echo 1 > /proc/sys/net/ipv4/ip_forwarding)
7. Add a route to the AWS route table via the AWS management interface. For example, I added a route for 192.168.10.0/23 to the openvpn instance in the public network on the us-east side. You cannot choose the openvpn instance as a target until step 5 above is complete.
8. Create an instance in the private subnet in both regions.

If everything is set up correctly (and your security groups permit it), you should be able to ping hosts in the private subnet on both sides of the VPN tunnel.