AWS Hardware VPN and MikroTik

Recently a client decided to set up an AWS Hardware VPN to their site. The simplest way to research this seemed to be to set up a test VPN to my own router – a MikroTik 951G-2HnD running RouterOS 6.30.2. Here’s how I did it. My thanks to those on the Australian MikroTik users mailing list who provided assistance, and to Mike Everest of Duxtel for his advice. Another valuable resource was this blog entry by Arnt Gulbrandsen.

This article is not about the AWS end; the Amazon documentation is very complete. This article is about how to set up the MikroTik end.

You will need…

As part of setting up the AWS end (a VPC with a Hardware VPN) you will have supplied AWS with the outside IP address of your MikroTik router. AWS will have provided you with some configuration details – these are downloadable from AWS. I used the “generic” instructions.

Here are the mappings from those details to the variables used in the configuration statements below:

  • $AWS_OUT_1 the “Virtual Private Gateway Outside address” for Tunnel #1
  • $AWS_IN_1 the “Virtual Private Gateway Inside address” for Tunnel #1
  • $LOCAL_IN_1 the “Customer Gateway Inside address” for Tunnel #1
  • $SECRET_1 the “Pre-Shared Key” for Tunnel #1
  • $AWS_OUT_2 the “Virtual Private Gateway Outside address” for Tunnel #2
  • $AWS_IN_2 the “Virtual Private Gateway Inside address” for Tunnel #2
  • $LOCAL_IN_2 the “Customer Gateway Inside address” for Tunnel #2
  • $SECRET_2 the “Pre-Shared Key” for Tunnel #2
  • $LOCAL_OUT the Customer Gateway Outside address

For routing and NAT bypass, you will also need these two items, not provided in the AWS configuration instructions:

  • $LOCAL_NET your local network (including prefix length!)
  • $VPC_NET the network inside the VPC (including prefix length!)

Configuration

A single IPsec proposal can be used for both tunnels; it needs to look like this:

/ip ipsec proposal
    add name=AWS \
    add auth-algorithms=sha1 \
    enc-algorithms=aes-128-cbc \
    lifetime=8m \
    pfs-group=modp1024

Obviously if AWS changes their parameters you will need to adjust these to match.

Then you will need two peer statements, one for each of the AWS Virtual Private Gateway Outside Addresses:

/ip ipsec peer
    add comment="AWS Peer 1" \
    address=$AWS_OUT_1/32 \
    local-address=$LOCAL_OUT \
    secret=$SECRET_1 \
    dpd-interval=10s dpd-maximum-failures=3 \
    enc-algorithm=aes-128 lifetime=8h \
    nat-traversal=no
/ip ipsec peer
    add comment="AWS Peer 2" \
    address=$AWS_OUT_2/32 \
    local-address=$LOCAL_OUT \
    secret=$SECRET_2 \
    dpd-interval=10s dpd-maximum-failures=3 \
    enc-algorithm=aes-128 lifetime=8h \
    nat-traversal=no

And you will need four IPsec policies, two for each VPN. One of the two covers traffic from your local network to the VPC network; the other covers traffic from your inside address to the AWS inside address:

/ip ipsec policy
    add comment="AWS Tunnel 1" \
    proposal=AWS \
    src-address=0.0.0.0/0 dst-address=$VPC_NET
    sa-src-address=$LOCAL_OUT sa-dst-address=$AWS_OUT_1 \
    tunnel=yes
/ip ipsec policy
    add comment="AWS Tunnel 1" \
    proposal=AWS \ 
    src-address=0.0.0.0/0 dst-address=$AWS_IN_1/32 \
    sa-src-address=#$LOCAL_OUT sa-dst-address=$AWS_OUT_1  \
    tunnel=yes
/ip ipsec policy
    add comment="AWS Tunnel 2" \
    proposal=AWS \
    src-address=0.0.0.0/0 dst-address=$VPC_NET
    sa-src-address=$LOCAL_OUT sa-dst-address=$AWS_OUT_2 \
    tunnel=yes
/ip ipsec policy
    add comment="AWS Tunnel 2" \
    proposal=AWS \ 
    src-address=0.0.0.0/0 dst-address=$AWS_IN_2/32 \
    sa-src-address=#$LOCAL_OUT sa-dst-address=$AWS_OUT_2  \
    tunnel=yes

If your outside interface is doing NAT, you will need to exempt traffic between the local and remote networks. Add these lines before any masquerade NAT statements:

/ip firewall nat
    add chain=srcnat \
    action=accept \
    dst-address=$VPC_NET src-address=$LOCAL_NET
/ip firewall nat
    add chain=srcnat \
    action=accept \
    dst-address=$LOCAL_NET src-address=$VPC_NET

If you have more than one network at either end, add more such lines to exempt them from NAT too.

And you need to allow IKE (IPsec’s key negotiation protocol) through your firewall, as well as IPsec-encrypted traffic. Put the following four lines somewhere suitable in your filter rules. If you are not sure where to put them, insert them in front of all other rules.

/ip firewall filter
    add comment="AWS Tunnel 1" \
    chain=input \
    protocol=ipsec-esp \
    src-address=$AWS_OUT_1 dst-address=$LOCAL_OUT
/ip firewall filter
    add comment="AWS Tunnel 1" \
    chain=input \
    protocol=udp \
    src-address=$AWS_OUT_1  src-port=500 \
    dst-address=$LOCAL_OUT dst-port=500
/ip firewall filter
    add comment="AWS Tunnel 2" \
    chain=input \
    protocol=ipsec-esp \
    src-address=$AWS_OUT_2 dst-address=$LOCAL_OUT
/ip firewall filter
    add comment="AWS Tunnel 2" \
    chain=input \
    protocol=udp \
    src-address=$AWS_OUT_2  src-port=500 \
    dst-address=$LOCAL_OUT dst-port=500

Finally, you need to tell your MikroTik how to route traffic to your VPC:

/ip route
   add distance=1 dst-address=$VPC_NET gateway=$AWS_IN_1
   add distance=1 dst-address=$VPC_NET gateway=$AWS_IN_2

Yes – there are two routes for the same destination, because the VPC can be reached over either tunnel. If you have more than one network in your VPC, add suitable route statements for them too. The key point is that each route must be via an AWS Virtual Gateway Inside address.

Testing

Using the AWS console, check the tunnel status of your VPN. You should see that one of the tunnels is UP, the other DOWN. See “A MikroTik limitation” below for why this is so and (update) a possible workaround.

If a tunnel is UP, you should also be able to see valid security associations for it. Use this command:

/ip ipsec installed-sa print

You can test your new VPN by sending a ping to the AWS inside address of whichever tunnel is up.

If you have an instance running inside your VPN, you should be able to ping its IP address too. If your VPC is only routing to specific networks at your end of the VPN, you will need to specify an address in one of those networks as the source address of your ping. And of course, it has to be an address on the router!

If one of your tunnels is UP, but you can’t seem to get a ping through, double check the filters on the router at your end and the security groups and ACLs that apply to the VPC at the AWS end.

Using the command above, look at the installed security associations, particularly the “current-bytes” value. If current-bytes is increasing, then the SA is being used to transfer data; the tunnel is working, and the problem is probably filters.

 A note on virtual interfaces

Some router platforms implement IPsec on virtual interfaces. The AWS documentation talks about “tunnel interfaces”. On the MikroTik these are not necessary – the local “inside” addresses do not need to be assigned to any interface. They are referenced only in route statements and IPsec policy statements.

If you have only one interface to the Internet, and the router’s default route points over that interface, then packets to the AWS inside addresses will be routed over that interface and be handled by the relevant IPsec policy.

If you have more than one Internet-connected interface on your router, or if the single Internet-connected interface is not the default route, you will need to route the AWS inside addresses out the correct interface.

A MikrotTik limitation

In practice, one of your tunnels will always be “invalid”, because MikroTik has a limitation (maybe even a bug) in that regard – it will not permit two IPsec policies to cover the same traffic, and flags one as “invalid” if you try.

If you followed the above instruction and now look at your configured policies, you will see that two of them are marked “Invalid”. That’s OK – one or the other tunnel will work fine. If you want to be in control, you can disable one pair of policies yourself rather than leave it up to chance.

Since first writing this article I’ve come up with a sort of a workaround. For the MikroTik to invalidate a policy, the policy has to be cover the identical address range as another. I tricked it by setting up the second policy as a pair of policies, each covering half the desired address range.

For example, if your policy was supposed to send 192.168.100.0/24 over two VPNs, you could have one policy for that range and VPN1, then two more policies, for 192.168.100.0/2 and 192.168.10.128/25, for VPN2.

I’ve tried this and it does work, though you will not be able to use the network address of the second subnet nor the broadcast address of the first. And I have not tried it with with BGP, only with static routes.

Using a supernet works too. For example if you have 192.168.100.0/24 at the AWS end, you could make the policy on one of the VPNs for 192.168.100.0/24, and the other for 192.168.100.0/23. Of course, the supernet must not overlap any local networks! If you have the address range available, using a supernet is better than using two subnets, because you don’t lose two addresses out of the middle of the range, and don’t therefore have to do anything special at the AWS end.

Good error messages

I recall our little team getting into trouble many moons ago. We were writing a creditor system, and one of the requirements was for comments to be attachable to individual invoice lines. In COBOL every data structure has to be predefined. One of us thought that surely, surely, 400 comment lines would be enough for any one invoice line. This turned out not to be the case. Continue reading

APC BR900GI UPS tips

As the proud owner of a Synology DS415+ NAS (network attached storage) device, I thought I had better protect the large amount of data accumulating on it by also becoming the proud owner of a suitable UPS (uninterruptible power supply). That way, when the power fluctuates or goes off unexpectedly (as is quite common in rural Australia, where I live) the NAS is protected and will have time to shut itself down in an orderly fashion. So I purchased a Schneider (APC) BR900GI UPS. This article is about how I set things up.

Continue reading

A voice for heterogeneity

In a recent email conversation, the relative virtues of Hyper-V and VMWare were being discussed. I stuck my hand up to suggest going with both. Why? Because heterogeneity is a good thing. In this post the topic is two competing hypervisors, but the arguments are the same for almost any technology. And they mostly apply to open technologies as well as to closed, proprietary technologies. Continue reading

Fibre? Don’t make me laugh.

Apparently some people at University College London (UCL) have achieved data transmission speeds of greater than a terabit per second to a single receiver:

http://www.smh.com.au/technology/innovation/breakthrough-enables-downloads-50000-times-faster-than-superfast-broadband-20160211-gms3bd.html

To which I say – phooey!

It’s on fibre – but copper is the future.

Copper is faster, cheaper, and can be installed more quickly. That’s why all the really top-shelf research is concentrating on getting super high speeds over copper. Apparently people are already pushing tens of gigabits over inch-long snippets; it’s just a simple matter of scaling that up. And Australia’s world-class broadband network will be ready and waiting when they do!

UCL? Fibre? What a pack of amateurs. Sheesh.

Continue reading