IPSec vs OpenVPN: Cloud+ Encryption Technologies

VPNs. You can’t go anywhere without hearing about how you need a VPN to secure your work and your data. Any tech blog or publication worth their salt espouses the need for one. Your IT department is always pushing on everyone to use it when using your laptop outside of the office. Or maybe you have to use the VPN to access your files, apps, or the company intranet when working remote. Heck, you can’t even watch a YouTube video or listen to a podcast anymore without hearing, “Brought to you today by our sponsor, [insert VPN company here]!”

So, we get it, use the VPN when working/surfing anywhere not at home or at the office. Secure your data. Prevent cyber snooping. Anonymize your traffic. Got it, I guess, I’m just assuming this is important because I’m told it is. Maybe a better understanding of the principles of infosec will help untangle the whys around VPNs, instead of just IT/YouTube sponsors telling me I should.

Why VPNs anyway?

VPNs are a long-time security mechanism for securing data. They safeguards traffic by securing your confidentiality, integrity, and availability. Those aren’t just buzz words designed to make you feel good about using your laptop at Starbucks. They form the three sides of a key infosec concept known as the CIA triad.

First confidentiality. This is the easy one that we always think of first when it comes to VPNs. They essentially encrypt all the traffic that passes between your laptop and whatever endpoint your VPN software is connecting to. That endpoint can be a physical or virtual server running VPN software, it can be a hardware appliance like a firewall or VPN concentrator. It can be hosted in any cloud provider, in a data center, or in your office’s server closet. The point is that traffic between your laptop and that other point is all encrypted.

Why? So no one can snoop on that traffic, either on the WiFi at the hotel, or a nefarious ISP scraping data about you, or whoever you don’t want to see what you are doing.

“But do I really need a VPN if so much web traffic is already encrypted with HTTPS?” A great question, especially because browsers like Chrome are pushing hard for HTTPS encryption on all websites. Essentially, the VPN will add another layer of encryption on top of the HTTPS encryption. HTTPS on its own is proven to be secure, so this layer might be redundant, but there are other benefits.

Even with HTTPS, a site will see that your traffic is coming from the city that you’re surfing from. Not with terribly great accuracy, but generally at least within a certain region or metro area. VPNs eliminate this, as your traffic will appear to come from wherever the VPN is terminated from (that cloud host or your office). In terms of the CIA triad point of confidentiality, masking your location definitely falls into this category.

Confidentiality is the big driving factor of using a VPN, but let’s still touch on the other two briefly. Integrity is ensuring that the data is not tampered with in transit. VPN protocols all include hashing algorithms to provide integrity.

As a reminder, a hash is a fixed length value generated algorithmically from a block of data. If even one bit of the original data changes, the hash will be completely different. When the data is encrypted on one end, a hash is also sent. The other decrypts the data, hashes it again, and compares this hash to the transmitted hash. If it’s a match, data integrity has been achieved.

The final point of the CIA triad is availability. For any system to be useful to users it must be available, that is to say not taken down by an DDoS attack, files held ransom by cryptolocker, or some other nefarious act keeping you from working.

A VPN helps availability in the corporate setting by reducing public attack area. If a company puts access to email, apps, file shares, etc. behind a VPN instead of allowing access from anywhere on the internet, they dramatically reduce the number of targets they present to bad guys. Less targets = less attacks = less chance of downtime on mission critical services.

Hopefully, you are now fully briefed on why VPNs are necessary. Now let’s get now into our main topic around which protocol reigns supreme.

Your Choices Regarding VPNs

If you’ve ever tried to set up any VPN, either from one of those companies advertising during your favorites videos or, on the sysadmin side, setting up a corporate firewall for hundreds or thousands of users, you’ll inevitably run into a decision about protocols. While not super exciting, the protocol you chose for your VPN can lead to significant differences in areas like encryption quality or performance, so it’s worth it to make the right choice.

We’ll take a deep dive today into two of the most common VPN systems: IPSec and OpenVPN. Which is the most secure? The fastest? The easiest to configure? The easiest to use? Does any of this matter? And the million dollar question: why so many acronyms?

What is IPSec?

IPSec is an old guard, de facto standard for securing data in transit between devices. It is not one particular algorithm or protocol for encrypting traffic, but rather a protocol suite to authenticate the traffic source and also to encrypt it.

It secures a connection typically either between two hosts or between two sites or LANs, but can also handle a connection from a host to a site. If you use a corporate VPN from a network hardware maker like Cisco or SonicWall, this is almost definitely the mode you are using. Your computer is connecting to a hardware firewall at the office.

The encryption is handled by an encryption algorithm using keys like 3DES or AES. Message integrity comes from hashing algorithms like MD5 or SHA. At the beginning of the connection between two devices, they first establish which algorithms they will use for encryption and authentication, along with securely exchanging the cryptographic keys using an algorithm like Diffie-Helman.

Authentication can happen either with certificates or pre-shared keys (a password basically). Certificates are considered more secure, as a pre-shared key can be much more easily compromised or stolen and it cannot be automatically revoked like a certificate can.

IPSec can operate in either of two modes: transport mode or tunnel mode. Transport mode is commonly used when the encrypted connection is between two hosts and those hosts are responsible for encrypting and decrypting the traffic.

Because the traffic must still pass through the rest of the network (switches, routers, etc.), those packet layers must be left unencrypted to correctly route the traffic, so only the payload portion of the IP packet (the OSI Layer 3) is encrypted.

In other words, imagine a letter written in code, mailed in an envelope with the address plainly written on the outside for the post office to use for delivery. The address is necessary to get the letter to its destination, and anyone could intercept the letter and see where it’s going, but the contents are safe — as long as the code is secure.

Data over an IPSec transport mode connection works the same way. The actual data is encrypted, but the designation and source information is still in plain text on the packet.

Tunnel mode, however, encrypts all the data passing between two gateways, which are usually hardware devices like firewalls. The gateways establish an encrypted communication tunnel between the two networks, handling the encryption and integrity checks. Your computer on one end or the other doesn’t know or care that their traffic went across a VPN as everything it sends or receives has already been encrypted or decrypted by the firewall.

Going back to the letter analogy, the entire letter is encrypted, including the address, but there’s only one destination for the post office to take it to. For the packet with both encrypted payload and destination, this works because there is no public routing of the traffic because the tunnel takes care of getting the traffic to the other network.

What is OpenVPN?

OpenVPN, like the name suggests, is an open-source VPN protocol. Similar to IPSec, it provides encryption and authentication of data on connections between two hosts, between two gateways, or from a host to a gateway.

Encryption and authentication are all provided by OpenSSL, also an open source software library, which is an implementation of the SSL and TLS protocols that can be used anywhere secure communications are needed. Authentication can also be handled by certificates or key-shared keys. OpenVPN supports the same encryption and hashing algorithms as IPSec.

OpenVPN doesn’t follow an RFC standard like IPSec does, however it has become an incredibly popular and trusted open source VPN solution.

As mentioned, it uses SSL and TLS, specifically for the key exchange, which simplifies setup and implementation greatly. As an open source project, it can be implemented into other free and commercial projects. Some notable examples of implementations include pfSense, DD-WRT, and Tomato.

Ready for a VPN Showdown?

So after reading a comparison between IPSec and OpenVPN, you might be thinking, “So they sound pretty similar, which do I actually use? Similar encryption and authentication, similar architectures, is either actually better?” While fundamentally there are many similarities, there are small but important differences. Let’s break those down in the ultimate VPN showdown: IPSec vs OpenVPN!

IPSec vs OpenVPN: Encryption

Both are pretty equal here, particularly as they both implement AES-256, which is considered the standard for unbreakable data encryption.

IPSec vs OpenVPN: Security 

Same here, both are considered very secure given a correct implementation using a secure algorithm for encryption and certificates for authentication. Neither have a history of known vulnerabilities.

IPSec vs OpenVPN: Performance

This is where the differences start to emerge. IPSec is generally regarded as faster than OpenVPN. The main reason for this is actually a pro for OpenVPN in another area, and that is how it is implemented. IPSec is implemented in the IP stack of the kernel, whereas OpenVPN is implemented in the userspace. Being closer to the kernel allows for faster encryption and decryption. On the other hand…

IPSec vs OpenVPN: Implementation

Being in the user space means that OpenVPN is much easier to implement and maintain across different OSes, as no IP stack changes unique to each OS are needed. If you’ve ever tried to use a commercial VPN client on a macOS machine and had a subpar experience compared to Windows, this is partly why those companies are putting more time and energy into developing for the OS (Windows) that more of their customers are using.

Also IPSec only operates on specific TCP and UDP ports; these cannot be changed and therefore firewalls restricting outbound connections on ports other than 80 and 443 (for HTTP and HTTPS) can be problematic. OpenVPN on the other hand can run on any port, and is commonly run on 443 for this reason.

IPSec vs OpenVPN: Stability

Speed is fine but stability is just as important. OpenVPN wins here with its TCP mode that allows for retries on a shaky connection like a mobile hotspot or weak wifi.

Final Thoughts

The differences between IPSec and OpenVPN are slim. IPSec is usually faster, but OpenVPN is easier to implement and more reliable. Given the choice between both,if you aren’t working on the road a lot, we think IPSec is the winner by a hair. The setup will be a bit more complex, but should result in a solid VPN that you don’t even mind connecting to.

Whichever method you choose, a VPN is important, and in some cases essential to working securely. The important choice may not be which protocol you use, but that you’re just using something, securing your connection, and keeping your data and LAN unexposed from bad guys. Learn more about security, privacy, and more by signing up with CBT Nuggets today.