Posts Tagged ‘Security’

Getting your tunnel on with EC2 and OpenVPN (or wrap it before you ride it)

Tuesday, November 16th, 2010

I worry about data. I worry about whether it’ll be lost, compromised, or corrupted. I worry whether or not I am doing enough: did I check the backups, are they “good”, did any fail to run, do I have a “plan”, will it work? It is easier for me to get a handle on my fretting with regards to the server I tend but what about those assets that get to see the wider world and networks other than my own?

As a company, we are traveling more and it isn’t unusual for people to work from airports, coffee shops, clients, taxi cabs, or bars. Its just how business is getting done. But what about that data I worry about so much, both company and personal? Things like Firesheep intrigue me at the same time as leave me with the feeling that I need to do more for the people on the road.

The most obvious solution is to implement a VPN but which one, how, and moreover could we run it in our EC2 stack. I experimented with PPTP and loved the ease and simplicity of setting it up and it worked great with the built in OS X VPN client. However, once I left the free love environment that is my home network it was all but useless as I encountered plenty of public WiFi that disallowed TCP/1723. Next up was Openswan’s IPSec implementation, a complex beast full of options and configurations. It enjoyed wide support on the public networks I tried but, sadly, I seemingly could not get the built in OS X VPN client to consistently play nice. Last up was OpenVPN.

OpenVPN is easy to setup, configure, and roll out. We use it on our production servers as part of the CohesiveFT VPN-Cubed product that allows us to easily throw up IPSec tunnels with clients and hook it back stack so we know it is rock solid. The only drawback that I encountered with OpenVPN is that neither the iPad nor the iPhone support it so this does limit the devices that we can secure at the moment but life is about compromise.

Eric Hammond wrote an excellent article as a proof-of-concept for running OpenVPN over TCP/80 on EC2. Per usual he made it dead simple and easy to implement and with the slightest bit of finessing it can be adapted for rolling out to small teams.

For this I chose a micro instance running Ubuntu 10.04 for both the typical miserly and familiarity reasons. Prepping I ran with Hammond’s opening tasks:

sudo apt-get update && sudo apt-get upgrade -y
sudo apt-get install openvpn -y
sudo modprobe iptable_nat
echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward
sudo iptables-save > /some/place/you/can/recall
sudo iptables -t nat -A POSTROUTING -s 10.4.0.1/2 -o eth0 -j MASQUERADE

Once that inital prep work was out of the way I hoped over to the official Ubuntu OpenVPN documentation where we can set up OpenVPN to use asymmetrical keys instead.

But first we need to make a couple of tweaks to the server config (/etc/openvpn/server.conf). Look for the following and adjust them:

port 80
proto tcp
server 10.4.0.0 255.255.255.0

The port and protocol is so that our tunnel will run over TCP/80 and this, in theory, let you toss up your tunnel on nearly any network that has access to the Internet. That said, I’ve tested this on my free love network, a tighter administered open WiFi where my daughter goes to school (this is the one that disallows TCP/1723), and my T-Mobile data plan and while it worked on all three your mileage may vary.

The next step is to create a CA for OpenVPN and generate keys for clients and the server. Rather than do that work in /etc/openvpn/ as suggested in the documentation I opted to perform all that work in another directory so we can easily manage it with tools like git or puppet.

sudo mkdir /mnt/easy-rsa/
sudo cp -r /usr/share/doc/openvpn/examples/easy-rsa/2.0/* /mnt/easy-rsa/
sudo chown -R $USER /mnt/easy-rsa/

Adjust the variables in /mnt/easy-rsa/vars to best fit how you want things to look:

export KEY_COUNTRY="US"
export KEY_PROVINCE="CT"
export KEY_CITY="Hartford"
export KEY_ORG="Awesome Pancake, LLP"
export KEY_EMAIL="muffin.man@awepancake.com"

Then, because the devs and maintainers are awesome, just run some handy scripts and you’ll be all set:

cd /mnt/easy-rsa/
source vars
./clean-all
./build-dh
./pkitool --initca
./pkitool --server server
cd keys
openvpn --genkey --secret ta.key
sudo cp server.crt server.key ca.crt dh1024.pem ta.key /etc/openvpn/

Now we just need to generate some client certs:

cd /mnt/easy-rsa/
source vars
./pkitool jimmy

Replace “jimmy” with whatever you want to name your client, I recommend easy to remember nicknames that will cause you to snicker when you use them.

Now for the clients, it all depends on the what you’ll be using to connect to the server with. For testing I tried Tunnelblick and while it’s FLOSS I found it to be a little harder to configure than Viscosity, which is an affordable commercial client. The additional benefit to Viscosity is that it will allow you create easy to distribute connection bundles that include the CA cert, client key, and client cert that you can distribute to your clients for importing.

Setting up Viscosity is about as easy as it gets, you’ll need the CA.crt, and the host key and cert that you created above. Since pictures are a more valuable currency:

Once that is all set, connect to your tunnel and marvel at your secure connection. To check how things look on the client you can check ifconfig to ensure that the tunnel was added:

Also, do a quick nslookup to ensure that DNS is going to EC2 rather than to the local network:

Viscosity as a nifty detail page for visualizing the traffic over the tunnel:

On the server you can use tcpdump to watch the traffic and make sure that it is flowing like you want:

sudo tcpdump -i tun0 | grep "ip-10-4-0-6.ec2.internal"

Where 10-4-0-6 is the same as the ip address given by the tunnel.

That’s pretty much it, very easy thanks to Eric Hammond and the Ubuntu documentation team. From here you could set up connections in Viscosity for each of your users, export a zipped connection and distribute it (securely, of course!).

The next big test will be setting up all our road warriors and asking them to try it while they are traveling and report back with concerns, problems, or suggestions because in the end I want it to be easy as possible as they’ll be more inclined to use it.

Springtime Hack

Sunday, March 9th, 2008

I could easily mark this as the worst morning in as far back as I can remember. Without the first cup of coffee I sat down to scan our servers like I do everyday, just looking for anything out of the ordinary, like services that failed to run. For the most part it is a ten minute job that rarely varies day to day. This morning was an exception.

Nearly every nightly job failed. Worse than that there was an hour and ten minute hole in the logs, 0155 to 0305 was completely unaccounted. I scanned every log from authentication to our application logs and every single one of them showed this hole but checking our external monitoring service showed that we had zero downtime. What the hell happened?

A cold hand of desperation and fear gripped my stomach leaving me dizzy. I ran chkrootkit but came up clean so I mentally prepared myself to rebuild the server and possibly be eviscerated by my bosses. How would I explain this? How could I protect us from it happening again, that is if I still have my job?

Sitting helpless I realized, “Spring Ahead”.

(Worthless Sysadmin + Microsoft)^ Crap Police Work = Jail Time

Thursday, January 11th, 2007

The Norwich Bulletin reports that a substitute teacher has been found guilty in Norwich porn case and with her sentencing looming in March she could face upwards of 40 years in jail but there is a twist to the story.

When the facts of the case are laid out it looks to be a blizzard of incompetence ranging from the school’s IT staff, shoddy forensic police work, questionable judicial proceedings, and a the ribbon that ties it all together is one of Redmond’s flagship products. SunbeltBLOG worries about the notion of doing actual hard time for a spyware infection. A frightening though indeed.

Reading the articles I find myself question the capabilities of the police investigators and their apparent lack of understanding of how spyware operates. From another Norwich Bulletin article:

Norwich Police Det. Mark Lounsbury, who investigates computer crimes, said there was evidence that someone had directly accessed several sexually-oriented sites by clicking on a link.

Ok. Prove that to me. Do you have photographic or video evidence? Because guess what Det. Lounsbury, popup generators are designed to act like a human click-through and this is not to mention that there are no means to differentiate between a bot click or a human click. Now you might say, “But she had to install the software, therefore she must have visited those sites!” Wrong again. Nearly all Microsoft OSes log in as root by default and one of the charming aspects of this is that it allows for the surreptitious installation of software. In other words using Internet Explorer as root will allow sites to install software without your knowledge or consent, wonderful technologies like ActiveX facilitate this activity. The defense’s examination of the facts pointed to a hairstyling website as the source of the infection.

All this begs the question of what the lazy ass Norwich school IT staff was up to before, during, and after this incident. If I were Amero I would be giving serious consideration to a civil suit against the town and possibly the state. It is their responsibility to secure the PCs and the network, not the teachers, and you can scream all you want that she should have unplugged the PC but the fact still remains that shitty IT staff plus a crap OS is a disaster waiting to happen. Should she do time for the incompetence of others?

My $0.02 in closing: Don’t trust the cops and get yourself a secure OS.

eWeek asks “Is the Botnet Battle Already Lost?”

Saturday, October 21st, 2006

You might but I never get tired of this rant…

In a slightly sensational article, Ryan Naraine tackles the issue of botnets with regards to the what, how, and why and while he takes the approach that ISPs need to tackle the issue head on the real reason botnets are an issue is tucked away in his closing paragraphs.

…the large percentage of computer users running Windows versions without up-to-date patches, creates an environment that’s ripe for abuse.

B-I-N-G-O, Ryan. That is the problem, not the ISPs and not the security firms. The fact that the most popular OS is easy to exploit finds itself coupled with the average user who knows little to nothing about basic security practices is a recipe for disaster. the onus falls on the shoulders of Microsoft for these problems, for allowing its OSes to run as root, and worse for allowing its partners to write software that requires the user to run as root is a grievous transgression.

In all honesty, I cannot fully expect that my mother-in-law with limited computing experience can fully understand more than the basic computer safety practices, it is akin to expecting me to understand all of the physics and engineering involved in flying every time I travel. As I rely on the pilots and mechanics to ensure my safety as much as she relies on her software providers to ensure that she sold a safe product. She has enough presence of mind not to click on “security alert” pop ups and she knows the importance of staying patched and keeping anti-virus up-to-date, all of which goes a long way in securing the PC, but she does not how to troubleshoot spyware and malware issues and any amount of time spent online running as root will ensure that you will encounter infections of some sort. If you think otherwise you are either a liar or deluded.

So what is the average user to do? Not much until the market shifts towards OSes built with security first and ease of use second or until the government steps in and requires that Microsoft adhere to good security practices much like they require other manufactures not to sell a dangerous product. Like cars, imagine if they had recalls on OSes? Class action suits? However, it is unlikely as there is no real tangible cost to the user beyond frustration with a sluggish computer but that irritation bleeds away as they grow used to it. “This is just how my computer runs,” is the oft heard excuse and the average user does not know any better. No matter how you might explain to them why they should choose a different OS purchases are made with dollars in mind and with what they are most familiar with so the weakest OS will win for today and I’m going to continue advocating that people use a different system each and everytime they ask me.

Google Code Will Snarf Your Backups!

Friday, October 6th, 2006

Some of your db passwords are belong to us,” reports Death By Comet.  The issue at hand is that Google Code will expose passwords stored in config files, such as wp-config.php and mt-db-pass.cgi, potentially resulting in headaches for site administrators.  Thankfully, the solution is just plain common sense: Don’t store your backups on your production server.  File that under: Sherlock, No Shit.  Anyway, don’t get caught with your pants around your ankles and move your backups to a secure location.