Hot damn!
Tag Archive for 'Web-Services'
So this morning whilst perusing my feeds I came across a post about being more of a bookworm than I already am. Unfortunately, there were no tips about how to extend the day by another couple of hours so I have more time to read in my already cramped life but there was a little gem tucked inside: BookMooch. BookMooch is like what LaLa was to CDs in that you create an inventory of stuff you want to unload and a wishlist of what you’d like to get in return. Here’s my inventory and it is also in my side bar showing six random items.
Getting started was pretty easy requiring you to choose username and password and to make your postal address available so that people you want to mooch from can ship to you. Adding books to your inventory is as easy as banging in the ISBN number, books without one can be easily looked up by author and title, and failing that you can always hand enter the item in. I knocked out some 49 books in under an hour and had three people looking to relive me of four in a couple more hours. Not too bad.
The system relies on a sort of karma system in that each book added to your inventory nets you 1/10 of a point, shipping one domestically nets you 1 point and 3 for international shipments. You’ll need those points when you want to get books for yourself with a domestic mooch being -1 points and -2 internationally. Reading Kafka at Work lays out the system better than I.
So far the system beats trying to set up an auction on eBay and feels a little more social than just dumping them in a box and posting it on Freecycle. I am particularly impressed with how easy it is to use and the first three people mooching off me are awfully polite people who are just about as book-addled as myself. I’m looking forward to figuring out what I would like to mooch and see how that end of the system operates.
With EC2 and S3 Amazon has made available some seriously powerful and flexible tools for server and file hosting. Ec2 allows you to roll whatever flavor OS you want and get it up and running–think of it as virtual dedicated hosting–which while being incredibly cool has one major downside: if your server instance fails you loose your data as it will revert to the most current instance when you get it back online. In other words, a hiccup on EC2 could turn into a blistering nightmare.
This is where S3 comes in handy for storing anything that might change after you create and instance and launch it such as databases, files uploaded or created, and even logs. One of the first projects I hopped into was dumping the MySQL databases, compressing them and tossing them up on S3, a tedious process that can be automated with cron and s3sync (a ruby based tool that approximates rsync, kinda sorta).
Daily cron job to create new bucket
cd /root/s3sync/
DAYNOW=$(date +%j)
ruby s3cmd.rb createbucket WTF_db_$DAYNOW
Daily cron job to delete old bucket
cd /root/s3sync/
DAYTHEN=$(date +%j --date='2 days ago')
ruby s3cmd.rb deletebucket WTF_db_$DAYTHEN
Hourly job to back up the database
# get into the directory
cd /root/s3sync/
# set the environment
export AWS_ACCESS_KEY_ID=XXXX
export AWS_SECRET_ACCESS_KEY=XXXX/XXXX
export SSL_CERT_DIR=/root/s3sync/certs
# set date variables
DAYNOW=$(date +%j)
TIMENOW=$(date +%H%M)
# dump database
mysqldump WTF > /tmp/WTF-$DAYNOW-$TIMENOW.sql
# tar SQL dump
tar -czvf /tmp/backups/WTF-$STAMPD-$STAMPT.tar.gz /tmp/WTF-$DAYNOW-$TIMENOW.sql
# copy tar to S3
ruby s3sync.rb -r --ssl /tmp/backups/ MahBukit:WTF_db_$DAYNOW
# clean up after yourself
rm /tmp/*.sql
rm /tmp/backups/*.gz
Now one of the frustrations I have with this set up is while I am dropping the buckets from 48 hours prior it isn’t actually deleting the files off S3, a bit of a pain in the ass that I need to find some sort of resolution for in the near future. If anyone has an answer to this problem I would love to hear it! For now, I’m cheating by using S3Fox to purge those pesky backups.
