Thursday, December 29, 2011

Paperclip, Heroku, and AWS-S3

I recently had a requirement to build an “avatar for users” feature.  The application is a Rails 3 app running on Heroku.  Easy enough.  I chose to use the Paperclip gem provided by ThoughtBot.  Paperclip’s default is to save uploaded files to the filesystem. The catch, Heroku doesn’t let you write to the filesystem, and I couldn’t use Gravatar.  Well, there are a couple of ways to work around this Heroku limitation.  One is to save the file to the database as a blob.  There are pros and cons to this approach.  The other approach, and the one I chose, was to use Amazon’s AWS-S3 service.

Setting up Paperclip

Setting up paperclip is pretty easy.  There are various ways to do it, but I’m going to discuss how I did it.

First I added the paperclip and aws-s3 gems to my project’s Gemfile.

Then run bundle to install the gems.

Paperclip stores metadata about the uploaded files into the database, and therefore needs fields added to a table to do so.  I added those fields right along side the fields in the user table.  I didn’t have the need to track past history of upload avatar images.  So this was an acceptable approach.  Installing the paperclip gem gave us a generator to help us create the database fields we need.  Run the following command in a terminal window.

$ rails g paperclip user avatar

Lets quickly break this command down.

  • rails g is a short cut for rails generate.  You should know this.
  • paperclip is the name of the generator we want to run
  • user is the table we are going to add the fields to.  For me this table already existed.
  • avatar is the prefix name we want to use for the database fields

This will create a migration file that adds the necessary fields to the user table, or what ever table name you supplied the generator.

Now you can run this migration.

$ rake db:migrate

The last thing needed to set up the paperclip gem is to add a little bit of code to the model that is going to be responsible for the attachment.  In my case, this is the user model.  The little bit of code to add is the has_attachment_file method.  As you can see from the gist below the code is pretty simple, and maybe even a little self explanatory.  In traditional Ruby fashion, this gives you a bunch of extra functionality that you can take advantage of in your views.  Not going to cover that here though.

Take note of a few important items in the code.

  • storage tells paperclip where to save the uploaded file.  Default is the file system.  Here I’ve specified it as s3.
  • bucket is the name of the location in the AWS-S3 service that the file will be saved to.  Think of it as a folder in the cloud.
  • s3_credentials are used to authenticate to the AWS services.  There are a couple of ways to set the credentials.  I used a hash and set the access key and secret key individually for reasons that we’ll cover in a bit.  You can also use a Yaml file here as well.  See paperclip documentation for that.

And don’t forget to whitelist the avatar property for mass assignment by adding it to the attr_accessible list.  See the gist above.

Create an AWS Account

You are going to obviously need an AWS account.  I’m not going to go through all the steps to create an AWS account, but if you made it this far you should be pretty committed to going through the process.  It’s a pretty cheap service to use, and you only pay for what you use.  New users get the first year of S3 use for free.  To date, I’ve spent $0.14 for it.  Not a type-o, yes 14 cents.  Here is the link to the AWS-S3 service.  When you sign up for an AWS account you will be assigned an access key and a secret access key.  I try and keep both private.  The more important one is the secret access key.  You can change this if needed, or even add more keys if that is a requirement.  You can find your keys by going into the Security Credentials screen.

security_credentials

Once you find them, keep them handy, you’re going to need them in a bit.

Shell scripts – Local and Heroku

This is where things got a little different for me.  Since this is an open source project on GitHub, I didn’t want to have any reference to my AWS keys any where in my source code.  Hence the reason for using ENV variables that we saw earlier.

Local Development

In Rails applications you can set ENV variables at start up and have them exist for the duration of the application’s life.  As an example.

$ S3_KEY=123456789ABCDEF S3_SECRET=x+yabld235lll654 rails s

Now two environment variables exist with the names S3_KEY and S3_SECRET.  If you review the user model gist above you’ll see how you access these two values.  You probably see right away typing this at the command line is error prone and very tedious every time you want to start the server.  Sure there’s the bash history.  But, I chose to make a simple shell script to start the server and set these variables.

Important: First add the *.sh file extension to your .gitignore file.  This will ensure you don’t accidently check your AWS keys in and upload them to GitHub.

Next in the root of your application add the following command line, replacing the appropriate values with your AWS keys, to a file with the .sh file extension.  I used run.sh.

S3_KEY=<your access key> S3_SECRET=<your secret access key> rails s

Save and then set this file as executable.

$ chmod +x run.sh

Test it.  If all works, your application will start up as usual.  I like this cause it doesn’t really interrupt my workflow.

$ ./run.sh

Heroku Production

The process is very similar as it was for the local development section above.  For brevity I’m going to list the steps out instead of going into detail.

Create a .sh file in your application root.  I used deploy_heroku.sh.

Add the following lines to that file.

git push heroku master
heroku rake db:migrate
heroku config:add S3_KEY=<your access key>
heroku config:add S3_SECRET=<your secret access key>

Make the file executable

$ chmod +x deploy_heroku.sh

Test it.

$ ./deploy_heroku.sh && heroku open

I loaded this script up with a few more things.  I got tired of typing each step out every time I wanted to deploy to Heroku.  You are free to change these steps any way you want.  The important two pieces are where the config settings are getting set.

Conclusion

I hope you found this helpful and that it all makes sense.  Please leave a comment below if you like this, have a suggestion to improvement, or if you have any questions.

References

http://devcenter.heroku.com/articles/s3
http://www.labnol.org/internet/tools/amazon-s3-buckets-tutorial/3890/
http://www.railstoolkit.com/posts/fancyupload-amazon-s3-uploader-with-paperclip

Saturday, August 27, 2011

A new chapter

Next week Wednesday Aug 31st I start a new chapter in my computer career.  I’m going back to school.  Well, kind of.  I’m taking a Ruby on Rails class.  It’s a full semester 3 credit course, but I’m only taking it as a continuing education program.  The class itself is an elective for the college’s computer science program.  In order for me to take it I had to get an instructor authorization for enrollment.  But no biggy there.  I’m very excited for this.  The Ruby on Rails community has always intrigued me.  And there are .NET and Java developers moving to Ruby on Rails in droves.  Over the past year or so I’ve dabbled with Ruby on Rails from time to time.  But I never felt confident enough with it to take on a big project.  It is my hope that this class will get me to a more comfortable level.

Last weekend I attended the first ever Madison Ruby conference.  It was AWESOME!!!  Very enlightening being around that crowd.  I met some great people and discovered some new opportunities that I can get involved with.  For example the Mad-Railers group that meets here in town.  The Mad-Railers are a Ruby on Rails user group.  They meet the last Monday of each month.  Very excited for that as well.

It is my intention to do some more blogging over the next few months.  Mostly with my experiences as a .NET developer moving towards Ruby on Rails.  There is a mind shift that has to take place.

Friday, July 1, 2011

Performance issues with Visual Studio 2010

I’ve been noticing some performance issues with Visual Studio 2010.  Specifically when I type.  The editor wasn’t able to keep up.  And I’m not a super fast typer, 55 wpm.  I have a few plugins installed, mainly ones you can get via the Extension Manager along with DevExpress’ Code Rush Pro.  I thought maybe it was the accumulation of all the plugins trying to do their thing behind the scenes.  I’ve tried disabling or uninstalling plugins, but that didn’t really help.  Plus once you get use to using the added features that the plugins provide, you start to feel crippled after they are gone.  So I searched around a bit, and found that the culprit was, “My video card”????  Turns out its an easy fix.  Tools –> Options.  I unchecked the three check boxes from below.  Restarted Visual Studio, and that fixed it.

image

Friday, June 10, 2011

Caliburn.Micro Series

I’m working on a MVC3 personal project that exposes some services.  The reason for the services is because I see a need for a desktop (WPF) client as well.  A few months ago I was introduced to the Caliburn.Micro project.  I don’t know much about it, but from that discussion I got pretty interested in it.  As I was Googling around for some good documentation, tutorials, or code examples I ran into a Soup to Nuts series on Devlicio.us.  Below is a TOC of the series.  I put this together because over the next few months I’ll be using this as my reading guide.  I also noticed there is no part 4.  Not sure whats up with that.  I’ll update this post if I find out more.

  1. MVVM Study – Segue - Introducing Caliburn.Micro
  2. Caliburn.Micro Soup to Nuts Pt. 1 – Configuration, Actions and Conventions
  3. Caliburn.Micro Soup to Nuts Pt. 2 – Customizing The Bootstrapper
  4. Caliburn.Micro Soup to Nuts Pt. 3 – All About Actions
  5. Caliburn.Micro Soup to Nuts Pt. 5 – IResult and Coroutines
  6. Caliburn.Micro Soup to Nuts Part 6a – Screens, Conductors and Composition
  7. Caliburn.Micro Soup to Nuts Part 6b – Simple Navigation with Conductors
  8. Caliburn.Micro Soup to Nuts Part 6c – Simple MDI with Screen Collections
  9. Caliburn.Micro Soup to Nuts Part 6d – A “Billy Hollis” Hybrid Shell
  10. Caliburn.Micro Soup to Nuts Part 7 - All About Conventions
  11. Caliburn.Micro Soup to Nuts Part 8–The EventAggregator
  12. Caliburn.Micro Soup to Nuts Part 9–New WP7 Features

Wednesday, June 8, 2011

CQRS Presentation video

I have completed the editing and uploading of my CQRS presentation from 6-1-2011. It took me some time to edit the video, and get it uploaded, but I think the result is worth the effort. I have also included the Power Point slides for you to download. I hope you all like it, and any comments are welcome. As I watched the video back, I noticed a few places where I miss spoke. I'm going to do some follow up articles to clarify the instances where I miss spoke, and also to answer some of the questions that came out of the meeting. I believe there were some good questions that came out. I've been thinking about them and I've come up with some answers to a few of them. So be on the look out for those follow up posts.

Thanks again to all that attended.

Tuesday, April 26, 2011

Free .NET Libraries

There is a great list of free .net libraries posted at Qink that you can use in your applications. Check it out. The list is continually being updated from the comments.

Monday, August 9, 2010

Current Packages for the Nu Package Management

I’ve been doing Rub on Rails now for about 3 months, and I’m really getting into it.  So many nice libraries and conventions to help get applications out the door faster.  Unfortunately my RoR work doesn’t pay the bills yet.  By day I’m a .NET developer.  Recently I came across a blog post written by Ayende Rahien that mentions the use of a new package management utility for .NET developers.  For years Ruby on Rails developers have enjoyed the luxury of installing libraries using the gem utility.  Its really handy to have at your disposal.  Read up on it.  There is now a similar package management utility available for .NET developers called Nu, short for Nubular.  If you read Ayende Rahien’s blog you’ll see how it works.

Current Nu packages are available here

I hope this project catches on and the list continues to grow.  This is going to make things really handy and really quick to get the latest versions of the most common libraries that I use for development.  Best of all now I don’t have to do the package management myself on some file share of mine.