Pete Hamilton

Scaling Heroku by Time of Day

2014-05-08T15:00:00+01:00

Heroku is brilliant. However, I manage a rails application which has grown much larger recently. It’s a bespoke company-specific application so it’s only used during working hours. There are therefore long periods of time (at night, over the weekend) where I’m paying for dynos which simply aren’t going to get used.

All dynos in your application that are scaled above 0 will accrue usage–regardless of whether they’re actually receiving or processing requests

Adept Scale offers a solution which automatically scales but I know when my load is high and low, also it’s fairly consistent within working hours.

For fun, I decided to see if I could scale my heroku dynos by time period. Turns out it’s really easy.

Note: I use heroku’s scheduler combined with rake tasks below, but this could easily be done with scheduled jobs. If you have any trouble, comment and I’ll do my best to help you out.

Add the heroku-api gem

gem 'heroku-api'

Decide on limits and set them in your Heroku config

For a simple small application you might have something like:

heroku config:set HEROKU_WEB_HIGH=5
heroku config:set HEROKU_WEB_LOW=1
heroku config:set HEROKU_WORKER_HIGH=3
heroku config:set HEROKU_WORKER_LOW=1

Add rake tasks for scaling up/down

I also add logic to prevent scaling up if it’s the weekend

task :scale_up => :environment do
  return if Date.today.saturday? || Date.today.sunday?

  heroku = Heroku::API.new
  heroku.post_ps_scale(YOUR_APP_NAME, 'web', ENV['HEROKU_WEB_HIGH'])
  heroku.post_ps_scale(YOUR_APP_NAME, 'worker', ENV['HEROKU_WORKER_HIGH'])
end

task :scale_down => :environment do
  heroku = Heroku::API.new
  heroku.post_ps_scale(YOUR_APP_NAME, 'web', ENV['HEROKU_WEB_LOW'])
  heroku.post_ps_scale(YOUR_APP_NAME, 'worker', ENV['HEROKU_WORKER_LOW'])
end

Deploy

git push heroku master

Add the Heroku Scheduler addon

heroku addons:add scheduler

Add Heroku Scheduler Tasks

Visit the heroku scheduler dashboard and add your rake tasks.

Add rake scale_up in the morning e.g. 8:00 am
Add rake scale_up in the evening e.g. 7:00 pm

Extensions

You could add gems like holidays to run in your rake tasks and help prevent scaling up on public holidays as well as just the weekends.

You could also add more scheduled tasks throughout the day, or just have one task which runs more frequently and performs the logic of scaling up/down within it. It’s up to you!

What could we save?

Assume the app is only used 8:00am - 7:00pm
Assume high = 20 paid web dynos, 10 paid worker dynos.
Assume low = 0 paid web dynos, 0 paid worker dynos (one free web dyno)
In the UK, there are 253 working days in 2014
Dynos are $0.05/h, prorated to the second

Before: $0.05 * 30 dynos * 24 hours * 365 days = $13140
After: $0.05 * 30 dynos * 13 hours * 253 days = $4993.50
Savings: ~ $8000 a year!

Per Dyno Time: 8760 vs 3298 hours per year. Quite a difference.

In fact, I could more than double my number of operating dynos using this method and still save money.

Disclaimer

There are a number of cases where obviously this simply isn’t logical, for example if your application is used in multiple timezones or needs to handle unexpected traffic spikes. However if you know your requirements and they match mine, this could be pretty helpful at helping you keep your application usable without breaking the bank.

Skype Phone+Voicemail Sucks

2014-03-19T12:30:00+00:00

I was recently helping someone set up a business phone number such that they could make and receive calls from their mobile and not be limited to (or have to pay for) a physical landline.

I chose Skype, which I’m not sure was a great idea any more. I was originally intending to build something with Twilio but exams are coming up so I’ve banned all side projects for now. I just needed something simple.

I got a phone number set up on their account no problem and a quick check-list of the simplest requirements then went like this:

Call number when signed in - goes through to various devices fine.
Call number when offline - calls go to Skype voicemail
Call number but decline call - Call gets a busy signal till you hang up
Call number and leave the call - Call rings out until after a few minutes you get a busy signal till you hang up

Those last two are ridiculous. Anyone calling a normal phone number expects to either:

Be answered
Go to voicemail immediately (if person is engaged or can’t answer)
Go to voicemail after a while (if person isn’t around)

How Skype expects anyone to take their offering seriously without those last two is a mystery to me. After some digging however, I did manage to get some way to solving point 2 above.

In the desktop app (I’m on a Mac), in Skype’s preferences:

Select the Calls panel
Select Send to voice messages
Click Set up voice messaging
Tick Send declined calls to voice messages

On mobile it doesn’t seem you can enable this option. Why Skype, WHY?

Also, if you reject a call on your desktop while your mobile is signed in, it won’t send the call to voicemail but continue ringing on your phone, if you then reject it there, it goes straight to the busy tone as before.

I’m not the only one who’s discovered this.

If you have a better alternative to Skype for a simple UK business phone number setup, I’d love to hear it in the comments below.

ICHACK '14 - Hackathon Survival Tips

2014-03-07T16:00:00+00:00

What with ICHACK ‘14 coming up this weekend, I did a short presentation to the attendees with some pre-event hints and tips to help them hit the ground running.

O2 Blocking Childline, NSPCC under 'Parental Control' Policy?

2013-12-21T20:30:00+00:00

After playing with O2’s URL checker someone pointed out earlier that they seem to be blocking sites like Childline and the NSPCC under the ‘Parental Control’ filter. I’ve just checked again and it’s still the case.

I (and no doubt many others) think this is pretty awful. Charities dedicated to helping children who might be experiencing abuse are being blocked from those children under the ‘parental control’ policy. What?

@O2 it appears you block @nspcc and @childline for children under “Parental Control”. http://t.co/JSlh2MWG6o what!? pic.twitter.com/cCa9AwbYVg
— Peter Hamilton (@peterejhamilton) December 21, 2013

I understand these things sometimes slip through unchecked and I may even be completely misunderstanding what the URL checker results really mean, but on the off chance that someone somewhere has messed up and children are being prevented access to sites which are there to help them, I hope this post might help get the sites unblocked.

It seems that you can suggest reclassification of a site so if anyone sees this before O2 update their parental control filters I suggest you put a request in. To do so:

Visit O2’s URL checker
Enter the charity site(s)
Click continue
At the bottom where it says “If you think this website needs a different policy, click here” click continue

Oddly it seems that GOV.UK is also blocked along with pretty much anything else I search for. I’m wondering what logic has been applied to this parental control filter and whether it’s an over zealous list or maybe just blocks everything unless the parent unblocks it? Even if that’s the case, I find the idea that a parent could even opt-in to have an abuse website blocked just as terrible as the fact that O2 seem to have blocked the sites by default. In my personal opinion such sites should be supported fully and be immune from being blocked, even on an opt-in basis.

It seems O2 aren’t the only ones. Earlier this week BT took some flack from the BBC for similar reasons, specifically regarding for their guidelines on porn and sexuality, according to the original BBC article, they were blocking sites including Sexual Health Scotland, Doncaster Domestic Abuse Helpline, Reducing The Risk (a site which tackles domestic abuse) and Edinburgh Women’s Rape and Sexual Abuse Centre.

These new filters which a lot of major UK ISPs are implementing in the coming months/year seem well meaning and will undoubtedly have a few teething problems, but they need to either do a better job of classifying sites in the first place, or have a very clear and easy mechanism by which people can help re-classify wrongly blocked sites. If they don’t I can only see this getting worse.

Society Newsletters powered by Node.js

2013-11-07T10:00:00+00:00

Last year when I was secretary of Imperial College’s Computing Society or DoCSoc) we decided to clean up the amount of email we sent to students, compiling everything into a weekly newsletter. The mailing list is internal to university, so sending a pretty email using something like Mailchimp or Campaign Monitor is basically impossible.

At the time, I created a quick digest-email rubygem which took a YAML file containing all the news items, a basic html template, css file and process it all to produce html for the email. It was a pretty neat project and saved me a tonne of time. People contributed their YAML snippets and I just pressed “GO”.

This year, I wanted to continue using this newsletter style of communication. However I wanted to try something new technology-wise, so I’ve created a similar tool but using completely different technologies.

All email content is done in Markdown, your templating is done in Jade, your CSS in Stylus and NodeJS then pulls it all together, pushes it through Juice to inline all the CSS for email compatibility and outputs it to an HTML file.

You can see more on the project here

The obvious advantage is that people can design templates and CSS as they would normal webpages, albeit with a few caveats for HTML emails

Whilst this is similar to the ruby version, this project does a few nifty things like extracting primary headers automatically and using them to create an automatic agenda at the top of the newsletter.

This was just for fun, but who knows, maybe someone else will find it useful. From my point of view, it was just fun to play with NodeJS for the first time properly. If I’ve done something boneheaded and evidently just don’t understand javascript, please do comment below ;)

Stubborn Jekyll Deployments

2013-10-25T10:00:00+01:00

Update:

Seems it may have been a problem with Filey-Diff, I just did a bundle update (can’t work out how I did) and it bumped the version significantly.

Old grumpy post below for posterity…

I use the s3_website gem to deploy this blog.

However, for some reason, the occasional file (not the same one every time I don’t think) causes either the filey-diff or AWS gem to bomb out with the below:

Deploying _site/* to pete-hamilton.co.uk
/Users/MY_USER/.rbenv/versions/1.9.3-p374/lib/ruby/1.9.1/openssl/buffering.rb:174:in `sysread_nonblock': end of file reached (EOFError)
  from /Users/MY_USER/.rbenv/versions/1.9.3-p374/lib/ruby/1.9.1/openssl/buffering.rb:174:in `read_nonblock'
  from /Users/MY_USER/.rbenv/versions/1.9.3-p374/lib/ruby/1.9.1/net/protocol.rb:141:in `rbuf_fill'

  ...

  etc

  ...

  from /Users/MY_USER/Projects/blog/.bundle/gems/aws-sdk-1.8.5/lib/aws/s3/s3_object.rb:313:in `last_modified'
  from /Users/MY_USER/Projects/blog/.bundle/gems/filey-diff-1.2.1/lib/filey-diff/data-sources/aws_sdk_s3.rb:61:in `last_modified_and_md5'
  from /Users/MY_USER/Projects/blog/.bundle/gems/filey-diff-1.2.1/lib/filey-diff/data-sources/aws_sdk_s3.rb:48:in `map_s3_object_to_filey'
  from /Users/MY_USER/Projects/blog/.bundle/gems/filey-diff-1.2.1/lib/filey-diff/data-sources/aws_sdk_s3.rb:13:in `block in do_internal_load'
  from /Users/MY_USER/Projects/blog/.bundle/gems/filey-diff-1.2.1/lib/filey-diff/data-sources/aws_sdk_s3.rb:27:in `call'
  from /Users/MY_USER/Projects/blog/.bundle/gems/filey-diff-1.2.1/lib/filey-diff/data-sources/aws_sdk_s3.rb:27:in `block (2 levels) in in_parallel_or_sequentially'

I did look into it, but couldn’t see any good reason for the bomb-out. I reinstalled all gems, tried specifying versions manually, scoured google and stack overflow but to no avail.

What I did in the end was trade beauty for beligerence. I wrote a simple deploy.sh file which just does the following:

until s3_website push; do :; done

I’d say I’m sorry for being such an awful human being, but I’m not. Sometimes getting the job done is better than getting the job done perfectly…

If you have had this and worked out how to fix it, I’ll award you all my internet points, please leave a comment below!

London Ruby User Group Talk - Hutch

2013-08-14T10:00:00+01:00

I recently presented a talk on a new gem we’ve written at GoCardless called Hutch which allows you to do really easy inter-service communication with RabbitMQ. You can read the official GoCardless blog post by our own Harry Marr here

Well, I did a talk on it for LRUG (London Ruby User Group). You can see the video of the talk here on skillsmatter or view the presentation below.

It seemed to go down reasonably well for a first tech talk. Now to manage it without quite so many “Umm”s.

For a first tech talk on a complex subject, @peterejhamilton just knocked it out of the park - very clear @RabbitMQ intro #lrug
— Andy Piper (@andypiper) August 12, 2013

AngularJS + Rails with Bower

2013-07-13T10:00:00+01:00

I really don’t like committing library code to my repos.

For a project I’m hacking around on this weekend I’m using AngularJS, Normalize.css and a bunch of other front end JS and CSS libraries. Bower seemed perfect to organise all of this but there wasn’t too much information out there on how to combine it with rails.

So, here’s my quick guide to getting set up with front end vendor assets in rails with Bower.

1. You’ll need to have bower installed:

sudo npm install -g bower

2. Tell Bower where to put the components

In the root of your project, create a .bowerrc file to tell bower where to install the components.

{
  "directory": "vendor/assets/components"
}

3. Tell Bower which components to install

Also create a bower.json file to tell bower which components you’d like to install. See the full registry list here

This was mine:

// bower.json

{
  "dependencies": {
    "angular-activerecord": "*",
    "angular-all-unstable": "*",
    "normalize-css": "*"
  }
}

4. Tell Rails to include components in the assets pipeline:

Add vendor assets components to the asset pipeline

# config/application.rb

# Add Bower components to assets pipeline
config.assets.paths << Rails.root.join('vendor', 'assets', 'components')

5. Add your libraries to `app/assets/application.js`

Assets are included by package/jsfile. So, for example, to include angular and angular-resource in my rails app, I added:

// app/assets/javascripts/application.js

//= require angular-all-unstable/angular
//= require angular-all-unstable/angular-resource

See asset pipeline docs for more details on how/why this works.

6. Ignore components in source control

Finally, we don’t want to commit any of these libraries to our repository so in your .gitignore file (or equivalent if another vcs) add:

# Ignore Bower Assets
/vendor/assets/components

And that’s it, super simple, but worth remembering next time you’re tempted to paste JS or CSS libraries into your project

References:

Rails Guide - Asset Pipeline
Aidan Feldman’s guide to rails, bower and heroku - really useful, came across late in writing this post but definitely worth a look, very similar approach + includes guide for running on Heroku

Gmail ads looking a lot more like emails...

2013-07-10T10:00:00+01:00

I’m sure I’m not the first one to find this out, but someone pointed out this morning that if you’re using new category-based email, Google has inserted an ad at the top of the “Promotion” tab emails which is now looking more similar to a regular email than before. You can even bookmark/star the promo/ad if you want.

For comparison, this is usually how ads appear:

It’s currently only on the promotions tab and from an advertising perspective, it’s probably pretty smart. After all, I’m there for recent emails relating to deals and promotional material, why not use that knowledge to their advantage?

That said, with ads looking more and more like my normal emails, for me it’s approaching the grey area where smart advertising becomes trickery. If I’m clicking on an ad because it’s relevant to me, fine. If I click on something I instincitvely believe to be an email but then turns out to be an ad, that’s pretty uncool.

Is it just me? What does everyone else think about this? Is it even new?

Update: Seems I wasn’t the first to find this out, Martin Brinkman also writes about it and there’s a post on the GoogleSystem blog about precisely this, with similar viewpoints expressed to those in the comments below and on HackerNews. Overall it seems putting it in “Promotions” isn’t that invasive and people don’t really mind.

Notice Anything Different?

2013-06-23T19:30:00+01:00

I’ve redone my blog to have a cleaner look. It’s a slightly tweaked version of Fabien Penso’s design.

I’ve also changed the hosting stack as well. I’m now using:

Plain Jekyll (no longer octopress) for the framework
Amazon S3 for hosting. Deploying with jekyll-s3
Amazon Route 53 for DNS (guide)
Amazon Cloudfront for a CDN
Gauges for analytics, something I’ve always wanted to try.

I wasn’t aiming for any performance improvements or anything but I did notice that page load times went from ~1500ms to ~800ms so it’s been essentially cut in half.

However I did this for fun. I’m not exactly being slammed traffic-wise :)