Sunday, August 2, 2009

Amazon HMAC with ColdFusion

I recently received a work request to update our calls that we were making against Amazon's Web Services with a new authentication signature. This proved to be more complicated that it should have been. I thought I would share my experience, in hope that it will help remove a headache for someone else out there.

The purpose was to "sign" your requests with a secret key so Amazon could verify that the request had not been altered in transit (these are requests that are transmitted via REST calls (http). There have been questions about why this was needed. The biggest point of abuse that I can think of comes from the use of these webservices from a client side (AJAX) - or doing a DOS against another competitor since you can only make so many requests per minute. Of course, now AJAX calls will need to be designed so the secret key never ever ever get exposed to the client side.

Here is the "documentation" Amazon provided. They gave this Important note to us:
"You have until August 15, 2009 to authenticate requests sent to the Product Advertising API. After August 15, 2009, messages that aren't authenticated will be denied."
What they failed to mention was their "testing" schedule (aka planned outages) which was sent (and forwarded to me just a few days ago).
Here is the full schedule:
"These planned outages will help our developers test their signed requests implementation and also discover applications and code paths that they may have missed, giving them the opportunity to address these gaps before signed requests become mandatory on August 15."
  • Monday, Aug 3 at 2PM (ET) through Tuesday, Aug 4 at 2PM, a few requests (appx 20% of the requests) that do not implement this new method correctly will be rejected.
  • Monday Aug 10 from 2PM (ET) through 4PM, all requests that do not implement this new method correctly will be rejected.
Uhh, sorry what? Planned production outage 2 weeks earlier?!! Needless to say this led to an elevated blood pressure for many. I looked at some of the Sample REST requests, found some HMAC code already available on RIA Forge in several "amazon" related projects and thought "This will be easy enough".

Its not that it wasn't easy to reproduce the example they gave. That part WAS easy. The difficult / frustrating part was getting the REAL requests signed and authenticated. The good news for many is that you can simply check the status code of your original request and make it again if it has failed during this first round that lasts 24 hours and you will likely be fine.

Here is a checklist of what I would suggest looking at if you are having problems, this is from my own experiences and is bound to help someone. As I receive feedback, I'll update accordingly.
  1. Make sure you are using your OWN secret key.
  2. Use http://ecs.amazonaws.com - NOT http://webservices.amazon.com
  3. Do NOT use a trailing slash at the end of /onca/xml?...
  4. DO NOT put line breaks in your query string. Just use the normal & to separate your params.
  5. Make sure you are only doing a line break chr(10) like the following:
    'GET' & chr(10) & 'ecs.amazonaws.com' & chr(10) & '/onca/xml' & chr(10) & '#YourQueryStringHERE#'
For me, the big epiphany was after using the webservices URL, one type of responseGroup would work, but another would not. The error was coming from ecs.amazonaws.com, even though I was making my calls to webservices.amazon.com, just like their example.

Here is an example of the code that I used.

14 comments:

Unknown said...

do you have any code examples you'd like to show?

Phillyun said...

@westfork - last paragraph. :)

Mike said...

Have you got this working with cfhttp?

Unknown said...

>>@westfork - last paragraph. :)

Thanks, don't know how I missed that.

Nice code, it works beautifully.

>>@ Mike - Have you got this working with cfhttp?

Just change the local scope to the Request scope and call that.

cfhttp url="#Request.urlPassed#" method="get"

GrubberGamer said...

Saved me hours of work. Kudos. :)

Phillyun said...

@Jeff Glad it worked out for you, where do I send the bill?
Better yet, you help me out next time they add the next requirement.

Jim said...

When I try to CfHttp it I get:

Element URLPASSED is undefined in REQUEST.


The error occurred in C:\websites\books2b\testaws.cfm: line 103

101 :
102 :
103 : cfhttp url="#Request.urlPassed#" method="get">

Unknown said...

@Jim... Sorry I was not clear in how I used Phillyun's CFC file.

I have many different files accessing the CFC via the cfinvoke tag and passing arguments to it via cfinvokeargument tag's.

I then changed: cfset local.urlPassed = to cfset Request.urlPassed = so I could use the returned variables in any file involved in the operation.

You first have to instantiate the request scope variable urlPassed and then populate it to use it.

I'm sure there is a more elegant way to do this, but I was up against the Aug. 15 deadline and had a lot of code to rewrite.

Anonymous said...

Thanks a ton! Your code worked great!

Dom said...

Great stuff. I've managed to get the hmac encrytion working without the external code (using java objects instead). Feels much cleaner though only tested on Railo so far. Something like:

// setup java encryption objects...
_secretKey = CreateObject('java', 'javax.crypto.spec.SecretKeySpec' ).Init(secretKey.GetBytes(), 'HmacSHA256');
_hmacEncryptor = CreateObject('java', "javax.crypto.Mac").getInstance("HmacSHA256");
_hmacEncryptor.init(_secretKey);

//...

// Genereate raw HMAC SHA-256
raw = _hmacEncryptor.doFinal( stringToSign.GetBytes() );

// 4. base64 and url encode
Signature = Replace( Replace( ToBase64( raw ), '+', '%2B', 'all'), '=', '%3D', 'all');

Dom said...

I ended up creating an hmac encoding cfc that simply wraps the java encoding objects. You can see it on pastebin for now:

http://pastebin.com/f6e1ec14a

Dominic

DonOmite said...

That almost made it. But in the aaws.cfc you use 1 createObject. If you could get rid of that then even people on hosts that block createobject and cfobject could use it.
Really trying to get around that problem. Any suggestions?

Phillyun said...

My first thought is to get a new host. :)
A more useful approach would be searching / asking CF-Talk on http://www.houseoffusion.com/
lots of helpful folks there.

Unknown said...

It looks like this breaks on CF 11.

I have narrowed it down to something CF11 is putting in the request sent to Amazon, but I can't see anything weird when I dump the request ot the link...

Anyone sort this out?