January 9, 2012

Delphi and Facebook Base64 Url Encoding

Was working on a Delphi server used to build a Facebook application, when I bumped into an odd Base64 decoding issue. Thanks to some very approximate Facebook documentation.

Was working on a Delphi server used to build a Facebook application, when I bumped into an odd Base64 decoding issue. If you look at the Facebook page documenting their Signed Request parameter, you can see it has a first part with an encoded signature (which I won't really mention here) plus some JSON data encoded in Base64. This looked trivial, but as I started using Indy's TIdDecoderMIME class, the result would be correct... but only at times. Other times, the final } of the JSON string was missing. Now, since there were three closing braces at the end of the specific error situation, I didn't spot the error immediately, the JSON string apparently looked good:


If you look at it carefully, you'll see that the third final brace is missing. So now I could figure out while the JSON parser was crashing badly on me (by they way, I'm using JSON SuperObject for this specific project, nice class). Back to the Facebook documentation. They did say "a base64url encoded JSON object", with a link to Wikipedia, but the link doesn't really explain much:

"modified Base64 for URL  variants exist, where the ' + ' and ' / ' characters of standard Base64 are respectively replaced by ' - ' and ' _ '," and " Some variants allow or require omitting the padding ' = ' signs "

Saying there are some variants of an algorithm is not really the best way of documenting it. Facebook doesn't really say which "variant" it is implementing, but, well, at least you can guess among a few options. Moreover, the demo simply calls a PHP function, base_url_decode, and doesn't offer much advice.

Initially I though of a problem with the Indy decoding, tried a couple of alternatives, went to nowhere. In fact Delphi has another SOAP-related Base64 decoded, who was doing the opposite, that is adding some garbage at the end of the string. Now that I knew where the issue was, it was easy to google "base64 decoding facebook" and find a nice blog post, http://qugstart.com/blog/ruby-and-rails/facebook-base64-url-decode-for-signed_request/. This is written in Ruby, but having a minimum of description, I could translate it over to Delphi in very little time (see image below for formatted version):

  strFixup := strInput + StringOfChar ('=', Length (strInput) mod 4); // nope, fixed below
  strFixup := StringReplace (strFixup, '-', '+', [rfReplaceAll]);
  strFixup := StringReplace (strFixup, '_', '/', [rfReplaceAll]);

Update: as suggested in a comment below, this is not correct, as when there is one extra byte we need to add 3. However, in case of zero extra bytes, we shouldn't pad with 4, so evern the original Ruby code seems wrong. I've now fixed it with:  (4 - Length (strInput) mod 4) mod 4, as in the image below.

To be honest I have to say there is a Q&A in the Facebook page I didn't initially read. It says "if you are missing a few characters, you are probably using normal base64 decoding instead of base64url decoding. See  wikipedia ." But, again, wikipedia said there are several variants, come on Facebook! Even more surprising it that the provide a sample conversion at the very end of the page ("Do you have any example?)" , but that's actually only half of the what's encoded in the initial string!  

Thanks to some very approximate Facebook documentation I wasted a few hours. I think documentation for online APIs should be much more precise and do't take for granted everyone is using a single programming language in the world!



Delphi and Facebook Base64 Url Encoding 

I agree with you, documentation for online API should
be more precise and clear! Very nice article!  

PS: '{"algorithm":"HMAC-...' line is getting out from
the page on Mozilla 9.0.1
Comment by RBA on January 9, 22:56

Delphi and Facebook Base64 Url Encoding 

I think you misunderstand the "variants" concept.

base64url IS the variant.

Using normal base64, input of a URL will result in + 
becoming %2B and / becoming %2F.

Using base64url, which is the variant of normal base64, 
these characters are substituted.

There's no reason to blame Facebook just because the 
Wikipedia entry is not well written.

base64url is common knowledge, and there are many 
alternative sources for this information.
Comment by Corrie Engelbrecht on January 9, 23:30

Delphi and Facebook Base64 Url Encoding 

Shouldn't it be like this to make the string length a
multiple of 4?

strFixup := strInput + StringOfChar ('=', 4 - Length
(strInput) mod 4); 

For example, if the remainder is 1, we must add three
equal signs, not one.

Thanks for this nice blog post. I would suggest using
a spellchecker, though.
Comment by Ghasem Kiani [http://ghasemkiani.blogspot.com] on January 10, 05:46

Delphi and Facebook Base64 Url Encoding 

  "there is no reason to blame Facebook if the Wikipedia 
entry is not well written". I beg to disagree. Either 
you provide full documentation or you refer to good 
quality documentation. Referring to incomplete doc means 
your documentation is bad.

  right my padding algorithm is not correct, but your is 
not correct either. It fails if length mod 4 = 0, as its 
adds another 4 padding elements. Will fix it later.

Comment by Marco Cantu [http://www.marcocantu.com] on January 10, 08:04

Post Your Comment

Click here for posting your feedback to this blog.

There are currently 0 pending (unapproved) messages.