From mod_deflate to mod_gzip
As you might remember from an earlier post, I've been experimenting with compressed HTTP data transfer, using an Apache server and a .NET CF client application. As I didn't want to write extra server-side code, I've used mod_deflate of Apache 2.0 on one server, but had to switch back to mod_gzip for an Apache 1.3 server. mod_gzip is not that different from mod_deflate, although there are a few more configuration settings to get right, and the behavior is not identical. But it is close enough, and I could use the same test VCL app to retrieve gzipped SOAP content from the other server.
The CF Client
Now, let's look into the .NET side of the problems. First I found a decompression library. I found one among the #ZipLib project. Incorporating that in the project was the easy part. Than I've started modifying the code that uses the HttpWebRequest object to ask the data to the server, adding the Accept-Encoding header:
Request.Headers.Item['Accept-Encoding'] := 'gzip';
I made some experiments and after a while I got it working, until I start to do an HTTP POST operation sending the required SOAP Envelope. It ends up that if you post data (using the stream returned by the Request.GetRequestStream call) the .NET library adds an extra header: "Expect: 100-continue". This changes the HTTP protocol enough to fool mod_gzip, which doesn't compress the response any more.
Many Solutions, None Working
After googling around a little, I found other developers with the same problem. One suggested solutions was to use a specific configuration setting (called Expect100Continue) in the ServicePointManager class of the FCL. Turns out this was indeed added in version 1.1, but only in the full framework, not in the CF version. A dead end.
Another suggestion I found was to scale back to HTTP 1.0, as it doesn't support the Expect header, so that .NET won't send it. This is truly the case. However, with HttpVersion.Version10, the FCL omits also the Host header, causing quite some grief to the target apache server. As much as I tired to add this header, this seems to be impossible. .NET forbids you to manually add a header for which there is a corresponding property (even in the case of Host, for which there is no property). The FCL has the last word on the HTTP content, something I find foolish (and very different from, say, Indy).
Down to Sockets
My next thought was I had to be in control. So I tried using a TCPClient object and a low level NetworkStream object. Having already captured the output data packets (the only way I could figure out the problems above) I cloned one in a string, sent it over with some proper data, looked at the result and I did find the gzipped data. Some more time to split the response header from the compressed data, and I had my program up and running (although I still have to encode the user permission data -- now hardcoded in the encoded format).
Here Comes the GPRS
At the end, I made a test (same application, same device) using GPRS connectivity rather than the Web connectivity obtained via the host computer... and (as I feared) I found that the GPRS gateway trashes my socket connection data somehow (not easy to know how) so that the servers gets a slightly different request and does not compress the data any more. Of course, it did work with the Apache 2 mod_deflate!
So I'm not done yet. I have to figure out which data reaches the server and (i) tweak the header to make it work or (ii) add a custom gateway on the sever invariably returning the compressed data (iii) find some documentation about GPRS gateways to figure out how to make them work. Moreover, when you initiate the socket connection the FCL doesn't ask the device to connect to the web, differently from the HTTP connection, so there is one more issue to solve.