The J2ME HTTPConnection which comes with MIDP lets you make HTTP requests to your server. It doesn’t do much at a high level, for example the API doesn’t have methods like addCookie() – you need to manually add them with a request header. But the implementation is clever enough to turn any request body which is greater than around 2Kb into a chunked request.
With HTTP 1.0, the request had to contain a header called Content-Length which told the server how many bytes to read off the input stream when reading the body. HTTP 1.1 introduced the Transfer-Encoding header, which lets the client omit the Content-Length header, and instead create chunks of request body, which optimises the upload meaning that a) the server can start processing before it has everything, and b) more importantly for J2ME where memory might be a valuable resource, it lets the client send a bit of the request, free up that allocated memory and then send some more of the request.
For a POST request, with no chunking, the headers and body might look like this:
POST /log.jsp HTTP/1.1 User-Agent: Mozilla/4.0 (maxant J2ME Client) Accept: text/html,application/xhtml+xml,application/xml Content-Type: application/x-www-form-urlencoded Content-Length: 51 Host: wwwchaseamatecom:8089 problem=Failed%20to%20get%20installation%20response
Chunked, that becomes:
POST /ota/addInstallation.jsp HTTP/1.1 User-Agent: Mozilla/4.0 (maxant J2ME Client) Accept: text/html,application/xhtml+xml,application/xml Content-Type: application/x-www-form-urlencoded Host: wwwchaseamatecom:8089 Transfer-Encoding: chunked problem=Failed%20to%20get%20installation%20response
You’ll notice that the body of the second example, "problem=…" doesn’t contain chunk headers (search Wikipedia for chunking to see an example). The reason is that I copied that text out of the TCP/IP Monitor in Eclipse 3.5 and it seems kind enough to hide that information and simply show you the unchunked body. Not great if you aren’t expecting that, somewhat useful none the less.
Anyway, about time I got to the point of this blog article. Any version of Tomcat before 5.5.28 or 6.0.21 (ie. the latest versions at the time of writing!) had a nasty bug in them whereby POST requests did NOT get the body parsed and put into request.getParameter(String) or request.getQueryString(). Very nasty indeed for J2ME developers. I trawled the internet and found people were having problems from around 2002 when HTTP 1.1 became available in J2ME. Only with the following bug fix has this finally been corrected on Tomcat, some 8 years later.
https://issues.apache.org/bugzilla/show_bug.cgi?id=37794
Work arounds included using Apache to put together the chunks before sending the request on to Tomcat, or manually splitting the request on the device, into small chunks and upload each one and then joining them together at the end – what a load of work to have to do!
I’m working with an old 5.5.9 Tomcat installation and have just upgraded it to 5.5.28 to fix this problem.
However before I found this bug report I did consider creating a filter and configuring it in my web.xml. The idea was that the filter would read the request body, URL decode it, unlock the requests parameter map (Catalina has a lockable map which is locked after parsing the request), and put the request bodies request parameters into the map. Something like this:
/*
* Copyright (c) 2010 Ant Kutschera, maxant
*
* This file is part of Ant Kutschera's blog, http://blog.maxant.co.uk
*
* This is free software: you can redistribute it and/or modify
* it under the terms of the Lesser GNU General Public License as published by
* the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* It is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* Lesser GNU General Public License for more details.
* You should have received a copy of the Lesser GNU General Public License
* along with Foobar. If not, see .
*/
import java.io.IOException;
import java.io.InputStream;
import java.net.URLDecoder;
import java.util.HashMap;
import java.util.Map;
import java.util.Properties;
import java.util.Map.Entry;
import javax.servlet.http.HttpServletRequest;
//nasty, but we are afterall fixing a bug in catalina!
import org.apache.catalina.util.ParameterMap;
public final class RequestHandler implements Filter {
private static final String TRANSFER_ENCODING = "Transfer-Encoding";
private static final String CHUNKED = "chunked";
public void destroy() {
}
public void init(FilterConfig filterConfig) throws ServletException {
}
public void doFilter(ServletRequest request, ServletResponse response,
FilterChain chain) throws IOException, ServletException {
String header = request.getHeader(TRANSFER_ENCODING);
InputStream is = null;
try{
if(header != null && header.equals(CHUNKED)){
//time to work our magic
((ParameterMap)request.getParameterMap()).setLocked(false);
is = request.getInputStream();
//read the body as a map
Properties props = new Properties();
props.load(is);
//get the encoding to use for url-decoding
String enc = request.getCharacterEncoding();
if(enc == null){
enc = "UTF-8";
}
for(Entry e : props.entrySet()){
String key = (String) e.getKey();
//need to decode it!
key = URLDecoder.decode(key, enc);
if(!request.getParameterMap().containsKey(key)){
String val = (String) e.getValue();
//need to decode it!
val = URLDecoder.decode(val, enc);
request.getParameterMap().put(key, val);
}
}
}
((ParameterMap)request.getParameterMap()).setLocked(true);
}finally{
if(is != null) is.close();
}
}
}
I wasted hours because of this rubbish bug. The problem is that its full of red herrings, making you think for example that because the monitor shows the request body unchunked, that the device isn’t chunking the body properly. The web is also full of very old posts related to servers not being HTTP 1.1 compatible. And there are lots of work arounds which people went to the effort of because they had no other choice until most recently. So hopefully if you have had chunking problems with J2ME, you found this article without too much hassle!
Copyright (c) 2010 Ant Kutschera
