Accept header with quoted parameter causes IllegalArgumentException in HttpHeaders.getContentType for the outputMessage [SPR-8917]

[spring-projects-issues]

David Pacheco opened SPR-8917 and commented

A HTTP request with the following Accept header

Accept: application/xhtml+xml; profile="http://www.wapforum.org/xhtml", application/vnd.wap.xhtml+xml

to a String Controller, causes the following exception to be thrown:

SEVERE: Servlet.service() for servlet dispatcher threw exception
java.lang.IllegalArgumentException: Invalid token character ':' in token "http://www.wapforum.org/xhtml"
        at org.springframework.http.MediaType.checkToken(MediaType.java:282)
        at org.springframework.http.MediaType.checkParameters(MediaType.java:302)
        at org.springframework.http.MediaType.<init>(MediaType.java:263)
        at org.springframework.http.MediaType.parseMediaType(MediaType.java:584)
        at org.springframework.http.HttpHeaders.getContentType(HttpHeaders.java:286)
        at org.springframework.http.converter.StringHttpMessageConverter.writeInternal(StringHttpMessageConverter.java:97)
        at org.springframework.http.converter.StringHttpMessageConverter.writeInternal(StringHttpMessageConverter.java:1)
        at org.springframework.http.converter.AbstractHttpMessageConverter.write(AbstractHttpMessageConverter.java:181)
        at org.springframework.web.servlet.mvc.annotation.AnnotationMethodHandlerAdapter$ServletHandlerMethodInvoker.writeWithMessageConverters(AnnotationMethodHandlerAdapter.java:973)
        at org.springframework.web.servlet.mvc.annotation.AnnotationMethodHandlerAdapter$ServletHandlerMethodInvoker.handleResponseBody(AnnotationMethodHandlerAdapter.java:931)
        at org.springframework.web.servlet.mvc.annotation.AnnotationMethodHandlerAdapter$ServletHandlerMethodInvoker.getModelAndView(AnnotationMethodHandlerAdapter.java:880)
        at org.springframework.web.servlet.mvc.annotation.AnnotationMethodHandlerAdapter.invokeHandlerMethod(AnnotationMethodHandlerAdapter.java:429)
        at org.springframework.web.servlet.mvc.annotation.AnnotationMethodHandlerAdapter.handle(AnnotationMethodHandlerAdapter.java:415)
        at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:788)
        at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:717)
        at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:644)
        at org.springframework.web.servlet.FrameworkServlet.doGet(FrameworkServlet.java:549)
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:617)

In brief:

This is due to the AbstractHttpMessageConverter write method calling the headers.setContentType() method of the outputMessage parameter with a MediaType with the following value "application/xhtml+xml;profile=http://www.wapforum.org/xhtml".

Note the lack of quotes around the profile value part.

The program flow is:

The MediaType is parsed from the HTTP Accept header with the value

application/xhtml+xml;profile="http://www.wapforum.org/xhtml"

Note - The inclusion of the quotes means that this value does not trigger the checkToken(String) method of MediaType.

In the StringHttpMessageConverter write method, a call is made to set the content type for the outputMessage.
This uses the toString() method of MediaType to set the Content-Type header, this results in the value being set to "application/xhtml+xml;profile=http://www.wapforum.org/xhtml".

Next the writeInternal method of StringHttpMessageConverter is called, this tries to get the MediaType for the outputMessage by calling outputMessage.getHeaders().getContentType(), which in turn calls MediaType.parseMediaType(value) with the value of "application/xhtml+xml;profile=http://www.wapforum.org/xhtml", due to the missing quotes in the profile parameter value, this will now trigger the checkToken(String) method of MediaType which ultimately results in the IllegalArgumentException being thrown.

---

Example request headers

GET /index.html HTTP/1.1
Accept: application/xhtml+xml; profile="http://www.wapforum.org/xhtml", application/vnd.wap.xhtml+xml
Accept-Language: en-us
Host: app.example.com
Connection: Keep-Alive
Pragma: no-cache

---

Affects: 3.0.4

Issue Links:

• Allow parsing of media types with single-quotes [SPR-9734] #14368 Allow parsing of media types with single-quotes ("is depended on by")
• Invalid MIME-type causes a 500 error [SPR-9142] #13781 Invalid MIME-type causes a 500 error

1 votes, 7 watchers

[spring-projects-issues]

Mikhail Kadan commented

Workaround is to implement Filter and cut off "profile='http://www.wapforum.org/xhtml'" from header (see http://vangjee.wordpress.com/2009/02/25/how-to-modify-request-headers-in-a-j2ee-web-application/ for details).

[spring-projects-issues]

Rossen Stoyanchev commented

MediaType does not validate quoted parameter values but stores them internally without the quotes. That means HttpHeaders is vulnerable to IllegalArgumentException because its setters expect MediaType but actually store the media type as a String (with unquoted values). The next time a getter is used, the media type string is parsed and validation fails on unquoted values, i.e.:

@Test
public void acceptWithQuotedParameterValue() {
  MediaType mediaType = MediaType.parseMediaType("application/xhtml+xml; profile=\"http://www.wapforum.org/xhtml\"");
  HttpHeaders headers = new HttpHeaders();
  headers.setAccept(Arrays.asList(mediaType));
  headers.getAccept();  // IllegalArgumentException: Invalid token character ':' in token ..
}

[spring-projects-issues]

Maxim Valyanskiy commented

The same error on 3.1.1 when string is quoted with single quote:

java.lang.IllegalArgumentException: Invalid token character ':' in token "'http://www.wapforum.org/xhtml'"
       at org.springframework.http.MediaType.checkToken(MediaType.java:352)
       at org.springframework.http.MediaType.checkParameters(MediaType.java:372)
       at org.springframework.http.MediaType.<init>(MediaType.java:333)
       at org.springframework.http.MediaType.parseMediaType(MediaType.java:673)
       at org.springframework.http.MediaType.parseMediaTypes(MediaType.java:691)
       at org.springframework.web.servlet.mvc.condition.ProducesRequestCondition.getAcceptedMediaTypes(ProducesRequestCondition.java:214)
       at org.springframework.web.servlet.mvc.condition.ProducesRequestCondition.access$4(ProducesRequestCondition.java:211)
       at org.springframework.web.servlet.mvc.condition.ProducesRequestCondition$ProduceMediaTypeExpression.matchMediaType(ProducesRequestCondition.java:283)
       at org.springframework.web.servlet.mvc.condition.AbstractMediaTypeExpression.match(AbstractMediaTypeExpression.java:63)
       at org.springframework.web.servlet.mvc.condition.ProducesRequestCondition.getMatchingCondition(ProducesRequestCondition.java:163)
       at org.springframework.web.servlet.mvc.method.RequestMappingInfo.getMatchingCondition(RequestMappingInfo.java:175)
       at org.springframework.web.servlet.mvc.method.RequestMappingInfoHandlerMapping.getMatchingMapping(RequestMappingInfoHandlerMapping.java:64)
       at org.springframework.web.servlet.mvc.method.RequestMappingInfoHandlerMapping.getMatchingMapping(RequestMappingInfoHandlerMapping.java:1)
       at org.springframework.web.servlet.handler.AbstractHandlerMethodMapping.addMatchingMappings(AbstractHandlerMethodMapping.java:284)
       at org.springframework.web.servlet.handler.AbstractHandlerMethodMapping.lookupHandlerMethod(AbstractHandlerMethodMapping.java:251)
       at org.springframework.web.servlet.handler.AbstractHandlerMethodMapping.getHandlerInternal(AbstractHandlerMethodMapping.java:216)
       at org.springframework.web.servlet.handler.AbstractHandlerMethodMapping.getHandlerInternal(AbstractHandlerMethodMapping.java:1)
       at org.springframework.web.servlet.handler.AbstractHandlerMapping.getHandler(AbstractHandlerMapping.java:288)
       at org.springframework.web.servlet.DispatcherServlet.getHandler(DispatcherServlet.java:1063)

Header:

Accept: application/xml,application/vnd.wap.xhtml+xml,application/xhtml+xml;profile='http://www.wapforum.org/xhtml',text/html;q=0.9,text/plain;q=0.8,image/png,*/*;q=0.5

[spring-projects-issues]

Rossen Stoyanchev commented

Are you sure it's 3.1.1? Here is the test case for it.

[spring-projects-issues]

Maxim Valyanskiy commented

Yes, the same problem on master branch in git.

Please merge my fix: #93

[spring-projects-issues]

Dan Checkoway commented

I just want to reiterate that this problem was NOT fixed in 3.1.1 -- or 3.1.2.

Here's the 3.1.2.RELEASE source code:

projects/org.springframework.web/src/main/java/org/springframework/http/MediaType.java
378:    private boolean isQuotedString(String s) {
379-            return s.length() > 1 && s.startsWith("\"") && s.endsWith("\"") ;
380-    }

(I also used jad to decompile the code in org.springframework.web-3.1.2.RELEASE.jar, and I observed the same exact code.)

Here's the master branch in git:

spring-web/src/main/java/org/springframework/http/MediaType.java
378:    private boolean isQuotedString(String s) {
379-            if (s.length() < 2) {
380-                    return false;
381-            }
382-            else {
383-                    return ((s.startsWith("\"") && s.endsWith("\"")) || (s.startsWith("'") && s.endsWith("'")));
384-            }
385-    }

I think you should reopen this ticket and update the Fix Version field. And pretty pretty please...release a new version ASAP with this fix. Thanks! :-)

[spring-projects-issues]

Igor Mihalik commented

There are two problems mentioned in this JIRA issue. The original one was related to "quotes" that were deleted and this caused problems later. That's why "unquote" method call was removed (see commit log here). And this one was fixed in 3.1.1 as stated.

Then there's a second problem with single quote, which should have been reported as a separate issue. To accept single quote in addition to accepting double quote in strings. However RFC HTTP/1.1 defines:

quoted-string  = ( <"> *(qdtext | quoted-pair ) <"> ). 

It does not mention single quote. But in github master the pull request was already accepted to accept single quote.

IMO this case should not be reopened. But new case with "feature request for single quote" should be created and discussed with the impact of this as single quote is not the same as double quote according to the RFC. Including single quote can have regression consequences in existing applications that would later upgrade to new spring version.

[spring-projects-issues]

Rossen Stoyanchev commented

Igor, your analysis is correct. Will you please open a ticket? I will take care of it from there. As you wrote, the single quote issue was fixed in master but was not backported to 3.1.x.

[spring-projects-issues]

Igor Mihalik commented

new issue created #14368