"Unsupported transfer encoding: chunked" When using TestRestTemplate with Apache HTTP client and spring-cloud-starter-function-web

[matthew-js-porter]

When using TestRestTemplate to call and application using spring-cloud-starter-function-web with org.apache.httpcomponents.client5:httpclient5 a ClientProtocolException error is thrown. The same test runs when using JdkClientHttpRequestFactory instead.

Error

org.springframework.web.client.ResourceAccessException: I/O error on POST request for "http://localhost:61395/": Unsupported transfer encoding: chunked

	at org.springframework.web.client.RestTemplate.createResourceAccessException(RestTemplate.java:926)
	at org.springframework.web.client.RestTemplate.doExecute(RestTemplate.java:906)
	at org.springframework.web.client.RestTemplate.execute(RestTemplate.java:801)
	at org.springframework.web.client.RestTemplate.postForObject(RestTemplate.java:518)
	at org.springframework.boot.test.web.client.TestRestTemplate.postForObject(TestRestTemplate.java:390)
	at com.example.apache_test_resttemplate.ApacheTestRestTemplateApplicationTests.saysHello(ApacheTestResttemplateApplicationTests.java:19)
	at java.base/java.lang.reflect.Method.invoke(Method.java:580)
	at java.base/java.util.ArrayList.forEach(ArrayList.java:1596)
	at java.base/java.util.ArrayList.forEach(ArrayList.java:1596)
Caused by: org.apache.hc.client5.http.ClientProtocolException: Unsupported transfer encoding: chunked
	at org.apache.hc.client5.http.impl.classic.InternalHttpClient.doExecute(InternalHttpClient.java:177)
	at org.apache.hc.client5.http.impl.classic.CloseableHttpClient.execute(CloseableHttpClient.java:87)
	at org.apache.hc.client5.http.impl.classic.CloseableHttpClient.execute(CloseableHttpClient.java:55)
	at org.apache.hc.client5.http.classic.HttpClient.executeOpen(HttpClient.java:183)
	at org.springframework.http.client.HttpComponentsClientHttpRequest.executeInternal(HttpComponentsClientHttpRequest.java:99)
	at org.springframework.http.client.AbstractStreamingClientHttpRequest.executeInternal(AbstractStreamingClientHttpRequest.java:71)
	at org.springframework.http.client.AbstractClientHttpRequest.execute(AbstractClientHttpRequest.java:81)
	at org.springframework.web.client.RestTemplate.doExecute(RestTemplate.java:900)
	... 7 more
Caused by: org.apache.hc.core5.http.NotImplementedException: Unsupported transfer encoding: chunked
	at org.apache.hc.core5.http.impl.DefaultContentLengthStrategy.determineLength(DefaultContentLengthStrategy.java:90)
	at org.apache.hc.core5.http.impl.io.DefaultBHttpClientConnection.receiveResponseEntity(DefaultBHttpClientConnection.java:355)
	at org.apache.hc.core5.http.impl.io.HttpRequestExecutor.execute(HttpRequestExecutor.java:210)
	at org.apache.hc.client5.http.impl.classic.InternalExecRuntime.lambda$execute$0(InternalExecRuntime.java:236)
	at org.apache.hc.client5.http.impl.io.PoolingHttpClientConnectionManager$InternalConnectionEndpoint.execute(PoolingHttpClientConnectionManager.java:791)
	at org.apache.hc.client5.http.impl.classic.InternalExecRuntime.execute(InternalExecRuntime.java:233)
	at org.apache.hc.client5.http.impl.classic.MainClientExec.execute(MainClientExec.java:121)
	at org.apache.hc.client5.http.impl.classic.ExecChainElement.execute(ExecChainElement.java:51)
	at org.apache.hc.client5.http.impl.classic.ConnectExec.execute(ConnectExec.java:199)
	at org.apache.hc.client5.http.impl.classic.ExecChainElement.execute(ExecChainElement.java:51)
	at org.apache.hc.client5.http.impl.classic.ProtocolExec.execute(ProtocolExec.java:192)
	at org.apache.hc.client5.http.impl.classic.ExecChainElement.execute(ExecChainElement.java:51)
	at org.apache.hc.client5.http.impl.classic.ContentCompressionExec.execute(ContentCompressionExec.java:150)
	at org.apache.hc.client5.http.impl.classic.ExecChainElement.execute(ExecChainElement.java:51)
	at org.apache.hc.client5.http.impl.classic.HttpRequestRetryExec.execute(HttpRequestRetryExec.java:113)
	at org.apache.hc.client5.http.impl.classic.ExecChainElement.execute(ExecChainElement.java:51)
	at org.apache.hc.client5.http.impl.classic.RedirectExec.execute(RedirectExec.java:110)
	at org.apache.hc.client5.http.impl.classic.ExecChainElement.execute(ExecChainElement.java:51)
	at org.apache.hc.client5.http.impl.classic.InternalHttpClient.doExecute(InternalHttpClient.java:174)
	... 14 more

Steps to Reproduce

1. Generate a 3.4.0 project from https://start.spring.io/
2. add Dependency Management for Spring Cloud

	<dependencyManagement>
		<dependencies>
			<dependency>
				<groupId>org.springframework.cloud</groupId>
				<artifactId>spring-cloud-dependencies</artifactId>
				<version>2024.0.0</version>
				<type>pom</type>
				<scope>import</scope>
			</dependency>
		</dependencies>
	</dependencyManagement>

3. add spring-cloud-starter-function-web

<dependency>
	<groupId>org.springframework.cloud</groupId>
	<artifactId>spring-cloud-starter-function-web</artifactId>
</dependency>

4. Create function handler
   FunctionHandler.java

@Component
public class FunctionHandler implements Function<Person, String> {

    @Override
    public String apply(final Person person) {
        return "Hello %s %s".formatted(person.firstName(), person.lastName());
    }
}

Person.java

public record Person(String firstName, String lastName) {}

4. Create Test

@SpringBootTest(webEnvironment = SpringBootTest.WebEnvironment.RANDOM_PORT)
class ApacheTestRestTemplateApplicationTests {


	@Autowired
	private TestRestTemplate testRestTemplate;

	@Test
	void saysHello() {
		final String response = testRestTemplate.postForObject("/", new Person("First", "Last"), String.class);
		assertThat(response).isEqualTo("Hello First Last");
	}
}

5. Run test and observe that the test is successful.
6. Add httpclient5 dependency

<dependency>
	<groupId>org.apache.httpcomponents.client5</groupId>
	<artifactId>httpclient5</artifactId>
</dependency>

7. Run tests and observe error

[wilkinsona]

The response includes two Transfer-Encoding headers:

transfer-encoding: chunked
Transfer-Encoding: chunked

The Apache HTTP client doesn't tolerate this and throws an exception when it processes the second header. As far as I can tell, the transfer-encoding header is being set on the response by org.springframework.cloud.function.web.util.FunctionWebRequestProcessingHelper. It should probably be ignored as, for example, Content-Length currently is.

I think you should raise a Spring Cloud Function issue for this. You may also want to raise an Apache HTTP client issue to see if they want to tolerate multiple Transfer-Encoding headers with the same value.

[matthew-js-porter]

@wilkinsona Thanks! that explains it. Opened the Spring Cloud Function issue:
spring-cloud/spring-cloud-function#1220

[olegz]

@wilkinsona indeed it's on us so feel free to close this issue here and I'll handle it in functions.