Strange Issue with downloading file, using 'TCP' it works, using 'HTTP' breaks

Networking
1

Hi,
We are using Istio 1.4.1.
We create a service
“spec”: {
“ports”: [
{
“name”: “http”,
“protocol”: “TCP”,
“port”: 9609,
“targetPort”: 9000
}
],

We curl with
curl -v GET "http://fileservice.msdemo.svc.cluster.local:9609/file/o/81c54c13-f409-4060-825b-da10a7821f5d

it breaks with a 503 UC
[2020-03-05T19:15:51.896Z] “GET /file/o/81c54c13-f409-4060-825b-da10a7821f5d HTTP/1.1” 503 UC “-” “-” 0 95 11 - “-” “curl/7.52.1” “8ed89a2f-e1b1-948e-8542-eccde2302f20” “fileservice.msdemo.svc.cluster.local:9609” “127.0.0.1:9000” inbound|9609|http|fileservice.msdemo.svc.cluster.local - 10.244.2.15:9000 10.244.2.15:43992 - default

If we change the service spec to
“ports”: [
{
“name”: “tcp”,
“protocol”: “TCP”,
“port”: 9609,
“targetPort”: 9000
}
],

then it works and the same CURL now gets
< HTTP/1.1 200 OK
< transfer-encoding: chunked
< Content-Length: 3505
< Content-Disposition: attachment; filename=“download.jpg”; filename*=“UTF-8’'download.jpg”
< Access-Control-Allow-Origin: *
< Access-Control-Allow-Headers: Csrf-Token, X-Auth-Token, X-Requested-With, Accept, Content-Type, Authorization, Content-Length, Cache-Control, Pragma, If-Modified-Since
< Access-Control-Allow-Methods: HEAD,GET,POST,PUT,PATCH,DELETE
< Access-Control-Allow-Credentials: true
< Content-Type: image/jpeg
< Date: Thu, 05 Mar 2020 19:27:20 GMT

The best guess right now is that ‘http’ will trigger Istio to validate more things, such as the headers?
Does anyone know what’s wrong here? Thanks

2

Answering my own question here. I was able to set debug mode to Envoy proxy by doing a POST to the Envoy admin interface. Then I found the issue of
[Envoy (Epoch 0)] [2020-03-07 00:19:15.374][22][debug][client] [external/envoy/source/common/http/codec_client.cc:124] [C5668] protocol error: http/1.1 protocol error: HPE_UNEXPECTED_CONTENT_LENGTH
[Envoy (Epoch 0)] [2020-03-07 00:19:15.374][22][debug][connection] [external/envoy/source/common/network/connection_impl.cc:104] [C5668] closing data_to_write=0 type=1
[Envoy (Epoch 0)] [2020-03-07 00:19:15.374][22][debug][connection] [external/envoy/source/common/network/connection_impl.cc:193] [C5668] closing socket: 1
[Envoy (Epoch 0)] [2020-03-07 00:19:15.374][22][debug][client] [external/envoy/source/common/http/codec_client.cc:88] [C5668] disconnect. resetting 1 pending requests
[Envoy (Epoch 0)] [2020-03-07 00:19:15.374][22][debug][client] [external/envoy/source/common/http/codec_client.cc:111] [C5668] request reset
[Envoy (Epoch 0)] [2020-03-07 00:19:15.374][22][debug][router] [external/envoy/source/common/router/router.cc:911] [C5665][S11149092392049592664] upstream reset: reset reason connection termination

This is an issue with the ‘chunked’ encoding and ‘content-length’. I removed the ‘content-length’ header, then it is working.

3

FYI, it looks like this may have been tackled in Envoy: