Unable to access s3 after 1.1.1 upgrade

We have been using 1.0.6 for a while and accessing AWS S3 from pods running the istio-proxy without any issues. After the upgrade running a command like s3cmd ls from inside my container causes it to crash with .

error: [Errno 104] Connection reset by peer

If I run the same pod without a sidecar everything works fine.

I have the following service entry in place:

apiVersion: networking.istio.io/v1alpha3
kind: ServiceEntry
  name: https-traffic
  namespace: default
  - '*.amazonaws.com'
  - '*.com' 
  - '*.org' 
  - '*.net' 
  location: MESH_EXTERNAL
  - name: http
    number: 80
    protocol: HTTP
  - name: https
    number: 443
    protocol: HTTPS
  resolution: NONE

Hoping someone else might have some insight into why this would be happening, or perhaps some suggestions for debugging it better… I’m not quite sure if any relevant debug information exists, I certainly don’t see connection errors in the istio-proxy when attempting to run s3cmd.

I’m able to access the s3 API endpoint without any issues using curl:

root@jenkins-tlv-slave-58cc7898f5-nf7b5:/# curl https://s3.amazonaws.com -v             
* Rebuilt URL to: https://s3.amazonaws.com/
* Hostname was NOT found in DNS cache
*   Trying
* Connected to s3.amazonaws.com ( port 443 (#0)
* successfully set certificate verify locations:
*   CAfile: none
  CApath: /etc/ssl/certs
* SSLv3, TLS handshake, Client hello (1):
* SSLv3, TLS handshake, Server hello (2):
* SSLv3, TLS handshake, CERT (11):
* SSLv3, TLS handshake, Server key exchange (12):
* SSLv3, TLS handshake, Server finished (14):
* SSLv3, TLS handshake, Client key exchange (16):
* SSLv3, TLS change cipher, Client hello (1):
* SSLv3, TLS handshake, Finished (20):
* SSLv3, TLS change cipher, Client hello (1):
* SSLv3, TLS handshake, Finished (20):
* SSL connection using ECDHE-RSA-AES128-GCM-SHA256
* Server certificate:
* 	 subject: C=US; ST=Washington; L=Seattle; O=Amazon.com Inc.; CN=s3.amazonaws.com
* 	 start date: 2018-12-03 00:00:00 GMT
* 	 expire date: 2019-10-25 12:00:00 GMT
* 	 subjectAltName: s3.amazonaws.com matched
* 	 issuer: C=US; O=DigiCert Inc; OU=www.digicert.com; CN=DigiCert Baltimore CA-2 G2
* 	 SSL certificate verify ok.
> GET / HTTP/1.1
> User-Agent: curl/7.35.0
> Host: s3.amazonaws.com
> Accept: */*
< HTTP/1.1 307 Temporary Redirect
< x-amz-id-2: x9+rkYSgXYkqJX65JM7lkjKlrJKjg4RK9XYyHFo1NPvoRTifMYQSXImnKmjGfKrIwmkrjijsS3w=
< x-amz-request-id: EECE00AA4B893B6B
< Date: Thu, 28 Mar 2019 18:06:09 GMT
< Location: https://aws.amazon.com/s3/
< Content-Length: 0
* Server AmazonS3 is not blacklisted
< Server: AmazonS3
* Connection #0 to host s3.amazonaws.com left intact

I ended up figuring this out. Istio 1.1.1 still blocks port 80 egress, although it doesn’t block anything else. I upgraded to 1.1.3 and it solved my issue.