SSL_ERROR_SYSCALL when trying to access AWS url

1

I’m running within EKS us-west-2

kube-shell> k version
Client Version: version.Info{Major:"1", Minor:"11", GitVersion:"v1.11.2", GitCommit:"bb9ffb1654d4a729bb4cec18ff088eacc153c239", GitTreeState:"clean", BuildDate:"2018-08-07T23:17:28Z", GoVersion:"go1.10.3", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"12+", GitVersion:"v1.12.6-eks-d69f1b", GitCommit:"d69f1bf3669bf00b7f4a758e978e0e7a1e3a68f7", GitTreeState:"clean", BuildDate:"2019-02-28T20:26:10Z", GoVersion:"go1.10.8", Compiler:"gc", Platform:"linux/amd64"}

My istio version is 1.1.3

kube-shell> istioctl proxy-status | grep "my-shell\|VERSION"
NAME                                        CDS        LDS        EDS               RDS        PILOT                            VERSION
my-shell-678c4759c7-zwxpf.mobility          SYNCED     SYNCED     SYNCED (100%)     SYNCED     istio-pilot-7d6c946b9d-w9jvv     1.1.3

I have a very simple service entry

kube-shell> k get serviceentry --all-namespaces  | grep dynamo
istio-xevo     dynamo                             [dynamodb.us-west-2.amazonaws.com httpbin.org]   MESH_EXTERNAL   DNS          4m

which looks like …

kind: ServiceEntry
metadata:
  name: dynamo
  namespace: istio-xevo
spec:
  hosts:
    - dynamodb.us-west-2.amazonaws.com
    - httpbin.org
  location: MESH_EXTERNAL
  ports:
    - number: 443
      name: https
      protocol: HTTPS
  resolution: DNS

I’ve deliberately put both dynamodb.us-west-2.amazonaws.com and httpbin.org to prove a point. One of those works and one doesn’t

Example of working httpbin

bash-4.4# wget https://httpbin.org/headers
Connecting to httpbin.org (34.238.32.178:443)
headers              100% |******************************************************|    76  0:00:00 ETA
bash-4.4# cat headers 
{
  "headers": {
    "Host": "httpbin.org", 
    "User-Agent": "Wget"
  }
}

Now I’ll try the dynamo db entry

bash-4.4# wget https://dynamodb.us-west-2.amazonaws.com
Connecting to dynamodb.us-west-2.amazonaws.com (52.94.29.110:443)
ssl_client: dynamodb.us-west-2.amazonaws.com: handshake failed: Connection reset by peer
wget: error getting response: Connection reset by peer

These are my cluster routes for that pod

kube-shell> istioctl -n mobility proxy-config cluster account-67dd44f848-l72z9.mobility | grep amazon
dynamodb.us-west-2.amazonaws.com                          443       -                      outbound      &{STRICT_DNS}
metadata.amazonaws                                        80        -                      outbound      &{EDS}

and here is the istio-proxy log for that pod on just the dynamo call

{
	"bytes_sent": "0",
	"upstream_cluster": "outbound|443||dynamodb.us-west-2.amazonaws.com",
	"downstream_remote_address": "10.0.18.199:57334",
	"path": "-",
	"authority": "-",
	"protocol": "-",
	"upstream_service_time": "-",
	"upstream_local_address": "-",
	"duration": "6",
	"downstream_local_address": "52.94.10.110:443",
	"response_code": "0",
	"user_agent": "-",
	"response_flags": "UF,URX",
	"start_time": "2019-04-26T16:37:25.200Z",
	"method": "-",
	"request_id": "-",
	"upstream_host": "52.94.29.96:443",
	"x_forwarded_for": "-",
	"requested_server_name": "-",
	"bytes_received": "0",
	"istio_policy_status": "-"
}{
	"duration": "4",
	"downstream_local_address": "52.94.28.138:443",
	"response_code": "0",
	"user_agent": "-",
	"response_flags": "UF,URX",
	"start_time": "2019-04-26T16:37:53.423Z",
	"method": "-",
	"request_id": "-",
	"upstream_host": "52.94.28.138:443",
	"x_forwarded_for": "-",
	"requested_server_name": "-",
	"bytes_received": "0",
	"istio_policy_status": "-",
	"bytes_sent": "0",
	"upstream_cluster": "outbound|443||dynamodb.us-west-2.amazonaws.com",
	"downstream_remote_address": "10.0.18.199:36430",
	"path": "-",
	"authority": "-",
	"protocol": "-",
	"upstream_service_time": "-",
	"upstream_local_address": "-"
}