Received error 503 with Istio 1.16.3

1

I’m using Istio 1.16.3 with EKS 1.24. I’m trying to set up an app with service and virtual service YAMLs below. My app is exposing port 80.

Virtual Service YAML

kind: VirtualService
apiVersion: networking.istio.io/v1alpha3
metadata:
  name: vs-app
  namespace: example-ns
spec:
  gateways:
  - istio-system/gateway
  hosts:
  - example.com
  http:
  - route:
    - destination:
        host: app-svc
        port:
          number: 80
    retries:
      attempts: 0
    corsPolicy:
      allowOrigins:
      - prefix: "*"
      allowMethods:
      - POST
      - GET
      - PUT
      - DELETE
      - OPTIONS
      allowCredentials: false
      allowHeaders:
      - keep-alive
      - user-agent
      - cache-control
      - content-type
      - content-transfer-encoding
      - custom-header-1
      - x-accept-content-transfer-encoding
      - x-accept-response-streaming
      - x-user-agent
      - x-grpc-web
      - grpc-timeout
      - origin
      - accept
      - authorization
      - nonce
      - appinstanceid
      maxAge: "1h"
      exposeHeaders:
      - custom-header-1
      - grpc-status
      - grpc-message

Service YAML

apiVersion: v1
kind: Service
metadata:
  name: app-svc
  namespace: example-ns
  labels:
    app: app
spec:
  ports:
    - port: 80
      targetPort: 80
      name: http
  selector:
    app: app

Gateway YAML

apiVersion: networking.istio.io/v1alpha3
kind: Gateway
metadata:
  name: gateway-apps
  namespace: istio-system
spec:
  selector:
    istio: ingressgateway
  servers:
  - port:
      number: 80
      name: http
      protocol: HTTP
    hosts:
    - 'example.com'

I’m getting Error 503 upstream connect error or disconnect/reset before headers. reset reason: connection failure when I visit example.com from browser.

Below is the istio-ingressgateway logs

[2023-02-14T12:17:48.747Z] "GET / HTTP/1.1" 503 UF,URX upstream_reset_before_response_started{connection_failure} - "-" 0 114 23403 - "161.142.150.67,10.22.16.75" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/109.0.0.0 Safari/537.36" "db7ed4f3-731c-4521-835c-b3cd34d5d206" "example.com" "10.22.35.210:80" outbound|80||app-svc.example-ns.svc.cluster.local - 10.22.10.8:8080 161.142.150.67:0 - -

[2023-02-14T12:18:12.250Z] "GET /favicon.ico HTTP/1.1" 503 UF,URX upstream_reset_before_response_started{connection_failure} - "-" 0 114 30026 - "161.142.150.67,10.22.13.81" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/109.0.0.0 Safari/537.36" "b61fb196-a887-4919-a7f8-e6aeea42fa55" "example.com" "10.22.35.210:80" outbound|80||app-svc.example-ns.svc.cluster.local - 10.22.10.8:8080 161.142.150.67:0 - -

I’ve tried to port forward the app’s pod and app’s virtual service and it is working normally with localhost:80. This set of YAMLs also working fine with Istio 1.14.3 and breaks after the upgrade to 1.16.3.