Retry is not working when scaling pods to 0

Networking
1

Retry configurations are not working as I tried scaling the pods to 0 using the command kubectl scale --replicas=0. Checked the logs of the istio-ingressgateway pod, I am getting 503 UH no_healthy_upstream.

Here are my deployment, service, gateway and virtualservice that I use.

Deployment

apiVersion: apps/v1
kind: Deployment
metadata:
  name: prometheus-flask-app-prometheus-flask-app
  labels:
    app: prometheus-flask-app
    chart: sbd-prometheus-flask-app-1.1
    release: prometheus-flask-app
    heritage: Helm
spec:
  replicas: 1
  selector:
    matchLabels:
      app: prometheus-flask-app
      release: prometheus-flask-app
  template:
    metadata:
      labels:
        app: prometheus-flask-app
        release: prometheus-flask-app
    spec:
      containers:
        - name: sbd-prometheus-flask-app
          image: flask-app:latest
          imagePullPolicy: Always
          ports:
            - containerPort: 5000
          livenessProbe:
            httpGet:
              path: /healthz
              port: 5000
            initialDelaySeconds: 600
          readinessProbe:
            httpGet:
              path: /healthz
              port: 5000
            initialDelaySeconds: 30
          resources:
            requests:
              cpu: 20m
              memory: 100Mi

Service

apiVersion: v1
kind: Service
metadata:
  name: prometheus-flask-app-prometheus-flask-app
  labels:
    app: prometheus-flask-app
    chart: sbd-prometheus-flask-app-1.1
    release: prometheus-flask-app
    heritage: Helm
spec:
  type: ClusterIP
  ports:
    - port: 80
      targetPort: 5000
      protocol: TCP
      name: sbd-prometheus-flask-app
  selector:
    app: prometheus-flask-app
    release: prometheus-flask-app

Gateway

apiVersion: networking.istio.io/v1alpha3
kind: Gateway
metadata:
  name: prometheus-flask-app-prometheus-flask-app-gateway
spec:
  selector:
    istio: ingressgateway # use istio default controller
  servers:
  - port:
      number: 80
      name: http
      protocol: HTTP
    hosts:
    - "*"

Virtual Service

apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  name: prometheus-flask-app-prometheus-flask-app-vs
spec:
  hosts:
  - "*"
  gateways:
  - prometheus-flask-app-prometheus-flask-app-gateway
  http:
  - route:
    - destination:
        host: prometheus-flask-app-prometheus-flask-app.knative-deployment.svc.cluster.local #knative-deployment is the name of the namespace
        port:
          number: 80
    retries:
      attempts: 3
      perTryTimeout: 5s
      retryOn: 5xx,gateway-error,connect-failure,refused-stream,reset,retriable-status-codes

ScreenShot of the istio-ingressgateway pod logs when scaling to 0

istio-logs
istio-logs1365×166 29.1 KB