Getting connection_termination error on envoy sidecar on hitting GRPC endpoint

Networking
1

We have an application that is deployed to an eks cluster. This app is exposed to health-check on port 8080 and to a grpc endpoint on port 8888. I am successfully able to hit the health-check endpoint exposed at 8080 but on trying to run grpcurl -plaintext eks-test.my-app.abc.com:8888 list i am getting following error on envoy sidecar

{
    "x_forwarded_for":"10.164.21.112",
    "protocol":"HTTP/1.1",
    "authority":"eks-test.my-abc.com:8888",
    "start_time":"2023-06-21T20:15:50.453Z",
    "response_flags":"UC",
    "response_code_details":"upstream_reset_before_response_started{connection_termination}",
    "connection_termination_details":null,
    "upstream_host":"10.164.44.90:8080",
    "upstream_cluster":"inbound|8080||",
    "upstream_local_address":"127.0.0.6:35681",
    "requested_server_name":"outbound_.8080_._.my-app.gom.svc.cluster.local",
    "method":"POST",
    "request_id":"7e68bac0-83cb-9fd6-8b38-554dd52f9ea9",
    "upstream_transport_failure_reason":null,
    "response_code":200,
    "downstream_local_address":"10.164.44.90:8080",
    "downstream_remote_address":"10.164.21.112:0",
    "path":"/grpc.reflection.v1alpha.ServerReflection/ServerReflectionInfo",
    "upstream_service_time":null,
    "user_agent":"grpcurl/1.8.7 grpc-go/1.48.0",
    "route_name":"default",
    "bytes_sent":0
   }

I tried multiple things in my gateway, virtual-service and service resource but not able to make it work.

Here are the resources config

Deployment

apiVersion: apps/v1
kind: Deployment
metadata:
  annotations: {}
  labels:
    app: my-app
  name: my-app
  namespace: gom
spec:
  replicas: 1
  selector:
    matchLabels:
      app: my-app
  template:
    metadata:
      labels:
        app: my-app
    spec:
      affinity:
        nodeAffinity:
          requiredDuringSchedulingIgnoredDuringExecution:
            nodeSelectorTerms:
              - matchExpressions:
                  - key: nodegroup-name
                    operator: In
                    values:
                      - general
      containers:
        - envFrom:
            - configMapRef:
                name: package-orchestrator-app-env
          image: 'docker.repo.frg.tech/gom/my-app:0.0.1'
          imagePullPolicy: IfNotPresent
          livenessProbe:
            failureThreshold: 3
            initialDelaySeconds: 150
            periodSeconds: 10
            successThreshold: 1
            tcpSocket:
              port: rpc
            timeoutSeconds: 1
          name: my-app
          ports:
            - containerPort: 8080
              name: http
              protocol: TCP
            - containerPort: 8888
              name: rpc
              protocol: TCP
          resources:
            limits:
              cpu: 4
              memory: 4Gi
            requests:
              cpu: 2
              memory: 1Gi
      serviceAccountName: my-app

Service

apiVersion: v1
kind: Service
metadata:
  annotations: {}
  labels:
    app: my-app
  name: my-app
  namespace: gom
spec:
  ports:
    - name: http
      port: 8080
      targetPort: 8080
    - name: grpc-rpc
      port: 8888
      targetPort: 8888
      appProtocol: http2
  selector:
    app: my-app
  type: ClusterIP

Gateway

apiVersion: networking.istio.io/v1alpha3
kind: Gateway
metadata:
  labels:
    app: my-app
  name: my-app
  namespace: gom
spec:
  selector:
    istio: ingressgateway
  servers:
    - hosts:
        - eks-test.my-app.abc.com
      port:
        name: http
        number: 8080
        protocol: HTTP
      tls:
        httpsRedirect: true
    - hosts:
        - eks-test.my-app.abc.com
      port:
        name: https
        number: 8443
        protocol: HTTPS
      tls:
        credentialName: my-app
        mode: SIMPLE
    - hosts:
        - eks-test.my-app.abc.com
      port:
        name: grpc
        number: 8888
        protocol: GRPC

VirtualService

apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  labels:
    app: my-app
  name: my-app
  namespace: gom
spec:
  gateways:
    - my-app
  hosts:
    - eks-test.my-app.abc.com
  http:
    - match:
        - uri:
            prefix: /
      route:
        - destination:
            host: my-app.gom.svc.cluster.local
            port:
              number: 8080
    - match:
        - uri:
            prefix: /
      route:
        - destination:
            host: my-app.gom.svc.cluster.local
            port:
              number: 8888

Can I please get some assistance on my configuration ?