[SOLVED] Ingress Stops Working When TLS is Activated

I have an Istio gateway setup that works with HTTP. When I do the same request with HTTPS, I get the following in the istio-ingressgateway pod’s logs:

[2022-04-04T13:25:32.373Z] "- - -" 0 NR filter_chain_not_found - "-" 0 0 0 - "-" "-" "-" "-" "-" - - 10.10.125.61:443 10.10.136.192:23181 - -

I’m struggling with this because I can’t seem to find a way to debug whats going on. Any help with my config or ideas and info on how to debug whats going on in my setup would be greatly appreciated.

Gateway

---
apiVersion: networking.istio.io/v1alpha3
kind: Gateway
metadata:
  name: my-gateway
  namespace: istio-ingress
spec:
  selector:
    istio: ingress
  servers:
    - port:
        number: 80
        name: http
        protocol: HTTP
      hosts:
        - "*.example.com"
    - port:
        number: 443 
        name: https
        protocol: HTTPS
      hosts:
        - "*.example.com"                                                                                                                                                                                      
        # - hello.example.com
        # - "*"
      tls:
        mode: SIMPLE
        credentialName: my-tls-secret

Virtual Service

apiVersion: networking.istio.io/v1beta1
kind: VirtualService
metadata:
  name: hello-world
  namespace: istio-test
spec:
  gateways:
  - istio-ingress/my-gateway
  hosts:
  - hello.example.com
  http:
  - route:
    - destination:
        host: hello-world.istio-test.svc.cluster.local
        port:
          number: 80

Service

apiVersion: v1
kind: Service
metadata:
  labels:
    app.kubernetes.io/instance: hello-world
    app.kubernetes.io/name: hello-world
  name: hello-world
  namespace: istio-test
spec:
  clusterIP: 10.104.144.110
  clusterIPs:
  - 10.104.144.110
  internalTrafficPolicy: Cluster
  ipFamilies:
  - IPv4
  ipFamilyPolicy: SingleStack
  ports:
  - name: http
    port: 80
    protocol: TCP
    targetPort: http
  selector:
    app.kubernetes.io/instance: hello-world
    app.kubernetes.io/name: hello-world
  sessionAffinity: None
  type: ClusterIP
status:
  loadBalancer: {}

Stateful Set

apiVersion: apps/v1                                
kind: StatefulSet                                  
metadata:                                          
  name: hello-world                                
  labels:                                          
    app.kubernetes.io/name: hello-world
spec:                                              
  replicas: 1                                      
  serviceName: hello-world                         
  selector:                                        
    matchLabels:                                   
      app.kubernetes.io/name: hello-world                                                             
      app.kubernetes.io/instance: release-name                                                        
  template:                                        
    metadata:                                      
      labels:                                      
        app.kubernetes.io/name: hello-world                                                           
        app.kubernetes.io/instance: release-name                                                      
    spec:
      containers:                                  
        - name: hello-world
          image: "crccheck/hello-world:v1.0.0"                                                        
          imagePullPolicy: IfNotPresent                                                               
          ports:                                   
            - containerPort: 8000
              name: http
              protocol: TCP

So, I thinking that there is something wrong with my cert or something wrong with how I’m setting it in k8s.

> $ istioctl pc secret -n istio-ingress istio-ingressgateway-69495c6667-rzrj9
RESOURCE NAME                 TYPE           STATUS      VALID CERT     SERIAL NUMBER                               NOT AFTER                NOT BEFORE
kubernetes://mss-aws-tls3                    WARMING     false                                                                               
kubernetes://mss-aws-tls2                    WARMING     false                                                                               
default                       Cert Chain     ACTIVE      true           4921712835651231667149651655120096330       2022-04-05T20:06:54Z     2022-04-04T20:04:54Z
ROOTCA                        CA             ACTIVE      true           124342703908942626121325323359959850853     2032-03-22T19:45:28Z     2022-03-25T19:45:28Z

I’ve tried multiple certs. I’ve tried creating the secret manually and I’ve tried to set via helm with base 64 encoded strings.

What am I doing wrong?

Finally figured it out. The issue was that the secret was in a different namespace than the gateway.