Authorization policy istio 1.4+ is not letting whitelist service identity such as service account, namespace for traffic coming from non-mesh

Security
1

As discussed here Investigate authorization policy blocking prometheus scraping metrics at port 15090 · Issue #19975 · istio/istio · GitHub, I am facing similar issue.
Only processing namespace has istio-injection=enabled. prometheus is installed using stable operator charts and we use service monitors. we have ingress gateway installed in istio ingressgateway installed in istio-system. We upgraded istio from 1.3.7 to 1.4+
I am trying to whitelist these three types of traffic on top of deny-all.

apiVersion: "security.istio.io/v1beta1"
    kind: "AuthorizationPolicy"
    metadata:
      name: sender
      namespace: processing
    spec:
      selector:
        matchLabels:
          app: sender
      rules:
        - from:
          - source:
              principals: [
                **"cluster.local/ns/test/collector"**
               **"cluster.local/ns/prometheus/sa/default",**
               **"cluster.local/ns/istio-system/ingress-gw"**
              ]
    ---
    apiVersion: security.istio.io/v1beta1
    kind: AuthorizationPolicy
    metadata:
      name: deny-all
      namespace: processing
    spec:
      {}
    ---
    apiVersion: "authentication.istio.io/v1alpha1"
    kind: "Policy"
    metadata:
      name: "default"
      namespace: "processing"
    spec:
      peers:
      - mtls: {}

---
apiVersion: "networking.istio.io/v1alpha3"
kind: "DestinationRule"
metadata:
  name: "istio-mutual"
  namespace: "processing"
spec:
  host: "*.processing.svc.cluster.local"
  trafficPolicy:
    tls:
      mode: ISTIO_MUTUAL
---
apiVersion: "authentication.istio.io/v1alpha1"
kind: "Policy"
metadata:
  name: "istio-ingress-to-svc"
  namespace: "istio-system"
spec:
  targets:
  - name: ingress-gw
  peers:
  - mtls: {}

---
apiVersion: "networking.istio.io/v1alpha3"
kind: "DestinationRule"
metadata:
  name: "istio-mutual"
  namespace: "istio-system"
spec:
  host: "*.svc.cluster.local"
  trafficPolicy:
    tls:
      mode: ISTIO_MUTUAL

Result:
debug logs say, enforced denied for traffic coming from
“cluster.local/ns/prometheus/sa/default” and “cluster.local/ns/istio-system/ingress-gw”.
but, traffic from “cluster.local/ns/test/collector” is allowed as expected.

I also tried whitelisting entire namespaces like this -
rules:
- from:
- source:
namespaces: [“prometheus”, “istio-system”]

but it does not work. Has anybody faced and solved this problem without whitelisting ports as mentioned in the github issue ?

2

can you copy/paste the full envoy debug logs?

And if only processing namespace has istio-injection=enabled, do you mean your prometheus does not have sidecar? If so it won’t work because it does not have the mTLS identity, and then you cannot use the cluster.local/ns/prometheus/sa/default in the authz policy.

3

I will share logs soon. However, I do understand why it will not work with prometheus. but why does this not work with cluster.local/ns/istio-system/sa/ingress-gw?? istio ingress gateway would have the mTLS identity right?