Need help to add authorization policy to filter internal and external calls

Security
1

Hi,

I must add authorization policy on my app to filter external access coming from ingress gateway.
This is my configuration :

  • k3s on premise (1 controller / 3 workers)
  • Istio installed with its default configuration
  • Nginx server as reverser proxy in front of my cluster. TLS termination on nginx and reverse to cluster in http
  • No RequestAuthentication is set

My app expose endpoints /api/internal/* for ms to ms communication and /api/external/* for client to ms communication.
This an example of my istio configuration :

---
apiVersion: networking.istio.io/v1alpha3
kind: Gateway
metadata:
  name: gtw-foobar
  namespace: myns
spec:
  selector:
    istio: ingressgateway
  servers:
    - hosts:
        - api.foo.bar
      port:
        name: http
        number: 80
        protocol: HTTP
---
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  name: vs-foobar
  namespace: myns
spec:
  gateways:
    - gtw-foobar
  hosts:
    - api.foo.bar
  http:
    - match:
        - uri:
            prefix: /api/
      route:
        - destination:
            host: foobar
            port:
              number: 8080
---
apiVersion: security.istio.io/v1beta1
kind: PeerAuthentication
metadata:
  name: peer-foobar
  namespace: myns
spec:
  mtls:
    mode: STRICT
---
apiVersion: security.istio.io/v1beta1
kind: AuthorizationPolicy
metadata:
  namespace: myns
  name: authz-foobar
spec:
  selector:
    matchLabels:
      app: foobar
  action: ALLOW
  rules:
    - from:
      - source:
          principals:
            - "cluster.local/ns/istio-system/sa/istio-ingressgateway-service-account"
      to:
      - operation:
          paths:
            - "/api/external/*"
    - from:
      - source:
          principals:
            - "cluster.local/ns/otherns/sa/otherapp-sa"
      to:
      - operation:
          paths:
            - "/api/internal/*"

Internal filter works well, but i have troubles with external communication. Proxy container logs on foobar pod :

2022-10-27T15:37:13.298992Z     debug   envoy rbac      checking request: requestedServerName: outbound_.8080_._.foobar.myns.svc.cluster.local, sourceIP: 10.42.2.1:10755, directRemoteIP: 10.42.2.1:10755, remoteIP: 10.42.2.1:10755,localAddress: 10.42.2.161:8080, ssl: uriSanPeerCertificate: spiffe://cluster.local/ns/otherns/sa/otherapp-sa, dnsSanPeerCertificate: , subjectPeerCertificate: , headers: ':authority', 'foobar.myns.svc.cluster.local:8080'
':path', '/api/internal/init'
':method', 'POST'
':scheme', 'http'
...
, dynamicMetadata:
2022-10-27T15:37:13.299060Z     debug   envoy rbac      enforced allowed, matched policy ns[lot2]-policy[authz-foobar]-rule[1]
[2022-10-27T15:37:13.298Z] "POST /api/internal/init HTTP/1.1" 200 - via_upstream - "-" 109 171 20 17 "-" "OpenAPI-Generator/0.9/go" "8ae3127f-a931-9af7-915f-9ca3c76ec88a" "foobar.myns.svc.cluster.local:8080
...
2022-10-27T15:37:13.419824Z     debug   envoy rbac      checking connection: requestedServerName: outbound_.8080_._.foobar.myns.svc.cluster.local, sourceIP: 10.42.2.1:29336, directRemoteIP: 10.42.2.1:29336,remoteIP: 10.42.2.1:29336, localAddress: 10.42.2.161:8080, ssl: uriSanPeerCertificate: spiffe://cluster.local/ns/istio-system/sa/istio-ingressgateway-service-account, dnsSanPeerCertificate: , subjectPeerCertificate: , dynamicMetadata:
2022-10-27T15:37:13.420090Z     debug   envoy rbac      enforced denied, matched policy none

It seems it didn’t match policy but i don’t understand why…
Ingressgateway logs :

[2022-10-27T15:57:16.111Z] "POST /api/internal/init HTTP/1.1" 503 UC upstream_reset_before_response_started{connection_termination} - "-" 84 95 3 - "xx.xx.xx.xx,10.42.6.0" "curl/7.68.0" "296f8e09-fa66-9a51-b38d-ac831b1c8857" "api.foo.bar" "10.42.2.161:8080" outbound|8080||foobar.myns.svc.cluster.local 10.42.3.203:33048 10.42.3.203:8080 10.42.6.0:64815 - -

I searched for a long time on forums and docs and tried a lot of things but i am stuck…
Any help would be appreciate :slight_smile:

2

If i trust this doc it seems ok to filter requests coming from ingressgateway.
If i remove path filter it allows all request coming from gateway :

  rules:
    - from:
      - source:
          principals:
            - "cluster.local/ns/istio-system/sa/istio-ingressgateway-service-account"
    - from:
      - source:
          principals:
            - "cluster.local/ns/otherns/sa/otherapp-sa"
      to:
      - operation:
          paths:
            - "/api/internal/*"

But this configuration is too open in my case. It seems not possible to filter with a path requests coming from ingress gateway…

3

I finally found the solution. My services didn’t have appProtocol set, so Istio treated all the traffic TCP instead of HTTP and so path not worked. It explains why in proxy logs no headers was print from ingress requests.
Doc

4

So, what are your suggestions?
How to create a policy to allow all internal and filter external traffic? Do we need to change operator values gfile and add appProtocol?

5

You just need to set appProtocol in your service to force istio use http instead of raw TCP

kind: Service
metadata:
  name: svc-foobar
spec:
  ports:
  - port: 8080
    name: foobarhttp
    appProtocol: http
6

Firstly, just happened to me and was lost a day to debug from first principles (using JWT auth) until I found this post. Struggled with the 503 upstream reset as an error, so posting here in case it helps someone find in search. would love to see some better logging / error messaging here to debug why the RBAC rule wasn’t triggering. Thanks for sharing your result.