IP based blocks to specific services

I am trying to control access to certain services by IP. Some of the services behind gateway are management and I would like to add an additional layer of control by locking access to them by IP.

I am running on a GKE cluster with a L7 ingress load balancer.

I can apply an authorization policy at the gateway similar to the following and this work.

apiVersion: security.istio.io/v1beta1
kind: AuthorizationPolicy
metadata:
  name: auth-policy
  namespace: istio-system
spec:
  selector:
    matchLabels:
      istio: ingressgateway
  action: ALLOW
  rules:
  - from:
    - source:
        remoteIpBlocks: [
          <IP list>
        ]

However, I would like to limit the scope to specific destinations. We are wild carding the host name that enters the system (*.somedomain.com) and routing to the service based on the host. I should be able to block based on the host in the header, right?

I have tried adding a conditional component to the rule

apiVersion: security.istio.io/v1beta1
kind: AuthorizationPolicy
metadata:
  name: auth-policy
  namespace: istio-system
spec:
  selector:
    matchLabels:
      istio: ingressgateway
  action: ALLOW
  rules:
  - from:
    - source:
        remoteIpBlocks: [
          <IP list>
        ]
  - key: request.headers[Host]
      values: ["ne.somedomain.com"]

But it doesn’t seem to have any effect.

How would I debug this and why isn’t it working?

Thanks