Ingress gateway IP whitelist with AuthorizationPolicy

Thanks for the documentation. I was successful in restricting traffic for all workload that use the ingress gateay. However I am looking to restrict traffic for a single workload and allow trafic for the rest. Any recommended way to do this?