IP Whitelisting for Single Service

Hey everyone, so I have three deployments in my kubernetes cluster. Each has its own gateway and virtualservice, all linked to the ingressgateway. However, for one of my deployments, I only want two IPs to be able to access it as it’s an admin panel, whereas the other two services I want to be completely public. I can get IP Whitelisting working for all services by applying authorization rules to the ingressgateway, but I’ve looked through the docs and looked everywhere online, but I can’t seem to find an answer to IP whitelisting a single service. Is it possible to do this using Istio? Or will I need to create another ingressgateway connected to a loadbalancer just for the one service I want to IP whitelist for?