How to limit access to Helm Chart based on JWT token through Istio

We have an application where users can deploy Helm Charts into their own kubernetes namespace. We need a way to only allow that user to access these Helm Charts. All we have for the user is a JWT token. One idea we had was to create a Virtual Service programmatically (Java) for each Helm Chart deployed (also done through Java api to kubernetes) and have the Helm Chart available through a URL including the users ID, and then somehow (maybe Authorization Policy) lockdown the url based on the users token? Does anyone else have experience with doing this? or maybe an expert knows the route I need to go? Not sure if the Virtual Service and Authorization Policy together is enough.