EnvoyFilter workload selection on runtime change of pod labels

Howdy all,
(istio 1.7.3)
Im trying to apply an EnvoyFilter to an existing Pod with something like

apiVersion: networking.istio.io/v1alpha3
kind: EnvoyFilter
metadata:
  name: testfilter
  namespace: foo
spec:
  workloadSelector:
    labels:
      simplelabel: abcd

And then adding the label to the running Pod
But the istio operator doesn’t seem to pick up the change. As well if I remove labels that were there when the Pod was started it doest seem to started it doesn’t react and remove EnvoyFilters already applied to that Pods proxy.

Ive also tried adding the label to the deployment and then adjusting the value only of the label to match the workload selector and that doesn’t seem to work either.

Does istio only interact with the labels as they are set at pod creation time, or is the workload selection happening on a different object?