I am using Zipkin with a Java springboot application to store my traces. We also send a custom header called xxx-deviceid: abc-windows-10 in our requests, however this particular header never reaches the Zipkin system when we use Istio ingress gateway in our data path. It however works when we directly use ClusterIP of the service without a sidecar or even with a sidecar proxy. But when I make the exact same request using ingress gateway nodeport/clusterIP this header is no longer a part of trace we see in Zipkin
I also tried with a very dumb goLang server without any tracing logic and it seems to receive the header just fine. How can I debug this strange behaviour further?