Stats/prometheus do not include my pod stats with enablePrometheusMerge=true

I describe the pod and while i would expect to find my metrics on localhost:15090/stats/prometheus, i only find the envoy / istio ones.
Why doesnt it include the localhost:1399/actuator/prometheus?

@howardjohn any clues?

  ISTIO_META_POD_PORTS:          [
                                     {"name":"http","containerPort":1399,"protocol":"TCP"}
                                 ]
  ISTIO_META_APP_CONTAINERS:     app
  ISTIO_META_CLUSTER_ID:         <some-non-default-id>
  ISTIO_META_INTERCEPTION_MODE:  REDIRECT
  ISTIO_METAJSON_ANNOTATIONS:    ...
                                 
  ISTIO_META_WORKLOAD_NAME:      dev-...
  ISTIO_META_OWNER:              ...
  ISTIO_META_MESH_ID:            <some-non-default-name>.local
  DNS_AGENT:                     
  ISTIO_PROMETHEUS_ANNOTATIONS:  {"scrape":"true","path":"/actuator/prometheus","port":"1399"}

I also changed istio-proxy logging to debug and i dont see anything about scraping, is that normal?

It is on port 15020. 15020 has the merged ones, 15090 has the envoy stats only.

Same deal unfortunately

iam@dev-iam-backend-cf4dbfb5d-k7hm5:/iam-be$ curl localhost:1399/actuator/prometheus -s | tail -3   
# TYPE jvm_buffer_count_buffers gauge
jvm_buffer_count_buffers{id="mapped",} 0.0
jvm_buffer_count_buffers{id="direct",} 13.0
iam@dev-iam-backend-cf4dbfb5d-k7hm5:/iam-be$ curl localhost:15020/stats/prometheus -s | grep jvm
iam@dev-iam-backend-cf4dbfb5d-k7hm5:/iam-be$

in fact it’s only istio / envoy metrics

iam@dev-iam-backend-cf4dbfb5d-k7hm5:/iam-be$ curl localhost:15020/stats/prometheus -s | grep -v envoy | grep -v istio | grep -v -e '^$'
iam@dev-iam-backend-cf4dbfb5d-k7hm5:/iam-be$ 

Can i somehow see the scraping job logs? I really tried to troubleshoot this, via configuration or log browsing but i could not find anything

@howardjohn it’s not the port, the metrics are just not scraped as you can see on my above post.
Is there any way to debug ? Some configuration to try? How can i see those scraping logs?