When we use mtls in permissive mode Prometheus which is deployed with istio can get our applications’ metrics. When we change the mtls to strict mode, our applications listed as down in prometheus targets like this;
Get http://{applicatonIP}:8081/actuator/prometheus: read tcp {PrometheusIP}:53168->{applicatonIP}:8081: read: connection reset by peer.
Is there any additional, specific configuration for prometheus deployment to get applicaton metrics in strict mtls ?
Our cluster is Openshift 4.5.5
ServiceMesh 1.1.7 based on openshift documentation our istio version is 1.4.8
In prometheus deployment, prometheus args;
args:
- '-provider=openshift'
- '-https-address=:3001'
- '-http-address='
- '-email-domain=*'
- '-upstream=http://localhost:9090'
- '-htpasswd-file=/etc/proxy/htpasswd/auth'
- '-display-htpasswd-form=false'
- >-
-openshift-sar={"namespace": "istio-system", "resource": "pods",
"verb": "get"}
- >-
-client-secret-file=/var/run/secrets/kubernetes.io/serviceaccount/token
- '-openshift-service-account=prometheus'
- '-cookie-secret-file=/etc/proxy/secrets/session_secret'
- '-tls-cert=/etc/tls/private/tls.crt'
- '-tls-key=/etc/tls/private/tls.key'
- '-openshift-ca=/etc/pki/tls/cert.pem'
- '-openshift-ca=/var/run/secrets/kubernetes.io/serviceaccount/ca.crt'
In prometheus deployment, volumeMounts;
volumeMounts:
- name: config-volume
mountPath: /etc/prometheus
- name: istio-certs
mountPath: /etc/istio-certs
In prometheus deployment, volume definition;
- name: istio-certs
secret:
secretName: istio.default
defaultMode: 420