Health check with http/2

Config
1

Hello everyone,

I have a microservice that runs over http/2, my environment is configured with istio and mtls, I am trying to add health checks for the microservice following the guide here: Istio / Health Checking of Istio Services. When i check the logs from the sidecar container i get the message:

error Request to probe app failed: Get http://localhost:8081/health: net/http: HTTP/1.x transport connection broken: malformed HTTP response “\x00\x00\x00\x04\x00\x00\x00\x00\x00\x00\x00\b\a\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x02”, original URL path = /app-health/sample/readyz

initial probes look like this

 livenessProbe:
  httpGet:
    path: /health
    port: 8081
  ...
readinessProbe:
  httpGet:
    path: /health
    port: 8081
  ...

I have added the annotation sidecar.istio.io/rewriteAppHTTPProbers: “true” and the re-written values look like this

livenessProbe:
    httpGet:
      path: /app-health/sample/livez
      port: 15020
      scheme: HTTP
readinessProbe:
    httpGet:
      path: /app-health/sample/readyz
      port: 15020
      scheme: HTTP

I know that normally Kubernetes does not support probes over http/2 but I thought that maybe the sidecar can translate the protocol since the traffic already goes through it.

P.S: I keep reading that Envoy has First class HTTP/2 support and can operate as a transparent HTTP/1.1 to HTTP/2 proxy in both directions (see documentation) but I cannot find any guides on how to do so. It would be useful in case I have other 3rd party components that only work with http1.1.

2

Did you figure this out? I am seeing that problem now on 1.7.0. I did not have an issue on 1.5.1.