Loki and Istio? Anyone?

#1

Hello!

Has anyone tried to install Loki/Promtail along with Istio in a K8s Cluster?

I am using Loki’s official Helm Charts:

I am using Envoy Sidecars for Loki and Promtail without any mTLS MeshPolicy or DestinationRule.

The ports from Loki are http ports:

 ports:
   - containerPort: 3101
     name: http-metrics

After the deployment i see in the Promtail logs:

level=error ts=2019-05-02T09:16:23.403938144Z caller=client.go:186 msg="final error sending batch" status=500 error="server returned HTTP status 500 Internal Server Error (500): rpc error: code = Unavailable desc = all SubConns are in TransientFailure, latest connection error: <nil>"

…and in the Loki logs:

level=warn ts=2019-05-02T09:17:40.236262511Z caller=logging.go:49 traceID=765972c977acb165 msg="POST /api/prom/push (500) 224.321µs Response: \"rpc error: code = Unavailable desc = all SubConns are in TransientFailure, latest connection error: <nil>\\n\" ws: false; Accept-Encoding: gzip; Content-Length: 1061; Content-Type: application/x-protobuf; User-Agent: Go-http-client/1.1; X-B3-Parentspanid: 0400748391a49ec0; X-B3-Sampled: 0; X-B3-Spanid: 6c7b2c533a4ce85f; X-B3-Traceid: 85c8078d4e6b88f50400748391a49ec0; X-Forwarded-Proto: http; X-Request-Id: d000a4ba-4996-4c14-a437-fa9891372647; "

So i was thinking maybe are the port names wrong because the protocol is grpc.

When i rename the ports to grpc-metrics i am getting in the Promtail logs:

level=warn ts=2019-05-02T09:23:37.146138007Z caller=client.go:181 msg="error sending batch, will retry" status=503 error="server returned HTTP status 503 Service Unavailable (503): upstream connect error or disconnect/reset before headers. reset reason: connection termination"

so what’s now? :slight_smile: