Hey folks, I also posted this in the Stackdriver discussion forum, but haven’t had much joy there, so trying here too.
I’m trying to get:
- A log for inbound requests to istio-ingressgateway,
- with a nested HTTPRequest object attached to it,
- with specific MonitoredResource and MonitoredResourceDimensions,
- sent to Stackdriver,
- from a cluster running GKE’s istio add-on.
I first tried to do a simple access log, per documentation, but it seems as though there’s some internal process that causes the instance definition to keep being reset. Plus, it doesn’t seem easy to set the monitored resource type using this approach. Taking another tack, I put together the following logentry, rule, and handler:
apiVersion: config.istio.io/v1alpha2
kind: logentry
metadata:
labels:
k8s-app: istio
name: request-log-stackdriver
namespace: istio-system
spec:
monitored_resource_type: '"project"'
monitored_resource_dimensions:
project_id: '"pinian-app"'
severity: '"Default"'
timestamp: request.time
variables:
api_claims: request.auth.raw_claims | ""
api_key: request.api_key | request.headers["x-api-key"] | ""
api_name: api.service | ""
api_version: api.version | ""
client_trace_id: request.headers["x-client-trace-id"] | ""
destination_app: destination.labels["app"] | ""
destination_ip: destination.ip | ip("0.0.0.0")
destination_name: destination.name | ""
destination_namespace: destination.namespace | ""
destination_owner: destination.owner | ""
destination_principal: destination.principal | ""
destination_service_host: destination.service.host | ""
destination_service_name: destination.service.name | ""
destination_service_namespace: destination.service.namespace | ""
destination_uid: destination.uid | ""
destination_workload: destination.workload.name | ""
latency: response.duration | "0ms"
method: request.method | ""
protocol: request.scheme | context.protocol | "http"
received_bytes: request.total_size | 0
referer: request.referer | ""
request_id: request.headers["x-request-id"] | ""
request_operation: conditional((context.protocol | "unknown") == "grpc", request.path | "unknown", request.method | "unknown")
request_size: request.size | 0
response_code: response.code | 0
response_size: response.size | 0
response_timestamp: response.time
sent_bytes: response.total_size | 0
service_authentication_policy:
conditional((context.reporter.kind | "inbound")
== "outbound", "unknown", conditional(connection.mtls | false, "mutual_tls",
"none"))
source_app: source.labels["app"] | ""
source_ip: source.ip | ip("0.0.0.0")
source_name: source.name | ""
source_namespace: source.namespace | ""
source_owner: source.owner | ""
source_principal: source.principal | ""
source_uid: source.uid | ""
source_workload: source.workload.name | ""
trace_id: request.headers["x-b3-traceid"]
url: request.path | ""
user_agent: request.useragent | ""
service: "pinian-gateway"
---
apiVersion: config.istio.io/v1alpha2
kind: handler
metadata:
name: stackdriver
namespace: istio-system
spec:
compiledAdapter: stackdriver
params:
projectId: "pinian-app"
pushInterval: "1s"
appCredentials: true
logInfo:
request-log-stackdriver.logentry.istio-system:
payloadTemplate: '"HELLO WORLD"'
httpMapping:
status: response_code
requestSize: request_size
responseSize: response_size
latency: latency
url: url
method: method
userAgent: user_agent
referer: referer
labelNames:
- source_uid
- source_ip
- source_app
- source_principal
- source_name
- source_workload
- source_namespace
- source_owner
- destination_uid
- destination_app
- destination_ip
- destination_service_host
- destination_service_name
- destination_service_namespace
- destination_workload
- destination_name
- destination_namespace
- destination_owner
- destination_principal
- api_name
- api_version
- api_claims
- api_key
- request_operation
- protocol
- method
- url
- response_code
- response_size
- request_size
- request_id
- client_trace_id
- latency
- service_authentication_policy
- user_agent
- response_timestamp
- received_bytes
- sent_bytes
- referer
- trace_id
trace:
sampleProbability: 100
---
apiVersion: config.istio.io/v1alpha2
kind: rule
metadata:
labels:
k8s-app: istio
name: stackdriver-request-log
namespace: istio-system
spec:
actions:
- handler: stackdriver.handler
instances:
- request-log-stackdriver.logentry
match: "true"
This also does not work. The main roadblock I have now is that this doesn’t actually give any logs anywhere that I can see, so I don’t know if it’s a configuration problem, an auth problem, or something else.
It would be great if someone else who has tried to do something similar could share their approach. But, as a first step, how do I even go about debugging what’s wrong with this?