Out of process Mixer adapter to receive information about non-HTTP calls

Policies and Telemetry
1

Hello,

I’ve successfully implemented an out of process adapter for Mixer which receives instances with information about calls coming through Istio proxies. However, I’m not getting anything for non-HTTP calls (I’m testing with Redis non-SSL calls specifically). If I were to try to collect at least timestamp and source/destination IPs for such calls, how would I do that? Can I somehow instruct Envoy/Mixer to forward that information to me? Do I write a plugin for Envoy? What are my options here?

Thank you.

2

Hi!

OOPs works the same for all protocols. However, I don’t think we produce any redis-specific metadata, and it likely appears as a regular TCP traffic.

To enable the adapter for non-HTTP calls, you can take a look at an example stdio application here:

3

My rule looks like this:

# rule to dispatch to handler h1
apiVersion: "config.istio.io/v1alpha2"
kind: rule
metadata:
 name: r1
 namespace: istio-system
spec:
 actions:
 - handler: h1
   instances:
   - i1

As you can see, there is no match condition at all. Does this mean my adapter should be invoked for all traffic, including regular tcp? My instances are of the tracespan template if that matters.

What else can I check? I don’t think I’m seeing anything tcp-related in my adapter.

4

Hi!
No match condition means it is literally “true”, e.g. match always. What might be happening is that the instance construction for tracespan is failing. The example config https://istio.io/docs/reference/config/policy-and-telemetry/templates/tracespan/ uses attributes request.headers and request.time, for example, that are not defined by TCP traffic. You can confirm this by looking at mixer logs and looking for evaluation errors.

If you want to have a tracespan for TCP traffic, then you need to create a separate tracespan instance for TCP. Attributes from connection.* family are generally available for TCP traffic, as well as source and destination info. Then add a rule that applies your handle to the TCP instance if context protocol is TCP.

Best,
–kuat

5

You are exactly correct, Mixer logs are showing failing instance creation. I’ll try providing defaults for now…
Thanks!

6

Ok, I got it to work, I indeed get connection attributes, etc.

Are there any specific non-HTTP protocols supported in Istio right now? If I were to parse a certain TCP protocol that I’m particularly interested in, how would I go about doing that? Do I develop an Envoy plugin? If you could share some options I have I’d very much appreciate it.

7

Alex,

Yes, istio supplies several Envoy filters to collect the metadata from various protocols. We only have TCP and HTTP (+gRPC) filters right now, but envoy has a few others, thrift, mysql, redis, etc. The filters are available here https://github.com/istio/proxy/tree/master/src/envoy.

Istio filters are prepended to the generic envoy filter. They are configured through pilot, which knows about services and their ports. I can imagine a similar design would work for other protocols.

It is worth mentioning that the next gen architecture moves much of the machinery into envoy. That means you should expect istio filters in envoy to grow in complexity and configuration size. Given this plan, we’d need to be careful to support the new architecture in any new istio filter we add to envoy. I think a native envoy filter that exposes its collected data to other filters would be most flexible.

–kuat

8

Thank you very much for this information. A bunch of questions if you have time:

  1. What is an “Istio filter” exactly? For the HTTP filter, I see what appears to be a native Envoy component here (https://github.com/istio/proxy/blob/8455b9eb514c1a527fc4447951af00acd83104f5/src/envoy/http/mixer/filter.cc). Is there more to an Istio filter? How is it different from a regular native Envoy filter?

  2. Do I understand correctly that directly configuring Envoy filters through this configuration here (https://istio.io/docs/reference/config/networking/v1alpha3/envoy-filter/) is not a good long-term solution? It’s referred to as a break-glass configuration in the docs. What’s the appropriate way of configuring a filter then?

  3. Do I understand correctly that your preferred plan of actions would be roughly as follows?

  • pick a protocol to support (say, Redis)
  • write the native Envoy filter for that protocol (in case of Redis already exists). An implementation would be similar to the implementation for, say, HTTP filter found here (https://github.com/istio/proxy/blob/8455b9eb514c1a527fc4447951af00acd83104f5/src/envoy/http/mixer/filter.cc), except maybe it doesn’t directly send data to mixer through the mixer client, but rather generates attributes which other filters down the chain process and send.
  • write some sort of pilot-side configuration component which configures the Envoy-side filter (Is this needed? What would it look like?)

If there’s a documentation regarding this subject that I’m unable to find, please feel free to point me to it :slight_smile:

9

Hi!

  1. Istio filter is nothing more than a regular Envoy filter. We use a custom Envoy built with these filters, since they are not available from default Envoy build.

  2. “Envoy filter” is a bit of misnomer. It’s used to supply opaque Envoy config through istio config pipeline, so it by design breaks abstractions around Envoy encapsulation. If the goal is to add an entirely new protocol, then we’d need to modify the config pipeline itself:

It’s somewhat modularized since Mixer filters are configured using “plugins”.

  1. Yes, that sounds about right. The configuration component is Pilot from the link above. You could delay Mixer filter until after extending Pilot with another protocol. You can add networking features first, telemetry/policy second.

You are welcome to join Istio slack (istio.slack.com) for more questions about contributing to the project. Developer documentation quickly goes out of date, so it’s best to ask questions from active developers if you are confused.

1 Like