Istio 1.8: C++ WASM filter not working

1

Hi,
I tried to deploy a WASM filter as shown in banzaicloud.com/blog/envoy-wasm-filter/ and it is not working(not seeing anything in the logs).
I tried enabling trace logging, but I neither see any errors w.r.t WASM filter, nor I see any filter invocation related logs.
I am able to:

  1. Generate the WASM binaries
  2. Create the config map
  3. Validate the volume mounts containing the wasm binaries in the istio-proxy container
  4. Create the Envoy Filter
  5. Envoy filter is updated in the config_dump

EnvoyFilter:

kind: EnvoyFilter
metadata:
  name: example-wasm-filter
  namespace: service-mesh
spec:
  configPatches:
    - applyTo: HTTP_FILTER
      match:
        context: SIDECAR_INBOUND
        proxy:
          proxyVersion: '^1\.8.*'
        listener:
          filterChain:
            filter:
              name: "envoy.http_connection_manager"
              subFilter:
                name: envoy.router
      patch:
        operation: INSERT_BEFORE
        value:
          name: example_filter
          typed_config:
            "@type": type.googleapis.com/udpa.type.v1.TypedStruct
            type_url: type.googleapis.com/envoy.extensions.filters.http.wasm.v3.Wasm
            value:
              config:
                configuration:
                  "@type": "type.googleapis.com/google.protobuf.StringValue"
                  value: |
                    {}
                rootId: my_root_id
                vm_config:
                  runtime: envoy.wasm.runtime.v8
                  vmId: my_root_id
                  allow_precompiled: true
                  code:
                    local:
                      filename: /var/local/lib/wasm-filters/example-wasm-filter.wasm
  workloadSelector:
    labels:
      app: istio-example 
EOF

Am I missing something in this filter?

1 Like
2

C++ code

class ExampleRootContext : public RootContext {
public:
  explicit ExampleRootContext(uint32_t id, std::string_view root_id) : RootContext(id, root_id) {}

  bool onStart(size_t) override;
  bool onConfigure(size_t) override;
  void onTick() override;
};

class ExampleContext : public Context {
public:
  explicit ExampleContext(uint32_t id, RootContext* root) : Context(id, root) {}

  void onCreate() override;
  FilterHeadersStatus onRequestHeaders(uint32_t headers, bool end_of_stream) override;
  FilterDataStatus onRequestBody(size_t body_buffer_length, bool end_of_stream) override;
  FilterHeadersStatus onResponseHeaders(uint32_t headers, bool end_of_stream) override;
  void onDone() override;
  void onLog() override;
  void onDelete() override;
};
static RegisterContextFactory register_ExampleContext(CONTEXT_FACTORY(ExampleContext),
                                                      ROOT_FACTORY(ExampleRootContext),
                                                      "my_root_id");

bool ExampleRootContext::onStart(size_t) {
  LOG_TRACE("onStart");
  return true;
}

bool ExampleRootContext::onConfigure(size_t) {
  std::cout << "In onConfigure.\n";
  LOG_TRACE("onConfigure");
  proxy_set_tick_period_milliseconds(1000); // 1 sec
  return true;
}

void ExampleRootContext::onTick() { LOG_TRACE("onTick"); }

void ExampleContext::onCreate() { LOG_WARN(std::string("onCreate " + std::to_string(id()))); }

FilterHeadersStatus ExampleContext::onRequestHeaders(uint32_t, bool) {
  std::cout << "In onRequestHeaders.\n";
  LOG_DEBUG(std::string("onRequestHeaders ") + std::to_string(id()));
  auto result = getRequestHeaderPairs();
  auto pairs = result->pairs();
  LOG_INFO(std::string("headers: ") + std::to_string(pairs.size()));
  for (auto& p : pairs) {
    LOG_INFO(std::string(p.first) + std::string(" -> ") + std::string(p.second));
  }
  return FilterHeadersStatus::Continue;
}

FilterHeadersStatus ExampleContext::onResponseHeaders(uint32_t, bool) {
  std::cout << "In onResponseHeaders.\n";
  LOG_DEBUG(std::string("onResponseHeaders ") + std::to_string(id()));
  auto result = getResponseHeaderPairs();
  auto pairs = result->pairs();
  LOG_INFO(std::string("headers: ") + std::to_string(pairs.size()));
  for (auto& p : pairs) {
    LOG_INFO(std::string(p.first) + std::string(" -> ") + std::string(p.second));
  }
  addResponseHeader("newheader", "newheadervalue");
  replaceResponseHeader("location", "envoy-wasm");
  return FilterHeadersStatus::Continue;
}

FilterDataStatus ExampleContext::onRequestBody(size_t body_buffer_length,
                                               bool /* end_of_stream */) {
  std::cout << "In onRequestBody.\n";
  auto body = getBufferBytes(WasmBufferType::HttpRequestBody, 0, body_buffer_length);
  LOG_ERROR(std::string("onRequestBody ") + std::string(body->view()));
  return FilterDataStatus::Continue;
}

void ExampleContext::onDone() { 
  std::cout << "In onDone.\n";
  LOG_WARN(std::string("onDone " + std::to_string(id()))); 
}

void ExampleContext::onLog() { 
  std::cout << "In onLog.\n";
  LOG_WARN(std::string("onLog " + std::to_string(id()))); 
}

void ExampleContext::onDelete() { LOG_WARN(std::string("onDelete " + std::to_string(id()))); }

I also see below stats:
wasm.envoy.wasm.runtime.v8.active: 4
wasm.envoy.wasm.runtime.v8.created: 5
wasm_vm.null.active: 4
wasm_vm.null.created: 5

3

Can you share your envoy config dump?

4

Please let me know if this is good. Example is renamed to Shipping and it is a grpc endpoint
{
“name”: “virtualInbound”,
“active_state”: {
“version_info”: “2020-10-01T06:00:32Z/197”,
“listener”: {
“@type”: “type.googleapis.com/envoy.config.listener.v3.Listener”,
“name”: “virtualInbound”,
“address”: {
“socket_address”: {
“address”: “0.0.0.0”,
“port_value”: 15006
}
},
“filter_chains”: [
{
“filter_chain_match”: {
“prefix_ranges”: [
{
“address_prefix”: “0.0.0.0”,
“prefix_len”: 0
}
],
“transport_protocol”: “tls”
},
“filters”: [
{
“name”: “envoy.filters.network.ext_authz”,
“typed_config”: {
“@type”: “type.googleapis.com/envoy.extensions.filters.network.ext_authz.v3.ExtAuthz”,
“stat_prefix”: “authz”,
“grpc_service”: {
“google_grpc”: {
“target_uri”: “127.0.0.1:9191”,
“stat_prefix”: “ext_authz”
},
“timeout”: “1s”
}
}
},
{
“name”: “envoy.filters.network.tcp_proxy”,
“typed_config”: {
“@type”: “type.googleapis.com/envoy.extensions.filters.network.tcp_proxy.v3.TcpProxy”,
“stat_prefix”: “InboundPassthroughClusterIpv4”,
“cluster”: “InboundPassthroughClusterIpv4”
}
}
],
“transport_socket”: {
“name”: “envoy.transport_sockets.tls”,
“typed_config”: {
“@type”: “type.googleapis.com/envoy.extensions.transport_sockets.tls.v3.DownstreamTlsContext”,
“common_tls_context”: {
“alpn_protocols”: [
“istio-peer-exchange”,
“h2”,
“http/1.1”
],
“tls_certificate_sds_secret_configs”: [
{
“name”: “file-cert:/etc/identity/server/certificates/server.pem~/etc/identity/server/keys/server-key.pem”,
“sds_config”: {
“api_config_source”: {
“api_type”: “GRPC”,
“grpc_services”: [
{
“envoy_grpc”: {
“cluster_name”: “sds-grpc”
}
}
],
“transport_api_version”: “V3”
},
“resource_api_version”: “V3”
}
}
],
“combined_validation_context”: {
“default_validation_context”: {},
“validation_context_sds_secret_config”: {
“name”: “file-root:/etc/identity/ca/cacerts.pem”,
“sds_config”: {
“api_config_source”: {
“api_type”: “GRPC”,
“grpc_services”: [
{
“envoy_grpc”: {
“cluster_name”: “sds-grpc”
}
}
],
“transport_api_version”: “V3”
},
“resource_api_version”: “V3”
}
}
}
},
“require_client_certificate”: true
}
},
“name”: “virtualInbound”
},
{
“filter_chain_match”: {
“prefix_ranges”: [
{
“address_prefix”: “0.0.0.0”,
“prefix_len”: 0
}
],
“transport_protocol”: “tls”,
“application_protocols”: [
“http/1.0”,
“http/1.1”,
“h2c”,
“istio-http/1.0”,
“istio-http/1.1”,
“istio-h2”
]
},
“filters”: [
{
“name”: “envoy.filters.network.ext_authz”,
“typed_config”: {
“@type”: “type.googleapis.com/envoy.extensions.filters.network.ext_authz.v3.ExtAuthz”,
“stat_prefix”: “authz”,
“grpc_service”: {
“google_grpc”: {
“target_uri”: “127.0.0.1:9191”,
“stat_prefix”: “ext_authz”
},
“timeout”: “1s”
}
}
},
{
“name”: “envoy.filters.network.http_connection_manager”,
“typed_config”: {
“@type”: “type.googleapis.com/envoy.extensions.filters.network.http_connection_manager.v3.HttpConnectionManager”,
“stat_prefix”: “InboundPassthroughClusterIpv4”,
“route_config”: {
“name”: “InboundPassthroughClusterIpv4”,
“virtual_hosts”: [
{
“name”: “inbound|http|0”,
“domains”: [
“"
],
“routes”: [
{
“match”: {
“prefix”: “/”
},
“route”: {
“cluster”: “InboundPassthroughClusterIpv4”,
“timeout”: “0s”,
“max_grpc_timeout”: “0s”
},
“decorator”: {
“operation”: ":0/”
},
“name”: “default”
}
]
}
],
“validate_clusters”: false
},
“http_filters”: [
{
“name”: “envoy.filters.http.cors”,
“typed_config”: {
“@type”: “type.googleapis.com/envoy.extensions.filters.http.cors.v3.Cors”
}
},
{
“name”: “envoy.filters.http.fault”,
“typed_config”: {
“@type”: “type.googleapis.com/envoy.extensions.filters.http.fault.v3.HTTPFault”
}
},
{
“name”: “shipping_filter”,
“typed_config”: {
“@type”: “type.googleapis.com/udpa.type.v1.TypedStruct”,
“type_url”: “type.googleapis.com/envoy.extensions.filters.http.wasm.v3.Wasm”,
“value”: {
“config”: {
“root_id”: “my_root_id”,
“vm_config”: {
“vm_id”: “my_root_id”,
“runtime”: “envoy.wasm.runtime.v8”,
“code”: {
“local”: {
“filename”: “/var/local/lib/wasm-filters/shipping-wasm-filter.wasm”
}
},
“allow_precompiled”: true
},
“configuration”: {
“@type”: “type.googleapis.com/google.protobuf.StringValue”,
“value”: “{}\n”
}
}
}
}
},
{
“name”: “envoy.filters.http.router”,
“typed_config”: {
“@type”: “type.googleapis.com/envoy.extensions.filters.http.router.v3.Router”
}
}
],
“tracing”: {
“client_sampling”: {
“value”: 100
},
“random_sampling”: {
“value”: 1
},
“overall_sampling”: {
“value”: 100
}
},
“server_name”: “istio-envoy”,
“use_remote_address”: false,
“generate_request_id”: true,
“forward_client_cert_details”: “APPEND_FORWARD”,
“set_current_client_cert_details”: {
“subject”: true,
“dns”: true,
“uri”: true
},
“upgrade_configs”: [
{
“upgrade_type”: “websocket”
}
],
“stream_idle_timeout”: “0s”,
“normalize_path”: true
}
}
],
“transport_socket”: {
“name”: “envoy.transport_sockets.tls”,
“typed_config”: {
“@type”: “type.googleapis.com/envoy.extensions.transport_sockets.tls.v3.DownstreamTlsContext”,
“common_tls_context”: {
“alpn_protocols”: [
“h2”,
“http/1.1”
],
“tls_certificate_sds_secret_configs”: [
{
“name”: “file-cert:/etc/identity/server/certificates/server.pem~/etc/identity/server/keys/server-key.pem”,
“sds_config”: {
“api_config_source”: {
“api_type”: “GRPC”,
“grpc_services”: [
{
“envoy_grpc”: {
“cluster_name”: “sds-grpc”
}
}
],
“transport_api_version”: “V3”
},
“resource_api_version”: “V3”
}
}
],
“combined_validation_context”: {
“default_validation_context”: {},
“validation_context_sds_secret_config”: {
“name”: “file-root:/etc/identity/ca/cacerts.pem”,
“sds_config”: {
“api_config_source”: {
“api_type”: “GRPC”,
“grpc_services”: [
{
“envoy_grpc”: {
“cluster_name”: “sds-grpc”
}
}
],
“transport_api_version”: “V3”
},
“resource_api_version”: “V3”
}
}
}
},
“require_client_certificate”: true
}
},
“name”: “virtualInbound-catchall-http”
},
{
“filter_chain_match”: {
“destination_port”: 7443
},
“filters”: [
{
“name”: “envoy.filters.network.ext_authz”,
“typed_config”: {
“@type”: “type.googleapis.com/envoy.extensions.filters.network.ext_authz.v3.ExtAuthz”,
“stat_prefix”: “authz”,
“grpc_service”: {
“google_grpc”: {
“target_uri”: “127.0.0.1:9191”,
“stat_prefix”: “ext_authz”
},
“timeout”: “1s”
}
}
},
{
“name”: “envoy.filters.network.http_connection_manager”,
“typed_config”: {
“@type”: “type.googleapis.com/envoy.extensions.filters.network.http_connection_manager.v3.HttpConnectionManager”,
“stat_prefix”: “inbound_0.0.0.0_7443”,
“route_config”: {
“name”: “inbound|7443|grpc-shipping|istio-shipping.service-mesh.svc.cluster.local”,
“virtual_hosts”: [
{
“name”: “inbound|http|7443”,
“domains”: [
“"
],
“routes”: [
{
“match”: {
“prefix”: “/”
},
“route”: {
“cluster”: “inbound|7443|grpc-shipping|istio-shipping.service-mesh.svc.cluster.local”,
“timeout”: “0s”,
“max_grpc_timeout”: “0s”
},
“decorator”: {
“operation”: "istio-shipping.service-mesh.svc.cluster.local:7443/”
},
“name”: “default”
}
]
}
],
“validate_clusters”: false
},
“http_filters”: [
{
“name”: “istio_authn”,
“typed_config”: {
“@type”: “type.googleapis.com/istio.envoy.config.filter.http.authn.v2alpha1.FilterConfig”,
“policy”: {
“peers”: [
{
“mtls”: {}
}
]
},
“skip_validate_trust_domain”: true
}
},
{
“name”: “envoy.filters.http.grpc_stats”,
“typed_config”: {
“@type”: “type.googleapis.com/envoy.extensions.filters.http.grpc_stats.v3.FilterConfig”,
“emit_filter_state”: true,
“stats_for_all_methods”: false
}
},
{
“name”: “envoy.filters.http.cors”,
“typed_config”: {
“@type”: “type.googleapis.com/envoy.extensions.filters.http.cors.v3.Cors”
}
},
{
“name”: “envoy.filters.http.fault”,
“typed_config”: {
“@type”: “type.googleapis.com/envoy.extensions.filters.http.fault.v3.HTTPFault”
}
},
{
“name”: “shipping_filter”,
“typed_config”: {
“@type”: “type.googleapis.com/udpa.type.v1.TypedStruct”,
“type_url”: “type.googleapis.com/envoy.extensions.filters.http.wasm.v3.Wasm”,
“value”: {
“config”: {
“root_id”: “my_root_id”,
“vm_config”: {
“vm_id”: “my_root_id”,
“runtime”: “envoy.wasm.runtime.v8”,
“code”: {
“local”: {
“filename”: “/var/local/lib/wasm-filters/shipping-wasm-filter.wasm”
}
},
“allow_precompiled”: true
},
“configuration”: {
“@type”: “type.googleapis.com/google.protobuf.StringValue”,
“value”: “{}\n”
}
}
}
}
},
{
“name”: “envoy.filters.http.router”,
“typed_config”: {
“@type”: “type.googleapis.com/envoy.extensions.filters.http.router.v3.Router”
}
}
],
“tracing”: {
“client_sampling”: {
“value”: 100
},
“random_sampling”: {
“value”: 1
},
“overall_sampling”: {
“value”: 100
}
},
“http2_protocol_options”: {},
“server_name”: “istio-envoy”,
“use_remote_address”: false,
“generate_request_id”: true,
“forward_client_cert_details”: “APPEND_FORWARD”,
“set_current_client_cert_details”: {
“subject”: true,
“dns”: true,
“uri”: true
},
“upgrade_configs”: [
{
“upgrade_type”: “websocket”
}
],
“stream_idle_timeout”: “0s”,
“normalize_path”: true
}
}
],
“transport_socket”: {
“name”: “envoy.transport_sockets.tls”,
“typed_config”: {
“@type”: “type.googleapis.com/envoy.extensions.transport_sockets.tls.v3.DownstreamTlsContext”,
“common_tls_context”: {
“alpn_protocols”: [
“h2”,
“http/1.1”
],
“tls_certificate_sds_secret_configs”: [
{
“name”: “file-cert:/etc/identity/server/certificates/server.pem~/etc/identity/server/keys/server-key.pem”,
“sds_config”: {
“api_config_source”: {
“api_type”: “GRPC”,
“grpc_services”: [
{
“envoy_grpc”: {
“cluster_name”: “sds-grpc”
}
}
],
“transport_api_version”: “V3”
},
“resource_api_version”: “V3”
}
}
],
“combined_validation_context”: {
“default_validation_context”: {},
“validation_context_sds_secret_config”: {
“name”: “file-root:/etc/identity/ca/cacerts.pem”,
“sds_config”: {
“api_config_source”: {
“api_type”: “GRPC”,
“grpc_services”: [
{
“envoy_grpc”: {
“cluster_name”: “sds-grpc”
}
}
],
“transport_api_version”: “V3”
},
“resource_api_version”: “V3”
}
}
}
},
“require_client_certificate”: true
}
},
“name”: “0.0.0.0_7443”
}
],
“listener_filters”: [
{
“name”: “envoy.filters.listener.original_dst”,
“typed_config”: {
“@type”: “type.googleapis.com/envoy.extensions.filters.listener.original_dst.v3.OriginalDst”
}
},
{
“name”: “envoy.filters.listener.tls_inspector”,
“typed_config”: {
“@type”: “type.googleapis.com/envoy.extensions.filters.listener.tls_inspector.v3.TlsInspector”
}
},
{
“name”: “envoy.filters.listener.http_inspector”,
“typed_config”: {
“@type”: “type.googleapis.com/envoy.extensions.filters.listener.http_inspector.v3.HttpInspector”
}
}
],
“listener_filters_timeout”: “1s”,
“traffic_direction”: “INBOUND”,
“continue_on_listener_filters_timeout”: true
},
“last_updated”: “2020-10-01T09:28:31.988Z”
}
}

5

I saw the same issue with Istio 1.7.3 using the EnvoyFilter and C++ WASM filter. The config_dump shows the WASM EnvoyFilter patch has been applied and loaded into Envoy successfully. But the LOG_INFO/LOG_TRACE/LOG_DEBUG calls in the onRequestHeaders etc. APIs was not showing up in istio-proxy log, adding of the test response header is not taking effect.

6

The example does not build against 1.8 proxy:

extensions/example.cc:14:51: error: non-virtual member function marked 'override' hides virtual member function
  FilterHeadersStatus onResponseHeaders(uint32_t) override;
                                                  ^
external/proxy_wasm_cpp_sdk/proxy_wasm_api.h:470:31: note: hidden overloaded virtual function 'Context::onResponseHeaders' declared here: different number of parameters (2 vs 1)
  virtual FilterHeadersStatus onResponseHeaders(uint32_t, bool) {
                              ^

I suspect ABI changed which renders 1.7 wasm files incompatible with 1.8.

7

Yes. That example won’t compile with latest SDK. You can use the example in https://github.com/envoyproxy/envoy-wasm/blob/e2aca05ea5bb5301a081b642e30d50f409f8398e/examples/wasm/envoy_filter_http_wasm_example.cc

8

@rakella: Can you try with Istio 1.7.2 version as compared to master. Master is under development and thus there could be discrepancies that you will see… Keep versions same for envoy, proxy and istio…

9

@gargnupur I need to troubleshoot this in istio 1.8 only as I don’t have control on the istio version in that environment.
@kuat I tried with proxy_abi_version_0_2_1(current in master) and proxy_abi_version_0_2_0 and the result is same.
I do see some logs which indicate that the vm is alive and initialized:

2020-10-05T10:51:21.856021Z	debug	envoy filter	original_dst: New connection accepted
2020-10-05T10:51:21.856050Z	debug	envoy filter	tls inspector: new connection accepted
2020-10-05T10:51:21.856084Z	debug	envoy filter	http inspector: new connection accepted
2020-10-05T10:51:25.241240Z	debug	envoy wasm	WasmVm created envoy.wasm.runtime.v8 now active
2020-10-05T10:51:25.241492Z	debug	envoy wasm	Base Wasm created 1 now active
2020-10-05T10:51:25.251304Z	debug	envoy wasm	WasmVm created envoy.wasm.runtime.v8 now active
2020-10-05T10:51:25.311396Z	debug	envoy wasm	Thread-Local Wasm created 2 now active
2020-10-05T10:51:25.313135Z	info	envoy wasm	wasm log my_root_id: In ShippingRootContext.onStart.
2020-10-05T10:51:25.313287Z	info	envoy wasm	wasm log my_root_id: In onConfigure.
2020-10-05T10:51:25.313313Z	debug	envoy wasm	~Wasm 1 remaining active
2020-10-05T10:51:25.315049Z	debug	envoy wasm	~WasmVm envoy.wasm.runtime.v8 1 remaining active
2020-10-05T10:51:25.315844Z	debug	envoy wasm	WasmVm created envoy.wasm.runtime.v8 now active
2020-10-05T10:51:25.315878Z	debug	envoy wasm	WasmVm created envoy.wasm.runtime.v8 now active
2020-10-05T10:51:25.315879Z	debug	envoy wasm	WasmVm created envoy.wasm.runtime.v8 now active
2020-10-05T10:51:25.320029Z	debug	envoy wasm	Thread-Local Wasm created 2 now active
2020-10-05T10:51:25.321093Z	debug	envoy wasm	Thread-Local Wasm created 3 now active
2020-10-05T10:51:25.321173Z	debug	envoy wasm	Thread-Local Wasm created 4 now active
2020-10-05T10:51:25.410836Z	info	envoy wasm	wasm log my_root_id: In ShippingRootContext.onStart.
2020-10-05T10:51:25.410934Z	info	envoy wasm	wasm log my_root_id: In onConfigure.
2020-10-05T10:51:25.411240Z	info	envoy wasm	wasm log my_root_id: In ShippingRootContext.onStart.
2020-10-05T10:51:25.411308Z	info	envoy wasm	wasm log my_root_id: In ShippingRootContext.onStart.
2020-10-05T10:51:25.411333Z	info	envoy wasm	wasm log my_root_id: In onConfigure.
2020-10-05T10:51:25.411425Z	info	envoy wasm	wasm log my_root_id: In onConfigure.
2020-10-05T10:51:25.416123Z	debug	envoy filter	Called AuthnFilterConfig : createEmptyConfigProto
2020-10-05T10:51:25.416144Z	debug	envoy filter	Called AuthnFilterConfig : createFilterFactory
2020-10-05T10:51:25.419505Z	info	envoy wasm	wasm log my_root_id: In onConfigure.
2020-10-05T10:51:25.419512Z	info	envoy wasm	wasm log my_root_id: In onConfigure.
2020-10-05T10:51:25.419506Z	info	envoy wasm	wasm log my_root_id: In onConfigure.
2020-10-05T10:51:25.798985Z	debug	envoy filter	original_dst: New connection accepted
.
.
.
.
2020-10-05T11:07:48.303123Z	debug	envoy filter	tls inspector: new connection accepted
2020-10-05T11:07:48.303149Z	debug	envoy filter	http inspector: new connection accepted
2020-10-05T11:07:52.026140Z	trace	envoy wasm	wasm log my_root_id: [shipping-wasm-filter.cc:46]::onTick() onTick
2020-10-05T11:07:52.515505Z	trace	envoy wasm	wasm log my_root_id: [shipping-wasm-filter.cc:46]::onTick() onTick

2020-10-05T11:07:55.874691Z	debug	envoy filter	original_dst: New connection accepted
2020-10-05T11:07:55.874717Z	debug	envoy filter	tls inspector: new connection accepted
2020-10-05T11:07:55.874744Z	debug	envoy filter	http inspector: new connection accepted
2020-10-05T11:07:56.024028Z	trace	envoy wasm	wasm log my_root_id: [shipping-wasm-filter.cc:46]::onTick() onTick
2020-10-05T11:07:56.516734Z	trace	envoy wasm	wasm log my_root_id: [shipping-wasm-filter.cc:46]::onTick() onTick

2020-10-05T11:07:57.559094Z	trace	envoy wasm	wasm log my_root_id: [shipping-wasm-filter.cc:46]::onTick() onTick
2020-10-05T11:07:58.026516Z	trace	envoy wasm	wasm log my_root_id: [shipping-wasm-filter.cc:46]::onTick() onTick
2020-10-05T11:07:58.494298Z	debug	envoy filter	original_dst: New connection accepted
2020-10-05T11:07:58.494323Z	debug	envoy filter	tls inspector: new connection accepted
2020-10-05T11:07:58.494351Z	debug	envoy filter	http inspector: new connection accepted
2020-10-05T11:07:58.515957Z	trace	envoy wasm	wasm log my_root_id: [shipping-wasm-filter.cc:46]::onTick() onTick
2020-10-05T11:07:58.530381Z	debug	envoy filter	original_dst: New connection accepted
2020-10-05T11:07:58.530402Z	debug	envoy filter	tls inspector: new connection accepted
2020-10-05T11:07:58.530428Z	debug	envoy filter	http inspector: new connection accepted
2020-10-05T11:07:58.562494Z	trace	envoy wasm	wasm log my_root_id: [shipping-wasm-filter.cc:46]::onTick() onTick
2020-10-05T11:07:58.578730Z	debug	envoy filter	original_dst: New connection accepted
2020-10-05T11:07:58.578746Z	debug	envoy filter	tls inspector: new connection accepted
2020-10-05T11:07:58.578770Z	debug	envoy filter	http inspector: new connection accepted
2020-10-05T11:07:59.026527Z	trace	envoy wasm	wasm log my_root_id: [shipping-wasm-filter.cc:46]::onTick() onTick
2020-10-05T11:07:59.514962Z	trace	envoy wasm	wasm log my_root_id: [shipping-wasm-filter.cc:46]::onTick() onTick
2020-10-05T11:07:59.558819Z	trace	envoy wasm	wasm log my_root_id: [shipping-wasm-filter.cc:46]::onTick() onTick
2020-10-05T11:07:59.887875Z	debug	envoy filter	original_dst: New connection accepted
2020-10-05T11:07:59.887899Z	debug	envoy filter	tls inspector: new connection accepted
2020-10-05T11:07:59.887925Z	debug	envoy filter	http inspector: new connection accepted
2020-10-05T11:08:00.025460Z	trace	envoy wasm	wasm log my_root_id: [shipping-wasm-filter.cc:46]::onTick() onTick
2020-10-05T11:08:00.518144Z	trace	envoy wasm	wasm log my_root_id: [shipping-wasm-filter.cc:46]::onTick() onTick
2020-10-05T11:08:00.562251Z	trace	envoy wasm	wasm log my_root_id: [shipping-wasm-filter.cc:46]::onTick() onTick

When I tried with proxy_abi_version_0_1_0 , I see an error during wasm init. So this is not an issue with ABI compatibility?
Logs with proxy_abi_version_0_1_0:

2020-10-05T12:57:25.837395Z	debug	envoy wasm	WasmVm created envoy.wasm.runtime.v8 now active
2020-10-05T12:57:25.837638Z	debug	envoy wasm	Base Wasm created 1 now active
2020-10-05T12:57:25.845934Z	trace	envoy wasm	Bad function signature for: proxy_on_request_headers
2020-10-05T12:57:25.845956Z	trace	envoy wasm	Bad function signature for: proxy_on_response_headers
2020-10-05T12:57:25.846652Z	error	envoy wasm	Wasm VM failed Failed to initialize Wasm code
2020-10-05T12:57:25.846668Z	debug	envoy wasm	~Wasm 0 remaining active
2020-10-05T12:57:25.847981Z	debug	envoy wasm	~WasmVm envoy.wasm.runtime.v8 0 remaining active
2020-10-05T12:57:25.849939Z	trace	envoy wasm	Unable to create Wasm
2020-10-05T12:57:25.851285Z	warning	envoy config	gRPC config for type.googleapis.com/envoy.config.listener.v3.Listener rejected: Error adding/updating listener(s) virtualInbound: Unable to create Wasm HTTP filter

Any other ideas on how to debug this further?
Also, what is the purpose of WasmService in bootstrap.extensions? Is this required for my example as well?

10

Can you please use the same SHA as istio/proxy and envoy-wasm. The ABI versioning is still unstable, so I can’t promise it’s consistent across versions. The error is clear that signatures are mis-matched.

WasmService is just a singleton VM running on main Envoy thread. It’s not necessary to use.

11

We are using gcr.io/istio-testing/proxyv2:1.8-alpha.882778a679482da7205a05457c2027f10c76816a
I see the same result(filter not getting invoked) with WASM binary generated using corresponding SHA for CPP SDK.

Btw, in my previous post, I have shared logs for two cases

  1. One was compatible ABI for which we can see that the VM is alive as per the logs
  2. Other is incompatible and we can see an error in logs -->“Bad function signature for: proxy_on_request_headers”
12

How did you build your extension? Could you post your build command here?

13

@Pengyuan_Bian It finally worked for me. Did not have to change anything though. This filter is working for me in Istio 7.3 as well as the master build. I used the filter definition as below.

Thanks all for the help.

apiVersion: networking.istio.io/v1alpha3
kind: EnvoyFilter
metadata:
  name: example-cpp-wasm-filter
  namespace: service-mesh
spec:
  configPatches:
  - applyTo: HTTP_FILTER
    match:
      context: SIDECAR_INBOUND
      listener:
        portNumber: 7443
        filterChain:
          filter:
            name: envoy.http_connection_manager
            subFilter:
              name: envoy.router
    patch:
      operation: INSERT_BEFORE
      value:
        name: envoy.filters.http.wasm
        typed_config:
          "@type": type.googleapis.com/udpa.type.v1.TypedStruct
          type_url: type.googleapis.com/envoy.extensions.filters.http.wasm.v3.Wasm
          value:
            config:
              configuration:
               '@type': type.googleapis.com/google.protobuf.StringValue
               value: "dummy"
              rootId: my_root_id
              vmConfig:
                code:
                  local:
                    filename: /var/local/lib/wasm-filters/example-cpp-filter.wasm
                runtime: envoy.wasm.runtime.v8
                vmId: example_cpp_vm_id
                allow_precompiled: true
  workloadSelector:
    labels:
      app: istio-shipping
1 Like