Websocket & Buffering Filter not working together

Networking
1

I have an EnvoyFilter which increases HTTP request size.
Setting the filter results in websocket not working. I have read buffering interferes with websocket so tried skipping the filter for websocket but still no luck.
Any help on this is appreciated.

My config:

configPatches:
  - applyTo: HTTP_FILTER
    match:
      context: GATEWAY
      listener:
        filterChain:
          filter:
            name: envoy.filters.network.http_connection_manager
    patch:
      operation: MERGE
      value:
        name: with-matcher
        typed_config:
          '@type': type.googleapis.com/envoy.extensions.common.matching.v3.ExtensionWithMatcher
          extension_config:
            name: envoy.filters.http.buffer
            typed_config:
              '@type': type.googleapis.com/envoy.extensions.filters.http.buffer.v3.Buffer
              maxRequestBytes: 2052428800
          xds_matcher:
            matcher_tree:
              exact_match_map:
                map:
                  websocket:
                    action:
                      name: skip
                      typed_config:
                        '@type': type.googleapis.com/envoy.extensions.filters.common.matcher.action.v3.SkipFilter
              input:
                name: request-headers
                typed_config:
                  '@type': type.googleapis.com/envoy.type.matcher.v3.HttpRequestHeaderMatchInput
                  header_name: Upgrade
  workloadSelector:
    labels:
      app: istio-ingressgateway
      istio: ingressgateway