Help debugging unexpected ingress gateway 301s from POST to GET

1

Hi folks! I’m new here (to k8s, to Istio, and to this forum), so thanks in advance for bearing with me! This seems like the kind of issue that could only be caused by a silly mistake or config issue.

I finally have Istio configured the way I want it on GKE and started smoke testing my API via Postman. I’m using 1.4.0, but I’m pretty sure I saw the same behavior on 1.3.3 before upgrading.

  1. POST /api/v1/auth/token gives me a JWT. So far so good.
  2. GET /api/v1/notes returns an empty array because I haven’t added any notes yet. So far so good.
  3. POST /api/v1/notes returns an empty array because… err, what? Shouldn’t it return a copy of the freshly created note?

Looking through Envoy access logs I’m seeing this at my ingress gateway:

"POST /api/v1/notes HTTP/1.1" 301 - "-" "-" 0 0 0 - "10.2.0.7" "PostmanRuntime/7.19.0" "b5953389-dbde-43dc-b4f8-47228abe5077" "api-staging.<domain>.com" "-" - - 10.0.2.13:80 10.2.0.7:59101 - -

immediately followed by

"GET /api/v1/notes HTTP/1.1" 200 - "-" "-" 0 2 193 192 "10.2.0.9" "PostmanRuntime/7.19.0" "43e4e2d2-bf45-4fea-b968-a2adfba369b0" "api-staging.<domain>.com" "10.0.1.17:5000" outbound|5000||api-service.default.svc.cluster.local - 10.0.2.13:443 10.2.0.9:58879 api-staging.<domain>.com -

And only this on my service’s sidecar:

"GET /api/v1/notes HTTP/1.1" 200 - "-" "-" 0 2 191 190 "10.2.0.9" "PostmanRuntime/7.19.0" "43e4e2d2-bf45-4fea-b968-a2adfba369b0" "api-staging.<domain>.com" "127.0.0.1:5000" inbound|5000|http|api-service.default.svc.cluster.local - 10.0.1.17:5000 10.2.0.9:0 outbound_.5000_._.api-service.default.svc.cluster.local default

My interpretation is that the ingress gateway is 301ing the POST /api/v1/notes to a GET /api/v1/notes before it gets to my service. This doesn’t make sense to me, especially given that POST /api/v1/auth/token isn’t redirected.

Does my interpretation seem right? If so, any ideas about the root cause? Is there any additional info I can provide to aid debugging?

It’s worth noting that the API behaves as expected (with the exact same Postman config) when testing locally via docker-compose, so I wouldn’t expect it to be an issue with the server itself.

Thanks!

2

Figured it out. I have httpsRedirect turned on at my gateway, and I was mistakenly sending some of these requests as HTTP. POSTs via http are 301d to GET, presumably because 301s can’t preserve form data.