Routing not working as expected

#1

I am testing some traffic management concepts using https://github.com/istio/istio/tree/master/samples/helloworld as an example on istio-1.1.0 on k8s cluster which has PSP enabled but seems as if either my destinationrules or virtual services are not working. After debugging using istioctl proxy-config routes I am able to to see the correct route but when I actually hit the service i don’t see the rules being applied.
These are my virtualservice and destinationrules

apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  name: helloworld-vs
spec:
  hosts:
  - helloworld
  http:
  - route:
    - destination:
        host: helloworld
        subset: v2
---
apiVersion: networking.istio.io/v1alpha3
kind: DestinationRule
metadata:
  name: helloworld
spec:
  host: helloworld
  subsets:
  - name: v1
    labels:
      version: v1
  - name: v2
    labels:
      version: v2
---

One thing to note is that we have PSP enabled on this cluster and it is an on-prem cluster.

#2

Your virtual service does not seem to be using gateway (as in the examples), why have you removed it?

You could get some clues about misconfiguration from Kiali.

#3

@rvansa the original virtual-service has not been removed(helloworld-gateway.yaml which is shipped with istio).

All of these work fine on one of our cluster but on another one which has psp applied it does not work there.

#4

Ok, im working with @AshishThakur. let me get clear some confusion here.

  1. So we installed the 1.1.0 istio on our on-prem environment
  2. We then installed the helloworld sample service and its working as expected. When we do curl, the requests can be routed to either v1 or v2 helloworld
  3. Then we tried to test the request routing. We applied, for example, below virtual service to try to route all request(either external from curl or internal from other service) to v2 helloworld, but it does not work. Requests still go to either v1 or v2.

apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
name: helloworld
spec:
hosts:
- helloworld
http:
- route:
- destination:
host: helloworld
subset: v2

Any suggestions?

#5

FYI https://istio.io/help/ops/component-debugging/

#6

are you sure iptables are setup properly? and that envoy is running as user 1337? cc @Deepa_Kalani who was also tinkering with PSPs on pks clusters

#7

yea, i have been tinkering with psp but don’t see any issues (although my deployment might be slightly different) given i’m using istio-cni.

can you post the destination rules as well ?

#8

actually I just saw you did…, do you want to try applying the Virtual Service to the gateway ?

something like this :

gateways:

  • helloworld-gateway
#9

@Deepa_Kalani
Please find the destinationrules

apiVersion: networking.istio.io/v1alpha3
kind: DestinationRule
metadata:
  name: helloworld
spec:
  host: helloworld
  subsets:
  - name: v1
    labels:
      version: v1
  - name: v2
    labels:
      version: v2

and virtualservice

apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  name: helloworld-vs
spec:
  hosts:
  - helloworld
  http:
  - route:
    - destination:
        host: helloworld
        subset: v1

The weird issue is that in case I curl from sleep to helloworld…it seems envoy proxy is being bypassed

#10

could you help how I can validate if iptables have been set properly?

#11

yes envoy is running as user 1337.
38%20PM

#12

nsenter -t {pid} -n iptables -t nat -S

check the iptables rule setting , the pid is the id of your app pocess seen from host.

#13

In my env,

-P PREROUTING ACCEPT
-P INPUT ACCEPT
-P OUTPUT ACCEPT
-P POSTROUTING ACCEPT
-N ISTIO_IN_REDIRECT
-N ISTIO_OUTPUT
-N ISTIO_REDIRECT
-A OUTPUT -p tcp -j ISTIO_OUTPUT
-A ISTIO_IN_REDIRECT -p tcp -j REDIRECT --to-ports 15001
-A ISTIO_OUTPUT ! -d 127.0.0.1/32 -o lo -j ISTIO_REDIRECT
-A ISTIO_OUTPUT -m owner --uid-owner 1337 -j RETURN
-A ISTIO_OUTPUT -m owner --gid-owner 1337 -j RETURN
-A ISTIO_OUTPUT -d 127.0.0.1/32 -j RETURN
-A ISTIO_OUTPUT -j ISTIO_REDIRECT
-A ISTIO_REDIRECT -p tcp -j REDIRECT --to-ports 15001

#14

check the logs at “helloworld” proxy “kubectl logs -f -n manual -c istio-proxy” and see if the log appears when you hit curl