Access non mesh service in the istio-system namespace

Networking
1

I have deployed istio 1.1.6 with mtls enabled globally and prometheus enabled. I am attempting to use flagger for canary deployments (https://github.com/weaveworks/flagger)

If i deploy flagger and flagger-load within the mesh, flagger can’t communicate with prometheus. If i deploy flagger and flagger-load outside of the mesh, flagger-load can’t communicate with my applications. If i deploy flagger outside the mesh and flagger-load inside the mesh, flagger can’t communicate with flagger-load.

I opted to deploy both flagger and flagger-load within the mesh and deployed the below DR to the istio-system namespace. When i attempt to connect to prometheus i get this response: Connection reset by peer

apiVersion: networking.istio.io/v1alpha3
kind: DestinationRule
metadata:
 name: flagger-prometheus
spec:
 host: "prometheus.istio-system.svc.cluster.local"
 trafficPolicy:
   tls:
     mode: DISABLE

I expect this specific host DR to have precedence over the default *.local DR. Am i wrong about how the precedence works?

2

What namespace is the rule in? I think you need it in either the prometheus namespace (istio-system) or flagger namespace

3

I’ve tried creating the rule in both the flagger namespace and istio-system namespace. No cookie

4

One way around this would be to deploy Flagger outside the mesh and the load tester inside the mesh. You can remove the load tester ingress from the mesh and allow only egress in the sidecar with excludeInboundPorts annotation:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: flagger-loadtester
spec:
  selector:
    matchLabels:
      app: flagger-loadtester
  template:
    metadata:
      labels:
        app: flagger-loadtester
      annotations:
        prometheus.io/scrape: "true"
        traffic.sidecar.istio.io/excludeInboundPorts: "8080,80"
5

WIth flagger outside the mesh and the load tester inside the mesh:

Events:
  Type     Reason  Age    From     Message
  ----     ------  ----   ----     -------
  Normal   Synced  3m38s  flagger  New revision detected! Scaling up test.dev
  Normal   Synced  2m38s  flagger  Starting canary analysis for test.dev
  Normal   Synced  2m38s  flagger  Advance test.dev canary weight 5
  Warning  Synced  98s    flagger  Halt test.dev advancement external check generate-health-checks failed Post http://flagger-load-tester.flagger:80/: read tcp 10.72.103.91:56494->172.20.70.225:80: read: connection reset by peer
  Warning  Synced  38s    flagger  Halt test.dev advancement external check generate-health-checks failed Post http://flagger-load-tester.flagger:80/: read tcp 10.72.103.91:57258->172.20.70.225:80: read: connection reset by peer

I think flagger needs to be able to connect to the load tester yah?

$ kubectl exec -it flagger-75458b7d69-h78qm -n flagger -- ash
/home/flagger $ wget flagger-load-tester
Connecting to flagger-load-tester (172.20.70.225:80)
wget: error getting response: Connection reset by peer
6

I was able to get this working with flagger outside the mesh and the load tester inside the mesh. Added the rules below:

---
apiVersion: networking.istio.io/v1alpha3
kind: DestinationRule
metadata:
  name: flagger-namespace
  namespace: flagger
spec:
  host: "*.flagger.svc.cluster.local"
  trafficPolicy:
    tls:
      mode: DISABLE
---
apiVersion: authentication.istio.io/v1alpha1
kind: Policy
metadata:
  name: loadtester-mtls-disabled
  namespace: flagger
spec:
  targets:
  - name: flagger-load-tester
    ports:
    - number: 80
1 Like