Hello Guys,
Today I works to implement istio in production but i have a problem,
because i dont find the answer in istio docs.
When i configure istio canary, this works fine and the traffic is distribute correctly but if I scale v2 to 0 containers, the istio canary continue send traffic to the service without containers.
I trying implement circuit breaking but don’t work for me.
EXAMPLE USE 2 differents services:
---
apiVersion: v1
kind: Service
metadata:
name: hello-dev-stable
namespace: hello-dev
labels:
app: 'hello'
env: 'dev'
version: 'stable'
annotations: { }
spec:
type: ClusterIP
ports:
- name: http
port: 80
selector:
app: 'hello'
env: 'dev'
version: 'stable'
---
apiVersion: v1
kind: Service
metadata:
name: hello-dev
namespace: hello-dev
labels:
app: 'hello'
env: 'dev'
version: 'canary'
annotations: { }
spec:
type: ClusterIP
ports:
- name: http
port: 80
selector:
app: 'hello'
env: 'dev'
version: 'canary'
---
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
name: hello-dev
namespace: hello-dev
spec:
gateways:
- hello-dev
hosts:
- 'hello-dev.prd-k8s.pepito.com'
http:
- match:
- headers:
user-x:
regex: ".*Eze.*"
route:
- destination:
host: hello-dev-stable
subset: v1
weight: 70
- destination:
host: hello-dev
subset: v2
weight: 30
- route:
- destination:
host: hello-dev-stable
subset: v1
---
apiVersion: networking.istio.io/v1alpha3
kind: DestinationRule
metadata:
name: hello-dev-stable
namespace: hello-dev
spec:
host: hello-dev-stable
trafficPolicy:
outlierDetection:
consecutiveErrors: 1
interval: 1s
baseEjectionTime: 1h
maxEjectionPercent: 100
subsets:
- name: v1
labels:
version: stable
---
apiVersion: networking.istio.io/v1alpha3
kind: DestinationRule
metadata:
name: hello-dev
namespace: hello-dev
spec:
host: hello-dev
trafficPolicy:
outlierDetection:
consecutiveErrors: 1
interval: 1s
baseEjectionTime: 1h
maxEjectionPercent: 100
subsets:
- name: v2
labels:
version: canary
➜ ~ k scale --replicas=0 deployment/hello-dev -n hello-dev
➜ ~ k get po -n hello-dev
NAME READY STATUS RESTARTS AGE
hello-dev-stable-6667775dc8-6ndtt 2/2 Running 0 11h
➜ ~ k get deploy -n hello-dev
NAME DESIRED CURRENT UP-TO-DATE AVAILABLE AGE
hello-dev 0 0 0 0 12h
hello-dev-stable 1 1 1 1 15h
➜ ~ for i in {1..1000}; do curl hello-dev.prd-k8s.pepito.com/api/helloprim-ms/ -H"user-x: Eze" ; done
stable
no healthy upstreamstable
stable
stable
no healthy upstreamstable
stable
stable
no healthy upstreamstable
I trying the replicate the problem using one host(service) and 2 host differents.
EXAMPLE USE 1 service same host:
---
apiVersion: v1
kind: Service
metadata:
name: hello-dev
namespace: hello-dev
labels:
app: 'hello'
env: 'dev'
annotations: { }
spec:
type: ClusterIP
ports:
- name: http
port: 80
selector:
app: 'hello'
env: 'dev'
---
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
name: hello-dev
namespace: hello-dev
labels:
app: hello
env: dev
spec:
gateways:
- hello-dev
hosts:
- 'services.prd-k8s.pepito.com'
http:
- match:
- uri:
prefix: /api/helloprim-ms/
route:
- destination:
host: hello-dev
subset: stable
weight: 40
- destination:
host: hello-dev
subset: canary
weight: 60
timeout: 30s
---
apiVersion: networking.istio.io/v1alpha3
kind: DestinationRule
metadata:
name: hello-dev
namespace: hello-dev
spec:
host: hello-dev
trafficPolicy:
outlierDetection:
consecutiveErrors: 1
interval: 1s
baseEjectionTime: 1h
maxEjectionPercent: 100
subsets:
- name: stable
labels:
version: stable
- name: canary
labels:
version: canary
➜ ~ for i in {1..1000}; do curl services.prd-k8s.pepito.com/api/helloprim-ms/ ; done
no healthy upstreamno healthy upstreamstable
stable
no healthy upstream
stable
no healthy upstream
no healthy upstream
no healthy upstream
stable
stable
stable
stable
no healthy upstream
no healthy upstream
stable
no healthy upstream
stable
stable
It’s possible don’t send traffic to version down using canary??