Hello Everyone,
I want only traffic-routing and telemetry features of Istio. Not gateways or security.
I followed minimal istio install mentioned here - https://istio.io/docs/setup/kubernetes/minimal-install/
I used helm to create istio yaml:
helm template install/kubernetes/helm/istio --name istio --namespace istio-system
–set security.enabled=false
–set ingress.enabled=false
–set gateways.istio-ingressgateway.enabled=false
–set gateways.istio-egressgateway.enabled=false
–set galley.enabled=false
–set sidecarInjectorWebhook.enabled=false
–set mixer.enabled=true
–set prometheus.enabled=true
–set global.proxy.envoyStatsd.enabled=false
–set pilot.sidecar=false > $HOME/istio-minimal.yaml
I also set mixer and Prometheus to true here. After this when I inject sidecars to my microservices. all the microservices going on crashloopback off with following logs/errors in istio-proxy container:
_[2019-02-15 22:39:28.946][183][warning][config] bazel-out/k8-opt/bin/external/envoy/source/common/config/virtual_includes/grpc_mux_subscription_lib/common/config/grpc_mux_subscription_impl.h:70] gRPC config for type.googleapis.com/envoy.api.v2.Listener rejected: Error adding/updating listener 10.206.70.12_8080: unable to read file: /etc/certs/root-cert.pem
[2019-02-15 22:39:28.946][183][info][config] external/envoy/source/server/listener_manager_impl.cc:908] all dependencies initialized. starting workers
_[2019-02-15 22:39:28.948][188][critical][backtrace] bazel-out/k8-opt/bin/external/envoy/source/server/virtual_includes/backtrace_lib/server/backtrace.h:125] Caught Segmentation fault, suspect faulting address 0x0
_[2019-02-15 22:39:28.949][188][critical][backtrace] bazel-out/k8-opt/bin/external/envoy/source/server/virtual_includes/backtrace_lib/server/backtrace.h:94] Backtrace thr<188> obj</usr/local/bin/envoy> (If unsymbolized, use tools/stack_decode.py):
_[2019-02-15 22:39:28.951][188][critical][backtrace] bazel-out/k8-opt/bin/external/envoy/source/server/virtual_includes/backtrace_lib/server/backtrace.h:114] thr<188> #0 0x49f684 Envoy::Http::Mixer::Control::Control()
If I dont have envoy in the microservices how can I make use of istio traffic routing features using “VirtualService” ?
I am also seeing livelinessprobe error, this could be the reason too for pod restarts.
here is my liveliness and readiness configuration:
livenessProbe:
failureThreshold: 3
httpGet:
httpHeaders:
- name: Accept
value: application/json
path: /health
port: 8080
scheme: HTTP
initialDelaySeconds: 180
periodSeconds: 30
successThreshold: 1
timeoutSeconds: 3
name: auth-springboot
ports:
- containerPort: 8080
name: auth-sb
protocol: TCP
readinessProbe:
failureThreshold: 3
httpGet:
httpHeaders:
- name: Accept
value: application/json
path: /health
port: 8080
scheme: HTTP
initialDelaySeconds: 60
periodSeconds: 30
successThreshold: 1
timeoutSeconds: 1
Altough Istio authors claim that you can run only subset of Istio components in reality you better install full blown Istio to make it to work. Some features may require other dependent Istio components to run besides. Thats our lesson learned.
You do not need Envoy proxy sidecar for “VirtualService” traffic shifting to work. At least not for Weight traffic routing. May not be true for routing based on headers or other methods.
I recommend to install full Istio bundle and see if you get pass this error. You do not need to enable envoy sidecars at first iteration. See what works for you first.
Hope that helps
Thanks for your reply tvalasek.
Even with the full istio install, if we use nginx(without envoy) as the entrypoint that will pass the calls to istio-enabled microservices instead of using istio’s native “gateway” for it. We are not able to apply virtualservices to the calls
so you do not use Istio Ingress, but Nginx Ingress?
Yes, I use only Nginx-ingress, due to some restrictions I can not use istio-ingress.
I see. I doubt Nginx Ingress understands Istio CRDs (like VirtualService) and can take advantage of Istio’s advanced routing. You will need to deploy the full Istio stack
I am deploying envoy too with nginx-ingress pod. nginx-ingress we use is nothing but a reverse proxy setup we get the calls at nginx and it routes the calls to microservices.
I tested I can apply virtual-services to the calls coming from nginx to my microservices and get the benefit of delays, timeouts, mirroring, canary etc.
I am not using Istio-Gateway as there is some complex header manipulation tat happens at nginx level which istio can not handle at virtualservice or gateway level.
i’m bit confused here. In earlier post you say, quote
We are not able to apply virtualservices to the calls
now you stated, quote
I can apply virtual-services to the calls
If Nginx ingress doesnt have envoy. I am not able to use virtual service.
That would be ideal for me to not to add any extra layer to nginx-ingress
I don’t think that is doable. Even Istio Ingress is actually an envoy proxy implementation in background
An idea on getting this to work might be to inject the sidecar on the ingress controllers themselves. That should allow you to take advantage of the internal virtual service routing. One caveat, which I ran into myself, is that routing might not work at all if the controller routes traffic directly by IP, which seems to be unsupported by Envoy if I remember correctly.
Thanks Fernando.
I tried what you suggested. I have envoy in my nginx pod and this is working fine for me but ideal situation would have been no envoy in nginx.
Also with “envoy in nginx” I am not able to make https work… Any ideas on that ?
I think what you mentioned makes sense. Do you have a load balancer in front of nginx? If so, are you letting the LB terminate SSL, or letting it pass through? If it’s the latter, then it might involve additional configuration which I’m not too familiar with. Terminating SSL at the load balancer level would be the easiest, but potentially less secure option, relatively speaking.
I am able to run curl with https inside the nginx pod.
TLS termination is happening at nginx only but envoy is blocking it.
As I am not using Istio-Gateway, Does anyone know how to fix this ? Do I ahve to create a virtualservice to fix this or make use of Istio-citadel ?