I changed my istio config so it pulls the proxy image from a private repo and while the pod is able to retrieve the image and attempts to start the container it is crashing with errors and failing.
Below is the config before in the istio config map (relevant part only)
"enableHelmTest": false, "enabled": true, "hub": "docker.io/istio", "imagePullPolicy": "", "imagePullSecrets": [], "istioNamespace": "istio-system", "istiod": { "enableAnalysis": false, "enabled": true }, "jwtPolicy": "third-party-jwt",
I changed it to
"enableHelmTest": false, "enabled": true, "hub": "myprivaterepo.jfrog.io/example-docker-virtual", "imagePullPolicy": "", "imagePullSecrets": ["repo-secret"], "istioNamespace": "istio-system", "istiod": { "enableAnalysis": false, "enabled": true }, "jwtPolicy": "first-party-jwt",
As mentioned it pulls the image as normal but I get the below errors in the logs for the container. They continue on a loop and my application never starts up
2022-02-08T09:37:14.696190Z info parsed scheme: "" 2022-02-08T09:37:14.696210Z info scheme "" not registered, fallback to default scheme 2022-02-08T09:37:14.696234Z info ccResolverWrapper: sending update to cc: {[{istiod.istio-system.svc:15012 <nil> 0 <nil>}] <nil> <nil>} 2022-02-08T09:37:14.696242Z info ClientConn switching balancer to "pick_first" 2022-02-08T09:37:14.696247Z info Channel switches to new LB policy "pick_first" 2022-02-08T09:37:14.696277Z info Subchannel Connectivity change to CONNECTING 2022-02-08T09:37:14.696391Z info Subchannel picks a new address "istiod.istio-system.svc:15012" to connect 2022-02-08T09:37:14.696490Z info sds SDS gRPC server for workload UDS starts, listening on "./etc/istio/proxy/SDS" 2022-02-08T09:37:14.696577Z info sds Start SDS grpc server 2022-02-08T09:37:14.696470Z info pickfirstBalancer: HandleSubConnStateChange: 0xc000e3a360, {CONNECTING <nil>} 2022-02-08T09:37:14.697083Z info Channel Connectivity change to CONNECTING 2022-02-08T09:37:14.697091Z info Starting proxy agent 2022-02-08T09:37:14.697103Z info Opening status port 15020 2022-02-08T09:37:14.697532Z info Received new config, creating new Envoy epoch 0 2022-02-08T09:37:14.697604Z info Epoch 0 starting 2022-02-08T09:37:14.703223Z info Envoy command: [-c etc/istio/proxy/envoy-rev0.json --restart-epoch 0 --drain-time-s 45 --parent-shutdown-time-s 60 --service-cluster example-service.example-prod --service-node sidecar~10.244.1.253~example-service-fb8b7975b-292pq.example-prod~example-prod.svc.cluster.local --max-obj-name-len 189 --local-address-ip-version v4 --log-format %Y-%m-%dT%T.%fZ %l envoy %n %v -l warning --component-log-level misc:error --concurrency 2] 2022-02-08T09:37:14.709823Z info Subchannel Connectivity change to READY 2022-02-08T09:37:14.709888Z info pickfirstBalancer: HandleSubConnStateChange: 0xc000e3a360, {READY <nil>} 2022-02-08T09:37:14.709898Z info Channel Connectivity change to READY 2022-02-08T09:37:14.764836Z warning envoy config [bazel-out/k8-opt/bin/external/envoy/source/common/config/_virtual_includes/grpc_stream_lib/common/config/grpc_stream.h:92] StreamAggregatedResources gRPC config stream closed: 14, no healthy upstream 2022-02-08T09:37:14.764904Z warning envoy config [bazel-out/k8-opt/bin/external/envoy/source/common/config/_virtual_includes/grpc_stream_lib/common/config/grpc_stream.h:54] Unable to establish new stream 2022-02-08T09:37:14.774788Z info sds resource:default new connection 2022-02-08T09:37:14.774879Z info sds Skipping waiting for ingress gateway secret 2022-02-08T09:37:15.123059Z warning envoy config [bazel-out/k8-opt/bin/external/envoy/source/common/config/_virtual_includes/grpc_stream_lib/common/config/grpc_stream.h:92] StreamAggregatedResources gRPC config stream closed: 14, no healthy upstream 2022-02-08T09:37:15.123096Z warning envoy config [bazel-out/k8-opt/bin/external/envoy/source/common/config/_virtual_includes/grpc_stream_lib/common/config/grpc_stream.h:54] Unable to establish new stream 2022-02-08T09:37:15.302811Z error citadelclient Failed to create certificate: rpc error: code = Unauthenticated desc = request authenticate failure 2022-02-08T09:37:15.302845Z error cache resource:default request:b057ab28-efd6-49c5-a554-01c48f1a0c2b CSR hit non-retryable error (HTTP code: 0). Error: rpc error: code = Unauthenticated desc = request authenticate failure 2022-02-08T09:37:15.302873Z error cache resource:default failed to generate secret for proxy: rpc error: code = Unauthenticated desc = request authenticate failure 2022-02-08T09:37:15.302884Z error sds resource:default Close connection. Failed to get secret for proxy "sidecar~10.244.1.253~example-service-fb8b7975b-292pq.example-prod~example-prod.svc.cluster.local" from secret cache: rpc error: code = Unauthenticated desc = request authenticate failure
I dont understand the failure as all I am changing is the source where the proxy image is pulled from.
The proxy image is the same one I was pulling from docker hub. I pulled it locally and pushed into my private repo
version: proxyv2:1.6.3