Issue integrating ELK with keycloak on istio-enabled setup, externally accessed via istio ingressgateway

We have integrated ELK (elasticsearch & kibana) with openid-based authentication via keycloak in kubernetes.

The high-level flow of interactions among the 3 (elasticsearch, kibana, keycloak) in-general is as follows:

  1. Kibana service start up
    During kibana service startup, kibana pod interacts with keycloak to check its availability (kibana-keycloak backend interaction).

  2. Kibana login
    i. User hits kibana access url which redirects to keycloak authentication page
    (Kibana-keycloak frontend interaction)
    ii. Keycloak authenticates the user and returns a jwt token to kibana.
    iii. Kibana sends this token in the backend to elasticsearch requesting for access. Elasticsearch sends this token to keycloak for token introspection. (Elasticsearch-keycloak backend interaction)

Key points:

  • There is only one parameter in kibana configuration to specify keycloak openid metadata url. The same parameter is used for both frontend & backend interactions.
  • The keycloak openid metadata url configured in elasticsearch should have the same hostname as configured in kibana. If they are different, token introspection would fail due to issuer hostname mismatch.

Issue:
When ELK and keycloak are running in istio-enabled k8s cluster, kibana & keycloak UI are independently (without ELK & keycloak integration) accessible on browser via istio ingressgateway loadbalancer.
Keycloak is externally accesible at https://{keycloak-fqdn}:{nodeport} , while kibana is externally accesible at https://{kibana-node-fqdn}:{nodeport}/kibana. (Required gateways and virtualservices created accordingly)

  1. To integrate ELK with keycloak, if the externally-accesible keycloak url with nodeport is configured in ELK, the internal communication between kibana & keycloak fails because the external keycloak fqdn is not resolvable from within the k8s cluster.

  2. If the internally accesible keycloak url (keycloak k8s service with service port) is configured in ELK, the internal communication between kibana & keycloak would succeed, but on accessing kibana in the browser, it would not be able to redirect to keycloak as the browser cannot resolve the keycloak k8s service and also the k8s service port would be unknown.

Expectation:
Is there a solution to overcome this problem?

Version Details:
Istio: 1.5.2
ELK: 7.0.1
Keycloak: 10.0.1
Kubernetes: 1.17.4