org.apache.http.conn.HttpHostConnectException

Hi All,

I have my spring boot services deployed to AKS cluster and had Istio installed.
While I am doing the load testing with JMeter am getting the below exception.

Non HTTP response code: org.apache.http.conn.HttpHostConnectException/Non HTTP response message: Connect to xxx.xxx.cloudapp.azure.com:80 [xxx.xxx.cloudapp.azure.com\/xxx.xx.xxx.xxx] failed: Cannot assign requested address (connect failed)

I have tried increased no:of pods for “istio-ingressgateway” & “istio-pilot” from https://github.com/istio/istio/issues/10904.

If the load is anything more than 1000 req/sec the above exception starts occurring and the remaining tests starts failing. Tried different versions of Istio as well.

Current confiruation is:
AKS - 1.12.8
Istio - 1.1.7

Thanks in advance,
Binoy

Hello,

The problem comes from TCP layer. You can figure out if the error is generated by client machine (where jmeter runs) by modifying the ip_local_port_range. Probably your client runs out of ports, because each jmeter thread can open several tcp connections.
What OS do you use for jmeter client?

Linux : http://www.linuxbrigade.com/reduce-time_wait-socket-connections/

Windows : https://support.microsoft.com/en-gb/help/929851/the-default-dynamic-port-range-for-tcp-ip-has-changed-in-windows-vista

You could play a bit and change the port range. First run your tests with a number of threads which doesn’t trow errors. Than decrease the port range and set only 1000 ephemeral ports for ex. and run the tests again and it should cause failures. If that happens, the problem is your jmeter client. In order to solve that set the range to the maximum number of ports, enable port reuse, reduce tcp reuse wait_time (They are all described in the links above). If that is not enough, you should go for distributed testing.

Cheers,
Mihai

1 Like

Thanks a lot for your feedback @mihai_barta. Much appreciated :blush:
I will follow the steps mentioned in the above link (Linux).

Another way for supporting more TCP connections is to allocate multiple IPs for client machine and configure jmeter to send requests from all the IPs ( the number of ephemeral ports are per IP) , but I haven’t tested it yet. Distributed testing ( or just run a few jmeter clients in master mode on different machines) is still a better approach for this purpose. Good luck.

1 Like