Istio-proxy container leaking HTTP connections?

tomhawk · April 8, 2021, 8:58am

I have Istio v1.8.4 installed and running well on K8s v1.18 cluster.
I made an app container with a (Go) code that creates an outbound connection to some external website.
Instead of re-using that connection , I prefer to close the connection after 1 minute (special case, I know)
I use Resty client for that.

client := resty.New()
client.SetRetryWaitTime(2 * time.Second).SetRetryCount(2).SetCloseConnection(true).SetTimeout(time.Second * 5)

after several days : I’ve noticed the Istio-proxy memory rise. If I turn SetCloseConnection(false) : the memory is flat.

github.com

go-resty/resty/blob/97a15579492cd5f35632499f315d7a8df94160a1/client.go#L698


func (c *Client) SetScheme(scheme string) *Client {
	if !IsStringEmpty(scheme) {
		c.scheme = scheme
	}

	return c
}

// SetCloseConnection method sets variable `Close` in http request struct with the given
// value. More info: https://golang.org/src/net/http/request.go
func (c *Client) SetCloseConnection(close bool) *Client {
	c.closeConnection = close
	return c
}

// SetDoNotParseResponse method instructs `Resty` not to parse the response body automatically.
// Resty exposes the raw response body as `io.ReadCloser`. Also do not forget to close the body,
// otherwise you might get into connection leaks, no connection reuse.
//
// Please Note: Response middlewares are not applicable, if you use this option. Basically you have
// taken over the control of response parsing from `Resty`.

two options here:

The remote server doesn’t honor closeConnection flag (i.e. close connection after reply)
istio-proxy doesn’t honor close connection … thus you can see the graph. the app container in purple. the istio proxy container in orange.

sysDig2388×1164 130 KB
. if I recall correctly , istio-proxy idle connection timeout is 1hr.
what is best approach to log this issue ?

Topic		Replies	Views
Application HTTP Keep Alive issues with Istio Networking	2	7788	May 26, 2021
Istio reset with AWS NLB and Slack access Config	0	955	February 4, 2022
Istio: Application/Service timing out Networking	0	353	September 9, 2020
How to handle downstream keep alive connections not closed when upstream closes	3	2570	May 3, 2023
Access to k8s API server is lost after ~90 minutes Networking	0	422	March 28, 2019

Istio-proxy container leaking HTTP connections?

Related topics