After pulling my hairs for days, playing with all permuation and combinations. Found my mistake at: https://github.com/istio/istio/issues/9696.
It was related to service named port. I had named it just “web”, updating it to “http-web”, worked for me, and everything else worked fine.
Stackoverflow answer link : https://stackoverflow.com/questions/54197734/istio-destination-rule-subsets-not-working/54209833#54209833.