I posted my setup in a separate thread: Adding headers for oauth2-proxy redirect
Check the oauth2-proxy logs now that you’ve got a redirect header, if there are any issues with your redirect URL it will be posted there (note that protocol and quotation marks are required, eg 'https://%REQ(Host)%').
Out of curiosity, how are you seeing the headers sent to oauth2-proxy? They don’t show up in my in my browser.