Mixing Varnish and HTTPS is usually a pain in the ass. Even more when Wordpress is involved.
The common setup is to expose Nginx on the 443 port, let him handle the HTTPS connections, proxying to Varnish, and fetching back from Nginx - in HTTP - listening on a high