https_fixup.py

[Mads Kiilerich]

On 10/07/2014 11:42 AM, Jim Hague wrote:
> Hi all,
>
> This rambles a bit, but basically turns into a question for the deployment and
> WSGI-knowledgeable.
>
> I wasted a bit of time Friday night working out why 'force_https' wasn't
> working.

Yes - that was just a bug. The fix was in my testing queue and had not 
been pushed yet. The workaround is to use the wrong name 'force_ssl'.

> Dolt that I am, there is a pull request fixing the problem. By the
> time Mads pointed me at it, I'd submitted a pull request of my own with the
> fix plus an update to the docs adding info on the 'https_fixups' config.
>
> Mads has commented on that:
>> I consider force_https and the other https_fixups for hacks that are needed
>> because of deficiencies in paster (or somewhere else in the stack we use).
>> WSGI web servers should set wsgi.url_scheme correctly. If the setup has
>> proxies that change protocols, hostnames, ports or paths, then the web
>> server (or in worst case: middleware) should make sure the WSGI environment
>> looks like it didn't. I am thus reluctant to advertise the workarounds we
>> have in Kallithea. We should make it simple to do it the right way.
> This is not something I know much/anything about. But I've done some
> investigation this morning. To recap, https_fixup.py offers 3 configs:
>
> 1. force_https. Always use HTTPS as the protocol for redirects.

The real problem definition here is of course to use https exactly when 
it should use https.

Forcefully overwriting what the web server WSGI tells the application is 
not an elegant solution to that problem. If you support both http and 
https, you must have two different ini files.

A better solution is to make sure that your WSGI server is good enough 
for your needs and can set/fake the WSGI environment as required when 
using a proxy.

> 2. https_fixups. Check for headers X-Url-Scheme, X-Forwarded-Scheme or
>     X-Forwarded-Proto in the request and if found use that protocol.

These custom headers are not standardized and will thus require custom 
configuration of the proxy. Hardcoding these header names in Kallithea 
is also not an elegant solution. If anything, it would be better if 
their names made it clear that they were Kallithea specific custom headers.

PasteDeploy also seems to be able to do this in PrefixMiddleware.

> 3. use_htsts. Add header
>     'Strict-Transport-Security=max-age=8640000; includeSubDomains'
>     to replies, telling compliant clients to use HTTPS for subsequent
>     queries. http://en.wikipedia.org/wiki/HTTP_Strict_Transport_Security.

This is something that is closely related to the actual use of SSL. 
These headers should be added in the proxy if it is the proxy that do 
the SSL encoding. If using a narrow-minded SSL concentrator, it should 
be added close to that. Worst case it could be done in WSGI middleware. 
But doing it inside a WSGI application is the wrong place to do it.

Anyway, we have this not-so-documented option to stay backward compatible.

> (1) and (2) are typically useful when used with reverse proxies. Which is my
> use case.
>
> The default production.ini uses Waitress 0.8.8 as the WSGI server. This can do
> (1) by setting config 'url_scheme=https'.
>
> To do (2) with Waitress requires upgrading to 0.8.9. Traffic from a server
> listed in config 'trusted_proxy' will observe X-Forwarded-Proto.
>
> I've not found a way to do (3) with Waitress.

But waitress can also not do SSL so that is not a big loss. It is better 
to do it and SSL in the proxy, as the wikipedia page also suggests.

> I take Mads' point. Equally, I think working behind a proxy and requiring
> either (1) or (2) is an important use case. So, as I see it, we should:

I think the most obvious solution is to recommend using a web/wsgi 
server that can do what you need it to do.

> a) Fix 'force_https'

Consider it done.

> and document 'https_fixups', or

The name is 'https_fixup' and it is already (briefly) documented in 
docs/setup.rst .

> b) Update to Waitress 0.8.9, change the documentation to specify how to do (1)
>     and (2) via Waitress, and remove the relevant Kallithea code, or

That sounds like a good solution. Depending on how stable and widespread 
that version is, we should consider to just also support that version 
and document how to utilize it.

I think it has value to keep the hacks and stay backward compatible. But 
we should not recommend using them.

> c) Update to Waitress 0.8.9, change the documentation to specify how to do (1)
>     and (2) via Waitress, and document the Kallithea settings in some
>     'Advanced' addendum.
>
> Question: How likely is it that users of another WSGI server will still need
> Kallithea to do (1) and/or (2)?
>
>  From my ignorant PoV, it seems that if Waitress only acquired the ability to
> do (2) in a release made in mid-May this year, the chances are reasonable that
> other WSGI servers will be lacking, and so (a) is the way to go.

/Mads