[PATCH] fix: UnicodeDecodeError: can't decode byte 0xad

[Valentin Kleibel]

Hi,

> We could also catch this exception in the big try-except clause in 
> __call__, and we could catch the more generic UnicodeError. But that 
> would perhaps catch too much - also things that really are programming 
> errors and shouldn't give a 400 reply.

I agree that this would catch too much.


> I think I would prefer to just catch this Unicode error if it happens, 
> rather than trying to trigger it early. Perhaps by wrapping the call of 
> _basic_security_checks. Do  you think that would catch too much or too 
> little?

I would prefer to catch this in webob where user provided data is 
decoded and raise a more specific error there...
I just checked the code and noticed that request.GET will contain the 
parsed URL parameters for all request methods, so no reason to access 
request.POST.
I think good options are:
* wrap the _basic_security_checks call
* wrap the check if webutils.session_csrf_secret_name in request.GET: in 
_basic_security_checks
* trigger it early in a separate check on request.GET

As webob will only do the processing once and store the processed GET 
for further access i still think triggering early in a separate check 
would be the cleaner option.
I'm open to provide or test a patch using any of the options above.

Thanks for your help,
Valentin


> On 26/08/2024 17:06, Valentin Kleibel wrote:
>> Hi,
>>
>> we have recently noticed a lot of errors in Kallithea from probing for 
>> a php vulnerability [1] looking like:
>> "WebApp Error: UnicodeDecodeError: 'utf-8' codec can't decode byte 
>> 0xad in position 0: invalid start byte"
>>
>> This can be reproduced with curl:
>> curl https://example.com/?%AD
>>
>> The error stems from webob naively trying to utf-8 decode all 
>> %-encoded bytes in URL-parameters.
>>
>> In my opinion this exception should be handled and a error 400 should 
>> be returned.
>>
>> Attached you can find a small patch i created to check for this in 
>> kallithea/controllers/base.py:_basic_security_checks().
>>
>> Best Regards,
>> Valentin
>>
>>
>> [1] 
>> https://devco.re/blog/2024/06/06/security-alert-cve-2024-4577-php-cgi-argument-injection-vulnerability-en/
>>
>> _______________________________________________
>> kallithea-general mailing list
>> kallithea-general at sfconservancy.org
>> https://lists.sfconservancy.org/mailman/listinfo/kallithea-general
> 
>