binascii base64 & base32 decode ignore excess padding bits in their input

[gpshead]

Bug description:

CPython has had this behavior forever on these, but base64 and base32 decode both ignore excess bits in their input data. Per https://datatracker.ietf.org/doc/html/rfc4648.html#section-3.5 "decoders MAY chose to reject an encoding if the pad bits have not been set to zero".

a2b_base64() has a strict_mode= option, so doing this in that mode makes sense.

base32 is documented as always being strict so just fixing its glitch there makes sense to me. We could add a strict_mode= parameter to a2b_base32() but that would be a little sad. I doubt anyone depends on a behavior like this. It is unnatural to have excess bits in encoded data because encoders MUST (per the rfc) set those bits to 0.

This is a follow on to these recent changes:

• Excess Base64 data ignored after padding by default #145264
• C accelerator for Base32 character encoding #146192

CPython versions tested on:

3.15alpha

Linked PRs

• gh-146311: Reject non-canonical padding bits in base32, 64, & 85 decoding #146312

[gpshead]

@serhiy-storchaka thoughts on what to do about base32 here? it is an observable behavior change, but I don't intend to backport this. Should it gain a strict_mode= parameter like base64 has so that anyone depending on the behavior has an easy way out?

[serhiy-storchaka]

This option is the least important of all RFC 4648 variations, because it is off by default. It is "decoders MAY chose to reject", not "decoders MAY chose to ignore". On other hand, all other strictness checks should be enabled by default. I think that strict_mode should be True by default, strict_mode=False should only be supported for backward compatibility, but there should be other options to disable different strictness checks. For example, ignorechars covers some of such options (ignoring newlines/whitespaces/non-alphabetic characters/"=" not at the end).

I think that:

• it is not a big deal if we continue to ignore the lowest bits in the last quad for a while
• if we add an option, it should be independent from strict_mode.

There are still several issues with higher priority (#73613, #71986/#76672/#64331). This issue can wait.

[gpshead]

Given the RFC section is "Canonical Encoding" naming a kwarg for this canonical= has some appeal to it.

…

On Sun, Mar 22, 2026 at 5:43 PM Serhiy Storchaka ***@***.***> wrote: *serhiy-storchaka* left a comment (python/cpython#146311) <#146311 (comment)> This option is the least important of all RFC 4648 variations, because it is off by default. It is "decoders MAY chose to reject", not "decoders MAY chose to ignore". On other hand, all other strictness checks should be enabled by default. I think that strict_mode should be True by default, strict_mode=False should only be supported for backward compatibility, but there should be other options to disable different strictness checks. For example, ignorechars covers some of such options (ignoring newlines/whitespaces/non-alphabetic characters/"=" not at the end). I think that: - it is not a big deal if we continue to ignore the lowest bits in the last quad for a while - if we add an option, it should be independent from strict_mode. There are still several issues with higher priority (#73613 <#73613>, #71986 <#71986>/#76672 <#76672>/#64331 <#64331>). This issue can wait. — Reply to this email directly, view it on GitHub <#146311?email_source=notifications&email_token=AAAQXC6NTJFKDB5KB2JNAR34SCCCNA5CNFSNUABFM5UWIORPF5TWS5BNNB2WEL2JONZXKZKDN5WW2ZLOOQXTIMJQG4ZTIOBVGQ22M4TFMFZW63VGMFZXG2LHN2SWK5TFNZ2KYZTPN52GK4S7MNWGSY3L#issuecomment-4107348545>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAAQXC2DLJ4LJ5YB3R7OBED4SCCCNAVCNFSM6AAAAACW27G4ZGVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHM2DCMBXGM2DQNJUGU> . You are receiving this because you were assigned.Message ID: ***@***.***>

[serhiy-storchaka]

Yes, that was the first thing that came to my mind. We should keep this in mind, but the final name will be chosen after adding options for other features.

[serhiy-storchaka]

This is relevant also for Base85. It cannot be formulated in terms of bits, but decoding different inputs can produce the same output.

>>> binascii.a2b_base85(b'VE')
b'a'
>>> binascii.a2b_base85(b'VF')
b'a'

Even more, Base85 does not produce encoded data of length 1 modulo 5. But the decoder accepts it and silently ignores that redundant character.

>>> binascii.a2b_base85(b'')
b''
>>> binascii.a2b_base85(b'V')
b''

Base32 and Base64 decoders are slightly more strict.

[gpshead]

True, being consistent would be good. draft PR updated.

[serhiy-storchaka]

This is not so simple for Ascii85.

• The document that specifies Ascii85 (https://www.adobe.com/jp/print/postscript/pdfs/PLRM.pdf, section 3.13.3) does not have restriction on unambiguousity of representation of the last incomplete group.
• There may be ambiguous representation in the middle -- a group of 4 zeroes should be represented as 'z', but we do not reject '00000' which is decoded to the same value.
• Finally, any whitespaces are ignored, so we can encode a lot of hidden information by inserting whitespaces.

I think it would be nice to add such option, but we should be careful with its description.

Base85 is less fluid by default, so we can speak about "canonical" representation, except that it is not officially defined, it is just projection from Base32 and Base64. Maybe use "canonical" in quotes?