Fix: HTTPDigestAuth for Non-Latin Credentials

The Bug

Repo: psf/requests Issue: #6102 Status: PR-submitted PR: https://github.com/psf/requests/pull/7450

Description: HTTPDigestAuth fails on non-latin credentials

Fix scope: 4 lines changed in requests/auth.py

The Fix

This is a focused fix — every line is deliberate and scoped to exactly the problem [1].

 if isinstance(username, str):
-    username = username.encode("latin1")
+    username = username.encode("utf-8")

 if isinstance(password, str):
-    password = password.encode("latin1")
+    password = password.encode("utf-8")

What This Teaches

Encoding assumptions in HTTP libraries are a common source of UnicodeErrors. The latin1 (ISO-8859-1) encoding only supports 256 characters — any character outside that range (Cyrillic, CJK, emoji, etc.) throws a UnicodeEncodeError.

The HTTP spec allows UTF-8 in header values, so latin1 was an unnecessary restriction. Modern Python defaults to UTF-8 for a reason: it handles the full Unicode space.

Pattern: When a library function encodes strings for transmission, check if the encoding choice predates Unicode-awareness. Switching latin1 → utf-8 is often the fix.

How to Detect This in Your Codebase

Run this grep to find the same encoding bug in your own HTTP auth code:

grep -rn '\.encode("latin1")' --include='*.py' .
grep -rn '\.encode("latin-1")' --include='*.py' .

Any result where the encoded string comes from user input (credentials, header values, form data) is a candidate for the same fix. Also check for latin1 in email, http.client, and urllib modules — the pattern repeats across Python’s standard library. For HTTP auth specifically, the requests documentation on authentication shows where digest auth encodes credentials — if your project uses custom auth handlers, audit the encoding path.

How to Apply the Fix

1. Find the encoding call: text.encode("latin1") or text.encode("latin-1")
2. Verify the encoded value is used in an HTTP header or body (where UTF-8 is valid per spec)
3. Replace with text.encode("utf-8")
4. Add a test with non-Latin characters: username="用户", password="пароль"
5. Run existing auth tests to confirm no regression on ASCII-only inputs

Transfer Potential

Would reading this post help fix a similar bug in another repo?

High — this exact pattern (latin1 encoding in HTTP auth) appears in multiple HTTP client libraries. The same fix applies to any Python project that encodes user-provided strings for HTTP headers.

References

[1] psf/requests#6102 — HTTPDigestAuth fails on non-Latin username/password. The original issue report describes UnicodeEncodeError when using Cyrillic, CJK, or other non-Latin credentials with HTTPDigestAuth.

[2] requests documentation — Authentication. The official docs show how digest auth encodes credentials, confirming the encoding happens in the library’s auth module.

---

Auto-generated from PR #7450. View all patches on GitHub.