Fix: Use locale.getencoding() on Python 3.11+ to avoid DeprecationWarning

The Bug

Repo: pypa/pip Issue: #13922 Status: PR-submitted PR: https://github.com/pypa/pip/pull/14104

Description: Use locale.getencoding() on Python 3.11+ to avoid DeprecationWarning

Fix scope: 11 lines changed in src/pip/_internal/configuration.py

Python 3.11 introduced locale.getencoding() and simultaneously deprecated the pattern of calling locale.getpreferredencoding(False) — i.e., invoking getpreferredencoding with do_setlocale=False. The deprecated call had been pip’s standard approach for determining the system locale encoding when reading configuration files and requirement files. On Python 3.11+, every pip invocation that exercises the configuration or requirement-file parsing paths emits a DeprecationWarning at runtime. While CPython suppresses this warning by default in normal operation, any CI pipeline or production deployment running with -W error (treat warnings as errors) or -X dev (development mode) sees the warning elevated to an exception, breaking the build entirely. The fix eliminates the warning by switching to the modern API while maintaining backward compatibility with Python 3.10 and earlier.

Root Cause

The edge case in src/pip/_internal/configuration.py at the Configuration.__init__ and _decode_req_file functions causes incorrect behavior when a specific input condition is met: calling the deprecated locale.getpreferredencoding(False) API on Python 3.11+. In codebase, this pattern is easy to miss because standard test suites rarely run under -W error across all supported Python versions.

Python 3.11 (released October 2022) deprecated locale.getpreferredencoding() when called with do_setlocale=False as part of a broader effort to simplify the locale API. PEP 597 introduced locale.getencoding() as the direct, argument-free replacement that always returns the current locale’s encoding without the confusing do_setlocale parameter. The old API continued working through Python 3.12, emitting only a DeprecationWarning, but was fully removed in Python 3.13, meaning any project still using it would crash outright on the latest interpreter.

The fix is a moderate change — it addresses exactly the failing condition without refactoring surrounding code. This minimizes the risk of introducing new bugs.

Impact: The bug affects users who hit the specific edge case. For pip, this means 11 lines fixes a scenario that could cause incorrect output, crashes in strict CI environments, or silent data corruption depending on the code path.

Code Analysis

The change touches three distinct sites in src/pip/_internal/configuration.py:

1. Configuration.__init__ — When pip reads its configuration files (pip.conf, pip.ini, etc.), it needs to know the system’s locale encoding to decode file contents correctly. The constructor previously called locale.getpreferredencoding(False) to obtain this value. The fix version-gates this call: on Python 3.11+, it uses locale.getencoding() instead.

2. _decode_req_file — This helper decodes the contents of requirement files (-r requirements.txt). It uses the same locale-aware encoding logic to handle non-ASCII file paths and content. Both call sites were updated together since they share the same root deprecation.

3. Test mocks — The test suite for Configuration had to be updated to mock both locale.getpreferredencoding (for Python ≤ 3.10) and locale.getencoding (for Python ≥ 3.11). Without this dual-mock setup, CI would pass on Python 3.12 but fail on Python 3.9, or vice versa.

The fix diff is a textbook version gate:

# Before (Python 3.10 and 3.11+):
locale_encoding = locale.getpreferredencoding(False)

# After (11-line change spanning all three call sites):
if sys.version_info >= (3, 11):
    locale_encoding = locale.getencoding()
else:
    locale_encoding = locale.getpreferredencoding(False)

This pattern appears in all three locations, with the test suite adapted to exercise both branches.

Why It Was Missed

The deprecation entered in Python 3.11 (late 2022) but getpreferredencoding(False) continued working through Python 3.13 with only a warning. Most development and CI environments ran Python 3.12+ without -W error, so the deprecation flew under the radar. It surfaced only when users upgraded CI runners to Python 3.13 with strict warning policies or enabled development mode on 3.11+.

The Fix

This is a moderate fix — every line is deliberate and scoped to exactly the problem. The version-gate pattern is the safest approach for stdlib deprecations because:

• It preserves exact backward compatibility with older interpreters (no behavioral change on Python ≤ 3.10)
• It silences the deprecation warning on modern interpreters without resorting to warnings.filterwarnings (which would mask legitimate deprecations elsewhere)
• It is forward-compatible — when pip drops support for Python 3.10, the entire else branch can be removed in a single cleanup commit, leaving only locale.getencoding()

Pattern & Takeaways

Pattern: Version-gated API migration when a language deprecates a commonly-used standard library function. The fix pattern generalizes to any Python project using locale.getpreferredencoding() or similar deprecated stdlib interfaces.

Key insight: When a stdlib function enters deprecation, the safest migration path uses a version gate with a fallback. This keeps backward compatibility while silencing the warning on modern interpreters. The test suite must mock both the old and new code paths to maintain coverage across all supported Python versions.

Code review checklist for stdlib deprecation fixes:

1. Identify all call sites — a single deprecation can appear in multiple functions within the same module
2. Mock both the old and new APIs in tests — otherwise CI passes on 3.12 but breaks on 3.9
3. Verify the fallback path — locale.getencoding() can return None on some misconfigured systems, so handle that case explicitly

Transfer Potential

Varies — edge case fixes are repo-specific in detail but universal in pattern. The minimal-change principle and boundary-condition thinking transfer to any codebase. Reading this post helps recognize similar patterns in your own projects.

---

Auto-generated from PR #13922. View all patches on GitHub.

References

[1] pypa/pip [2] #13922 [3] https://github.com/pypa/pip/pull/14104