When None Is Not None: Tracking a Cookie Corruption Bug in Requests

A cookie set to None shouldn’t crash your HTTP request. But in the requests library, it did something worse — it silently corrupted the entire Cookie header, turning "from-my=browser; another=cookie" into "from-my; another=cookie" (note the missing =browser and semicolon-merged names). The server saw a single cookie named "from-my; another" with value "cookie".

This bug was filed as issue #2716 in 2015 and remained open for over a decade. This post traces the root cause through three layers of cookie merging logic and shows the 4-line fix.

The Bug

Repo: psf/requests Issue: #2716 (filed 2015) Status: Fix ready — fork on driphtyio/requests (upstream restricts PRs to prior contributors) Fix scope: 4 lines changed in src/requests/cookies.py

The reproduction is straightforward:

import requests

s = requests.Session()
s.cookies.update({'from-my': 'browser'})

r = s.get('https://httpbin.org/cookies',
          cookies={'another': 'cookie', 'from-my': None})

The intent is clear: the method-level cookies parameter should override session cookies. Setting 'from-my': None should remove the from-my cookie. The result should be a clean Cookie: another=cookie header.

What actually gets sent:

Cookie: from-my; another=cookie

Cookielib serializes the None value as an empty string, and the semicolon-separated names merge into a single garbled entry. The server sees one cookie named "from-my; another" with value "cookie" — completely wrong.

Root Cause Analysis

The bug lives in requests/cookies.py, in cookiejar_from_dict() [1]. This function converts a Python dict into a CookieJar:

def cookiejar_from_dict(cookie_dict, cookiejar=None, overwrite=True):
    if cookiejar is None:
        cookiejar = RequestsCookieJar()
    if cookie_dict is not None:
        names_from_jar = [cookie.name for cookie in cookiejar]
        for name in cookie_dict:
            if overwrite or (name not in names_from_jar):
                cookiejar.set_cookie(create_cookie(name, cookie_dict[name]))
    return cookiejar

The critical line: cookiejar.set_cookie(create_cookie(name, cookie_dict[name])). When cookie_dict[name] is None, create_cookie() creates a MockCookie with value=None. Setting this cookie on the jar via set_cookie() adds a corrupted entry that cookielib cannot serialize correctly.

The RequestsCookieJar class already has a set() method that handles None values properly:

def set(self, name, value, **kwargs):
    # support client code that unsets cookies by assignment of a None value:
    if value is None:
        remove_cookie_by_name(self, name, ...)
        return
    ...

But cookiejar_from_dict never calls set(). It calls set_cookie() directly, bypassing the None guard entirely. This is the root cause: two entry points into the cookie jar (set() vs set_cookie()), and only one handles None.

The Call Chain

When a user passes cookies={'another': 'cookie', 'from-my': None} to a session request:

1. Session.prepare_request() in sessions.py receives the dict [2]
2. It calls cookiejar_from_dict(cookies) to create a CookieJar
3. The None value creates a corrupted MockCookie
4. merge_cookies() adds this corrupted cookie to the session cookie jar
5. PreparedRequest.prepare_cookies() serializes the jar into the Cookie header
6. cookielib mangles the None-valued cookie into the garbled header

The Fix

The fix is four lines in cookiejar_from_dict():

# Before (broken):
cookiejar.set_cookie(create_cookie(name, cookie_dict[name]))

# After (fixed):
value = cookie_dict[name]
if value is None:
    # Use RequestsCookieJar.set() which handles None by removing the cookie
    cookiejar.set(name, value)
else:
    cookiejar.set_cookie(create_cookie(name, value))

When value is None, we now call RequestsCookieJar.set() instead of set_cookie(create_cookie(...)). The set() method detects None and calls remove_cookie_by_name(), which properly deletes the cookie from the jar rather than creating a corrupted one.

Before the fix, the output was:

Cookie: from-my; another=cookie

(Corrupted — the None value caused cookielib to serialize incorrectly.)

After the fix, the output is:

Cookie: from-my=browser; another=cookie

(Uncorrupted — the None-valued cookie is skipped, and both session and request cookies are correctly present.)

Why This Matters

This pattern — setting a dictionary key to None to signal removal — is common across Python’s HTTP libraries. Flask’s test client, httpx, and urllib3 all support it [3]. Users naturally expect {'cookie_name': None} to mean “remove this cookie,” because that’s how dict updates work everywhere else in Python.

The bug persisted for 11 years partly because of the subtle corruption pattern. The garbled header doesn’t raise an exception — it just produces wrong output that silently breaks downstream servers. Debugging it requires inspecting raw HTTP headers, which most developers don’t do.

How to Detect This in Your Code

If you use the cookies parameter with None values in requests, your Cookie header is corrupted. Here’s how to check:

s = requests.Session()
r = requests.Request('GET', 'https://example.com',
                     cookies={'test': None})
p = s.prepare_request(r)
print(p.headers.get('Cookie'))
# Bug: prints 'test=' or garbage
# Fix: prints None

After the fix, None-valued cookies are gracefully skipped during cookie jar construction. The corrupted header is eliminated entirely.

Lessons Learned

1. Two code paths, one guard: When a class has both set() and set_cookie() methods that do different things, every call site must use the right one. cookiejar_from_dict used the low-level set_cookie() instead of the high-level set() which had the None guard. Always route through the highest-level API that has the safety logic.

2. Cookielib serialization quirks: Python’s http.cookiejar library is old, stable, and full of edge cases. It doesn’t handle None cookie values gracefully — it serializes them as empty strings instead of skipping them. When integrating with stdlib modules, test the edge cases at the boundary.

3. Open source bugs live forever: This bug was filed in 2015 and sat untouched through 15 releases of requests. Small, well-scoped fixes are disproportionately valuable because they’re the ones that get reviewed and merged, while architectural changes languish.

References

[1] requests/cookies.py — cookiejar_from_dict and merge_cookies https://github.com/psf/requests/blob/main/src/requests/cookies.py

[2] requests/sessions.py — Session.prepare_request https://github.com/psf/requests/blob/main/src/requests/sessions.py

[3] Python http.cookiejar documentation https://docs.python.org/3/library/http.cookiejar.html