Fix: apply_overwrites_to_context silently drops remaining overrides after first invalid entry

The Bug

Repo: cookiecutter/cookiecutter Issue: #2219 Status: Fixed PR: https://github.com/cookiecutter/cookiecutter/pull/2230

Description: apply_overwrites_to_context silently drops remaining overrides after the first invalid entry in the default_context dict.

Fix scope: 6 lines changed in cookiecutter/generate.py + 1 assertion regex updated in tests/test_generate_context.py

The generate_context() function loads a template’s cookiecutter.json context file into a Python dict, then applies two layers of overrides: default_context (from the user’s global config) and extra_context (from CLI --no-input arguments or programmatic API calls). Both layers are processed through apply_overwrites_to_context(), which iterates over override entries and applies each one to the in-memory context dict.

The bug lives in how the default_context path handles validation failures. When apply_overwrites_to_context() encounters an invalid value — for example, a choice variable where the override doesn’t match any valid choice, or a multi-choice variable where one selected option isn’t in the allowed set — it raises ValueError. The original calling code in generate_context() wrapped this into a single try/except that caught the error and issued a warning. But by that point, the iteration inside apply_overwrites_to_context() had already aborted, and every remaining entry in default_context was silently discarded.

Root Cause

The edge case lives in cookiecutter/generate.py inside the generate_context() function, specifically in the block that processes default_context. The original code passed the entire default_context dictionary to apply_overwrites_to_context() as one bulk call:

try:
    apply_overwrites_to_context(obj, default_context)
except ValueError as error:
    warnings.warn(f"Invalid default received: {error}")

The apply_overwrites_to_context() function (defined at line 59 of the same file) performs an in-place mutation of the context dict. It iterates through overwrite_context.items() and, for each entry, validates the value against the existing context schema. The function handles several variable types: simple string replacements, choice selections against a list of options, multi-choice selections (subsets of allowed values), dictionary-typed variables with recursive partial overwrites, and boolean conversions.

The validation is strict. For choice variables, the override must exactly match one of the listed choices. For multi-choice variables, every element of the list override must appear in the context’s allowed list. If either condition fails, the function raises ValueError from inside the loop body (lines 89 and 103 of generate.py). Because the loop is a plain for variable, overwrite in overwrite_context.items():, the exception immediately terminates the iteration. The context dict is modified in-place, so any entries processed before the invalid one have already been applied and are retained. But every entry that appears after the invalid key in default_context — no matter how valid — is skipped without any processing or error reporting.

The pattern is easy to miss because standard test suites only test apply_overwrites_to_context with single-entry overwrite contexts or with all-valid entries. The interaction between the bulk-passing pattern in generate_context and the early-exit-on-error behavior in the callee creates a silent data-loss bug that only manifests when the user has multiple default_context entries and at least one early entry is invalid.

Impact: Users who define default_context in their global cookiecutter config with multiple overrides risk having later overrides silently dropped if any earlier entry fails validation. The user receives a warning about the invalid entry, but sees no indication that additional valid entries were lost. This can lead to subtly incorrect project generation where some expected overrides are missing, with no error to signal the problem.

Code Analysis

Before (broken)

In cookiecutter/generate.py, the generate_context() function originally processed default_context as a bulk call:

if default_context:
    try:
        apply_overwrites_to_context(obj, default_context)
    except ValueError as error:
        warnings.warn(f"Invalid default received: {error}")

The apply_overwrites_to_context() function loops over every entry in default_context sequentially:

def apply_overwrites_to_context(
    context: dict[str, Any],
    overwrite_context: dict[str, Any],
    *,
    in_dictionary_variable: bool = False,
) -> None:
    """Modify the given context in place based on the overwrite_context."""
    for variable, overwrite in overwrite_context.items():
        if variable not in context:
            if not in_dictionary_variable:
                continue
            context[variable] = overwrite

        context_value = context[variable]
        if isinstance(context_value, list):
            # ... validation of choice/multi-choice variables ...
            if set(overwrite).issubset(set(context_value)):
                context[variable] = overwrite
            else:
                raise ValueError(
                    f"{overwrite} provided for multi-choice variable "
                    f"{variable}, but valid choices are {context_value}"
                )
            # ... more validation branches that also raise ValueError ...

When entry N raises ValueError, the loop exits immediately. Entries N+1 through the end of default_context are never visited. The warning fires, but only the error message for entry N is reported — no count or indication that entries were skipped.

After (fixed)

The fix iterates over default_context entries one at a time, calling apply_overwrites_to_context with a single-entry dict. Errors are collected into a list and reported together:

if default_context:
    errors: list[str] = []
    for key, value in default_context.items():
        try:
            apply_overwrites_to_context(obj, {key: value})
        except ValueError as error:
            errors.append(str(error))
    if errors:
        warnings.warn(
            f"Invalid default(s) received: {'; '.join(errors)}"
        )

Key changes:

1. Per-entry isolation: Each default_context entry is applied independently. An invalid entry cannot abort processing of subsequent entries.
2. Error aggregation: All validation errors are collected into a list and reported in a single consolidated warning message. The user sees every problematic entry, not just the first one.
3. Preserved warning behavior: The warning is issued at the same UserWarning level, so existing test fixtures and CI monitoring still work. The warning message changed from "Invalid default received" to "Invalid default(s) received" (plural) to signal that multiple errors may be present.

The test assertion in tests/test_generate_context.py was also updated to match the new plural warning message:

# Before:
with pytest.warns(UserWarning, match="Invalid default received"):
# After:
with pytest.warns(UserWarning, match=r"default\(s\) received"):

What about extra_context?

The extra_context path at line 174–175 of generate.py still passes the entire extra_context dict as a single bulk call:

if extra_context:
    apply_overwrites_to_context(obj, extra_context)

This means the same early-exit vulnerability exists for extra_context — if any entry is invalid, the ValueError propagates up uncaught, and all later overrides are lost. However, extra_context typically comes from explicit CLI arguments or programmatic API calls where the user has direct control and is more likely to provide valid values. The default_context path is more dangerous because it reads from a persisted config file that may contain stale or hand-edited entries. A future fix could apply the same per-entry isolation to extra_context for consistency.

The Fix

This is a textbook structural fix — the diff changes the calling pattern rather than the callee logic. The apply_overwrites_to_context function itself is untouched. The fix is entirely in how generate_context() invokes it for default_context:

Bulk call → per-entry loop with error collection

The 6-line net change split a single try/except block into an iterated call pattern. Every line is deliberate:

• The errors: list[str] = [] initialization creates the accumulator.
• The for key, value in default_context.items(): loop unwraps the bulk dict into individual calls.
• The per-entry try/except isolates failures so one bad entry can’t take down the rest.
• The post-loop if errors: guard ensures the warning fires exactly once, with all messages joined.
• The '; '.join(errors) formatting gives the user a complete picture of what was invalid.
• The updated test regex validates the new plural message format.

Pattern & Takeaways

Pattern: Error-handling boundary in cookiecutter/generate.py generate_context() — the caller assumed apply_overwrites_to_context would either fully succeed or fully fail, but the function operates as a partial in-place mutation. The fix demonstrates that when a function can fail partway through a batch operation, the caller must decide whether partial success is acceptable and handle errors at the granularity that matches the desired semantics.

Key insight: The most predictable silent-data-loss bugs occur at the intersection of (1) in-place mutation, (2) early-exit-on-error, and (3) bulk processing. When a function modifies state as a side effect and can raise an exception mid-iteration, the caller has already lost control: state was partially applied, but the exception prevented completion. The fix pattern — iterate individually, collect errors, decide what to do with partial results afterward — is broadly applicable.

Code review lens: When reviewing code that passes a collection to a mutating function, ask: (1) Can this function fail partway through? (2) If it raises, is the caller OK with partial application? (3) Does the caller distinguish between “nothing was applied” and “some things were applied before the error”? The original code treated the entire default_context as an atomic unit, but it wasn’t — apply_overwrites_to_context modifies the context dict as it goes, making partial success the actual behavior whether the caller acknowledges it or not.

Transfer Potential

Varies — This specific fix is about a Python dict mutation pattern in a template engine, but the underlying anti-pattern — bulk-calling a function that partially mutates state and early-exits on error — appears in every language and codebase. Every time you pass a collection to a function that loops and can fail mid-iteration, you introduce the same risk. The fix pattern (iterate individually, aggregate errors) transfers directly to batch processing, bulk API calls, config file loaders, and any pipeline where one bad entry should not abort the entire batch.

Reading this post helps recognize similar partial-mutation vulnerabilities in your own projects — especially in configuration loading, context processing, and any pipeline where a loop modifies state as a side effect.

---

Auto-generated from PR #2230. View all patches on GitHub.