Copy-on-Write

Core Principle: Copy-on-Write (CoW) ensures that indexing operations produce logically independent views whose underlying memory is copied only upon modification, eliminating chained assignment bugs and unexpected mutations.

This is the biggest behavioural change in pandas' history. It is worth understanding properly because it silently changes what old code does.

The old problem

Before pandas 3.0, whether df2 = df[df["x"] > 1] gave you a view or a copy depended on the dtypes, the memory layout, and internal block-manager details you had no visibility into. So this:

df2 = df[df["x"] > 1]
df2["y"] = 0

might modify df, or might not, depending on the data. pandas' answer was SettingWithCopyWarning: a warning famous for firing when nothing was wrong and staying silent when something was.

The new rule

Any object derived from another via an indexing operation never modifies the original.

df2 = df[df["x"] > 1]
df2["y"] = 0
# df is guaranteed unchanged. Always. Regardless of dtypes.

SettingWithCopyWarning no longer exists, because the ambiguity it warned about is gone.

Why "copy-on-write" and not just "copy"

The copy is lazy. df2 = df[cols] initially shares memory with df; pandas tracks the reference. Only when you write to df2 does it materialise its own buffer.

So the common read-only case: slice a frame, compute something, discard it: costs nothing. In many workloads pandas 3.0 uses less memory than 2.x, because 2.x made defensive copies eagerly to avoid the ambiguity.

What breaks

Chained assignment: silently

# pandas 2.x: worked sometimes, warned sometimes
# pandas 3.0: NEVER works, NO warning
df["col"][df["x"] > 5] = 100
df[df["x"] > 5]["col"] = 100
df.loc[df["x"] > 5]["col"] = 100

# correct
df.loc[df["x"] > 5, "col"] = 100

This is the one that matters. Old scripts that "worked" now produce unmodified data with no error, and the wrong result flows downstream.

The check: in any statement with =, there must be exactly one [] on the left-hand side, attached to .loc or .iloc.

Mutating an extracted Series

s = df["col"]
s.iloc[0] = 999          # df is NOT modified

df.loc[df.index[0], "col"] = 999    # correct

inplace=True is no longer a memory optimisation

df.fillna(0, inplace=True)     # still copies internally
df = df.fillna(0)              # equivalent, and chainable

inplace=True was always a performance myth; now it is unambiguously one. It also breaks method chaining and returns None, which produces the classic AttributeError: 'NoneType' object has no attribute .... Stop using it.

df.values no longer lets you write through

arr = df.to_numpy()
arr[0, 0] = 999          # does not affect df

What this means in practice

You can stop calling .copy() defensively.

# pandas 2.x habit
subset = df[df["x"] > 1].copy()
subset["new"] = ...

# pandas 3.0: the .copy() is redundant
subset = df[df["x"] > 1]
subset["new"] = ...

Functions can no longer accidentally mutate their arguments through an indexing operation:

Direct column assignment on the object you were handed still mutates it: CoW governs derived objects, not the object itself. Prefer returning a new frame:

Migration checklist

1. Run your code under pandas 2.3 first with pd.options.mode.copy_on_write = True and fix everything that warns.
2. Grep for ][ on the left of an =.
3. Grep for inplace=True and remove it.
4. Grep for .values[: writes through .values no longer propagate.
5. Check anywhere you relied on a function mutating a frame you passed in.
6. Compare outputs numerically before and after. The failure mode is silence, so a diff of your results is the only real test.

That last point is the important one. A migration that "runs without errors" tells you nothing here.

The other pandas 3.0 changes

While you are migrating, two more:

• String columns are str dtype, not object. df.dtypes == object no longer finds text columns. Use pd.api.types.is_string_dtype(). See pandas dtypes.
• Datetimes default to microsecond resolution, not nanosecond: which removes the old 1678–2262 range limit.

Common mistakes

• Assuming your code is fine because it runs. Chained assignment fails silently.
• Keeping .copy() everywhere out of habit: harmless, but noise.
• Thinking direct column assignment on a passed-in frame is now safe. It is not; that is not an indexing result.
• inplace=True for speed.
• Writing through .to_numpy().
• Not diffing your results across the upgrade.

See also

loc vs iloc · DataFrame · Series · Views vs Copies · pandas dtypes · Versions and Compatibility · pandas Performance

Exercise

Write flag_significant(df, alpha), which returns a new frame carrying two extra columns and leaves the caller's frame untouched. Direct column assignment on the frame you were handed still mutates it: Copy-on-Write governs objects derived by indexing, not the argument itself: so build the new columns with assign and set values on the result, never on df.

The added columns are sig, true where padj is present and less than alpha, and label, which is hit where sig is true, na where padj is missing, and ns otherwise. Set label with a single .loc[mask, "label"] assignment per case; a chained assignment such as out[mask]["label"] = "hit" writes to a temporary and is a silent no-op. The caller prints the argument's columns after the call, the returned frame's columns, the number of hits and the labels in row order, so a function that mutates its argument produces the wrong first line.