Copy-on-Write
In one line: since pandas 3.0, every indexing result behaves as an independent copy — the actual copying is deferred until you write, so it is both safer and faster.
This is the biggest behavioural change in pandas' history. It is worth understanding properly because it silently changes what old code does.
The old problem
Before pandas 3.0, whether df2 = df[df["x"] > 1] gave you a view or a copy depended on the dtypes, the memory layout, and internal block-manager details you had no visibility into. So this:
df2 = df[df["x"] > 1]
df2["y"] = 0
might modify df, or might not, depending on the data. pandas' answer was SettingWithCopyWarning — a warning famous for firing when nothing was wrong and staying silent when something was.
The new rule
Any object derived from another via an indexing operation never modifies the original.
df2 = df[df["x"] > 1]
df2["y"] = 0
# df is guaranteed unchanged. Always. Regardless of dtypes.
SettingWithCopyWarning no longer exists, because the ambiguity it warned about is gone.
Why "copy-on-write" and not just "copy"
The copy is lazy. df2 = df[cols] initially shares memory with df; pandas tracks the reference. Only when you write to df2 does it materialise its own buffer.
So the common read-only case — slice a frame, compute something, discard it — costs nothing. In many workloads pandas 3.0 uses less memory than 2.x, because 2.x made defensive copies eagerly to avoid the ambiguity.
What breaks
Chained assignment — silently
# pandas 2.x: worked sometimes, warned sometimes
# pandas 3.0: NEVER works, NO warning
df["col"][df["x"] > 5] = 100
df[df["x"] > 5]["col"] = 100
df.loc[df["x"] > 5]["col"] = 100
# correct
df.loc[df["x"] > 5, "col"] = 100
This is the one that matters. Old scripts that "worked" now produce unmodified data with no error, and the wrong result flows downstream.
The check: in any statement with =, there must be exactly one [] on the left-hand side, attached to .loc or .iloc.
Mutating an extracted Series
s = df["col"]
s.iloc[0] = 999 # df is NOT modified
df.loc[df.index[0], "col"] = 999 # correct
inplace=True is no longer a memory optimisation
df.fillna(0, inplace=True) # still copies internally
df = df.fillna(0) # equivalent, and chainable
inplace=True was always a performance myth; now it is unambiguously one. It also breaks method chaining and returns None, which produces the classic AttributeError: 'NoneType' object has no attribute .... Stop using it.
df.values no longer lets you write through
arr = df.to_numpy()
arr[0, 0] = 999 # does not affect df
What this means in practice
You can stop calling .copy() defensively.
# pandas 2.x habit
subset = df[df["x"] > 1].copy()
subset["new"] = ...
# pandas 3.0 — the .copy() is redundant
subset = df[df["x"] > 1]
subset["new"] = ...
Functions can no longer accidentally mutate their arguments through an indexing operation:
def add_flags(df):
df["sig"] = df["padj"] < 0.05 # ← this DOES still modify the caller's frame
return df
Direct column assignment on the object you were handed still mutates it — CoW governs derived objects, not the object itself. Prefer returning a new frame:
def add_flags(df):
return df.assign(sig=lambda d: d["padj"] < 0.05)
Migration checklist
- Run your code under pandas 2.3 first with
pd.options.mode.copy_on_write = Trueand fix everything that warns. - Grep for
][on the left of an=. - Grep for
inplace=Trueand remove it. - Grep for
.values[— writes through.valuesno longer propagate. - Check anywhere you relied on a function mutating a frame you passed in.
- Compare outputs numerically before and after. The failure mode is silence, so a diff of your results is the only real test.
That last point is the important one. A migration that "runs without errors" tells you nothing here.
The other pandas 3.0 changes
While you are migrating, two more:
- String columns are
strdtype, notobject.df.dtypes == objectno longer finds text columns. Usepd.api.types.is_string_dtype(). See pandas dtypes. - Datetimes default to microsecond resolution, not nanosecond — which removes the old 1678–2262 range limit.
Common mistakes
- Assuming your code is fine because it runs. Chained assignment fails silently.
- Keeping
.copy()everywhere out of habit — harmless, but noise. - Thinking direct column assignment on a passed-in frame is now safe. It is not; that is not an indexing result.
inplace=Truefor speed.- Writing through
.to_numpy(). - Not diffing your results across the upgrade.
See also
loc vs iloc · DataFrame · Series · Views vs Copies · pandas dtypes · Versions and Compatibility · pandas Performance