loc vs iloc

Core Principle: Data selection is bifurcated into .loc[] for label-based selection (where endpoint boundaries are inclusive) and .iloc[] for integer positional selection (where endpoint boundaries are half-open).

The two rules

df.loc["TP53":"EGFR"]     # INCLUSIVE of "EGFR"  ← unlike everything else in Python
df.iloc[0:3]              # excludes position 3  ← like Python slicing

The .loc inclusivity is deliberate: with labels, "up to and including EGFR" is what you mean, and there is no "the label after EGFR" to write instead. It is still the thing people forget.

The full syntax

df.loc[row_labels, col_labels]
df.iloc[row_positions, col_positions]

df.loc["TP53"]                        # one row → Series
df.loc[["TP53", "EGFR"]]              # several rows → DataFrame
df.loc["TP53", "log2fc"]              # a scalar
df.loc[:, "log2fc"]                   # a column
df.loc[df["padj"] < 0.05]             # boolean mask
df.loc[df["padj"] < 0.05, ["gene", "log2fc"]]    # mask + columns, ONE call
df.loc[lambda d: d["padj"] < 0.05]    # callable: works inside a chain

df.iloc[0]                            # first row
df.iloc[-1]                           # last row
df.iloc[0:5, 0:3]
df.iloc[[0, 2, 4], [1, 3]]            # position lists: cross-product, unlike NumPy
df.iloc[:, -1]                        # last column

Note that .iloc[[0,2],[1,3]] gives a submatrix, whereas NumPy's a[[0,2],[1,3]] pairs the indices. pandas is doing the np.ix_ thing for you. See Fancy Indexing.

Why bare df[...] is ambiguous

df["gene"]          # a COLUMN
df[0:3]             # ROWS by position
df[df["x"] > 1]     # ROWS by mask
df[["a", "b"]]      # COLUMNS

Four different meanings depending on what you pass. It is convenient interactively and a liability in scripts. Use .loc and .iloc in anything you will re-read.

The trap is sharpest with an integer index:

np.int64(10)

Chained indexing: the pandas 3.0 change

# BROKEN: silently does nothing in pandas 3.0
df[df["padj"] < 0.05]["sig"] = True

# CORRECT
df.loc[df["padj"] < 0.05, "sig"] = True

The first form selects a subset (which under Copy-on-Write is always a new object), then assigns to that temporary, which is discarded. In pandas 2.x this raised SettingWithCopyWarning. In pandas 3.0 the warning is gone: because the behaviour is now defined and consistent: so the failure is completely silent.

The rule: if there is an = in the statement, there must be exactly one [] on the left, and it must belong to .loc or .iloc.

Setting values

df.loc[mask, "col"] = value
df.loc[mask, ["a", "b"]] = [1, 2]
df.iloc[0, 2] = value
df.at["TP53", "log2fc"] = 1.5          # scalar only: faster
df.iat[0, 2] = 1.5                     # positional scalar: faster

.at and .iat bypass the general machinery for single-scalar access. Meaningfully faster in a loop, though if you are in a loop you should probably be vectorising. See pandas Performance.

Enlargement

.loc can create new labels; .iloc cannot.

df.loc["NEWGENE"] = [...]      # adds a row
df.loc[:, "newcol"] = 0        # adds a column
df.iloc[100] = [...]           # IndexError if there is no position 100

Mixing labels and positions

You cannot in one call. Convert:

df.loc[df.index[0:3], "gene"]                # positions → labels
df.iloc[:, df.columns.get_loc("gene")]       # label → position
df.iloc[0:3][["gene"]]                        # two steps (fine for reading)

Common patterns

# top N by a column, keeping all columns
df.nlargest(20, "log2fc")
df.sort_values("padj").iloc[:20]

# a random subsample
df.sample(n=1000, random_state=42)

# a specific set of genes, in a specific order
df.loc[PANEL_GENES]                    # raises if any are missing
df.reindex(PANEL_GENES)                # fills NaN for missing: often what you want

# every column except one
df.loc[:, df.columns != "gene"]
df.drop(columns=["gene"])

df.loc[list_of_labels] raising on a missing label is a feature: it catches typos and stale gene lists. Use reindex when absence is expected and NaN is an acceptable answer.

Bioinformatics examples

expr = pd.read_csv("counts.tsv", sep="\t", index_col=0)   # genes × samples

expr.loc["TP53"]                              # one gene, all samples
expr.loc[PANEL, TREATED_SAMPLES]              # submatrix by label
expr.iloc[:, 0:3]                             # first three samples

# filter genes and keep the labels aligned: the whole point of pandas
keep = (expr > 1).sum(axis=1) >= 3
filtered = expr.loc[keep]

# a genomic window from a position-indexed frame
v = variants.set_index("pos").sort_index()
v.loc[7_670_000:7_680_000]                    # inclusive of both ends

That last line is where .loc inclusivity is actually helpful: a genomic range is conventionally closed, and .loc matches that convention. See Coordinate Systems.

Common mistakes

• Chained assignment. Silent no-op in pandas 3.0. The most important item on this list.
• Forgetting .loc slices are inclusive.
• Bare df[0:3] meaning rows while df["a"] means a column.
• .loc with an integer index meaning label, not position.
• df.loc[missing_label] raising where you expected NaN. Use reindex.
• df.iloc[mask] with a boolean pandas Series. .iloc wants positions or a plain NumPy bool array; pass mask.to_numpy() or use .loc.
• .at on a non-unique index: undefined which row you get.

See also

DataFrame · Series · Index Objects · Copy-on-Write · Filtering and query · Indexing and Slicing

Exercise

Write compare_slices(df, start, end). The frame is indexed by unique gene labels in file order and has one column, value. start and end are labels. Return a list of report lines that shows how the two indexers differ on the same pair of endpoints.

Return ["missing"] if either label is absent from the index, and ["reversed"] if start sits at a later position than end. Otherwise return exactly two lines: loc rows=<n> sum=<total> for the label slice start:end, and iloc rows=<n> sum=<total> for the positional slice running from the position of start to the position of end. Format both totals to two decimal places. When start and end are the same label the two lines must disagree, because the label slice includes its endpoint and the positional slice does not.