MultiIndex

In one line: a hierarchical index — several label levels per row — which lets you represent higher-dimensional data in a 2-D frame.

Creating one

df.set_index(["chrom", "pos"])
df.groupby(["gene", "sample"]).mean()          # grouping by 2+ keys produces one
df.pivot_table(index="gene", columns=["tissue", "condition"])

pd.MultiIndex.from_tuples([("chr1", 100), ("chr1", 200)], names=["chrom", "pos"])
pd.MultiIndex.from_product([["chr1", "chr2"], [100, 200]], names=["chrom", "pos"])
pd.MultiIndex.from_arrays([chroms, positions], names=["chrom", "pos"])

from_product is how you build a complete grid — every chromosome × every bin — which is useful for reindexing sparse results into a full matrix.

Selecting

df.loc["chr1"]                        # whole first level → drops that level
df.loc[("chr1", 7676154)]             # a full key
df.loc[("chr1", 7676154), "ref"]      # key + column
df.loc["chr1":"chr3"]                 # slice the outer level (needs sorting)

# partial selection on inner levels needs slice(None) or IndexSlice
idx = pd.IndexSlice
df.loc[idx[:, 7676154], :]            # all chromosomes, that position
df.loc[idx["chr1", 100:200], :]

df.xs("chr1", level="chrom")          # cross-section, drops the level
df.xs(7676154, level="pos")           # select on an inner level directly

.xs is the readable option for "give me one value of one level". IndexSlice is more general but harder to read.

Sort before you slice

df = df.sort_index()
df.index.is_monotonic_increasing      # must be True for range slicing

Slicing an unsorted MultiIndex raises UnsortedIndexError, or is quietly slow. Sort immediately after creating one. This is the single most common MultiIndex frustration.

Level manipulation

df.index.names                            # ['chrom', 'pos']
df.index.get_level_values("chrom")        # flat array of that level
df.index.levels                            # the unique values per level

df.reset_index(level="pos")               # one level → column
df.reset_index()                           # all levels → columns
df.swaplevel("chrom", "pos").sort_index()
df.droplevel("chrom")
df.reorder_levels(["pos", "chrom"])

Grouping and stacking

df.groupby(level="chrom").size()
df.groupby(level=[0, 1]).mean()
df.sum(level=...)                          # removed — use groupby(level=...)

df.stack()      # columns → index level   (wider → longer)
df.unstack()    # index level → columns   (longer → wider)
df.unstack(level="condition")
df.unstack(fill_value=0)

stack/unstack are the MultiIndex-aware cousins of melt/pivot. See Reshaping with pivot and melt.

MultiIndex on columns

Just as common, and often the more useful direction:

expr.columns = pd.MultiIndex.from_arrays(
    [tissues, conditions, replicates], names=["tissue", "condition", "rep"]
)

expr["liver"]                                        # all liver samples
expr.loc[:, idx[:, "treated", :]]                    # all treated, any tissue
expr.T.groupby(level=["tissue", "condition"]).mean().T   # collapse replicates

That last line — averaging technical replicates within condition — is a genuinely elegant use of a column MultiIndex, and awkward without one.

When it earns its complexity

Worth it:

• Genuinely hierarchical keys: (chromosome, position), (gene, transcript, exon), (patient, timepoint).
• The output of a multi-key groupby you will immediately unstack.
• Repeated selection on the outer level of a large frame — the sorted index makes it fast.
• Column hierarchies over a wide experimental design.

Not worth it:

• As a habit. Flat columns plus groupby handles most work and is far easier to read, merge and debug.
• When you will immediately merge with something else. merge on MultiIndexes is fiddly; reset_index() first.
• When collaborators will read your code. MultiIndex selection syntax has a real learning cost.

The honest advice: reach for it when the data is hierarchical and you will exploit that; otherwise reset_index() and move on.

Bioinformatics examples

# a long-format expression table indexed by gene and sample
long = counts.melt(id_vars="gene", var_name="sample", value_name="count")
mi = long.set_index(["gene", "sample"]).sort_index()

mi.loc["TP53"]                              # all samples for one gene
mi.loc[("TP53", "sample_3"), "count"]       # one value
mi.groupby(level="gene")["count"].mean()

# variants keyed by locus
v = variants.set_index(["chrom", "pos"]).sort_index()
v.loc["chr17"]                              # all chr17 variants
v.loc[("chr17", slice(7_670_000, 7_680_000))]   # a genomic window

# a summary table that reads well
summary = de.groupby(["tissue", "direction"])["gene"].count().unstack(fill_value=0)

Common mistakes

• Slicing without sorting → UnsortedIndexError.
• Forgetting the tuple. df.loc["chr1", 7676154] is interpreted as (row, column), not as a two-level key. You need df.loc[("chr1", 7676154)].
• Levels retaining unused categories after filtering — df.index.levels still lists them. Use df.index.remove_unused_levels().
• Merging on a MultiIndex without resetting first.
• to_csv round-trip — a MultiIndex writes fine but needs index_col=[0, 1] on read.
• Using it when flat columns would do. Complexity you pay for on every subsequent line.

See also

Index Objects · groupby · Reshaping with pivot and melt · loc vs iloc · DataFrame