Fancy Indexing

Core Principle: Integer array indexing (fancy indexing) extracts elements using arbitrary integer sequences, supporting permutations and repeated indices while always returning an independent memory copy.

The basics

Contrast with slicing, which can only express regular strides. Fancy indexing can express any gather, which is why it must copy.

The shape rule

The result has the shape of the index array, not the source array.

This is what makes fancy indexing a lookup-table operation:

Multi-dimensional: pairing, not cross-product

This is the trap.

Two index arrays are broadcast against each other and paired elementwise. If you wanted the submatrix formed by rows {0,2} and columns {1,3}, use np.ix_, or index in two steps (m0,2[:, [1,3]], which copies twice).

You can also do it manually with broadcasting:

which is exactly what np.ix_ builds for you. See Broadcasting.

Sorting and ordering

The most common real use of fancy indexing is applying a sort order:

order = np.argsort(pvalues)          # indices that would sort
sorted_p = pvalues[order]
sorted_genes = gene_names[order]     # ← same permutation, labels stay aligned

top10 = np.argsort(pvalues)[:10]
np.argpartition(pvalues, 10)[:10]    # O(n), unsorted top-10: much faster for large n

np.argmax(x); np.argmin(x)           # single best

argpartition is the one people do not know about. For "give me the 100 most significant genes out of 20,000", it avoids the full sort.

Assignment through fancy indices

The duplicate-index case matters for accumulation. a[idx] += 1 reads, adds, and writes as separate buffered steps, so repeated indices overwrite each other. np.add.at (or np.bincount) does the accumulation properly.

# per-base coverage from a list of read start positions
coverage = np.zeros(chrom_len, dtype=np.int32)
np.add.at(coverage, read_starts, 1)          # correct
# or, faster:
coverage = np.bincount(read_starts, minlength=chrom_len)

take, put and choose

np.take(a, idx)                  # ~ a[idx], but with axis= and mode= options
np.take(m, [0, 2], axis=0)       # explicit axis
np.put(a, idx, values)           # flat scatter
np.take_along_axis(a, idx, axis=1)   # pair argsort with the values it sorted

take_along_axis is the right partner for argsort(axis=...):

order = np.argsort(expr, axis=1)                     # per gene, rank the samples
ranked = np.take_along_axis(expr, order, axis=1)

Bioinformatics examples

# reorder samples to match a metadata table
sample_order = np.array([sample_ids.index(s) for s in metadata["sample"]])
expr_ordered = expr[:, sample_order]

# bootstrap resampling of samples (with replacement)
rng = np.random.default_rng(0)
boot_idx = rng.integers(0, n_samples, size=n_samples)
expr_boot = expr[:, boot_idx]

# one-hot decode
"".join(BASES[onehot.argmax(axis=1)])

# codon table lookup by numeric codon index
aa = CODON_TABLE[codon_indices]

# counting variants per chromosome bin
np.bincount(positions // bin_size, minlength=n_bins)

Fancy vs boolean

np.flatnonzero(mask) converts a mask to indices when you need to combine both.

Common mistakes

• Expecting m[rows, cols] to be a submatrix. It pairs. Use np.ix_.
• a[dup_idx] += 1 for accumulation. Use np.add.at or np.bincount.
• Forgetting fancy indexing copies. On a large matrix, expr[big_idx_array] allocates a whole new matrix: watch memory.
• Applying a sort order to the data but not the labels. Same failure mode as in Boolean Masking.
• Using a float array as an index. IndexError. Cast with .astype(int).
• Full argsort when you only need the top k. Use argpartition.

See also

Indexing and Slicing · Boolean Masking · Views vs Copies · Broadcasting · Sorting and Ranking · NumPy Random Generator

Exercise

Write top_per_row(matrix, k), which selects the k largest entries of every row. Return ["bad k"] when k is below one or above the number of columns.

Otherwise emit one line per row, row<i> idx=<list> vals=<list>, giving the column indices of that row's k largest values ordered from largest to smallest, ties broken by the smaller index, and the values themselves rounded to two decimals. Gather the values with np.take_along_axis rather than a Python loop over rows. Follow those with ends=<list>, the row sums of the first and last rows selected in one step by an index array containing a negative index, rounded to two decimals. Close with gather_shares=<bool> slice_shares=<bool>, reporting whether the gathered block and the basic slice matrix[:, :k] each share memory with the source; fancy indexing and basic slicing differ here, and the two booleans must not agree unless the array is degenerate.