Boolean Masking

Core Principle: Boolean masking generates a boolean ndarray matching data dimensions to select, count, or conditionally update elements via vectorized indexation without interpreter-level loops.

The pattern

np.float64(0.4)

mask.sum() and mask.mean() are the idiomatic "how many" and "what fraction": no need for np.count_nonzero although that exists too and is marginally faster.

Combining conditions

Use &, |, ~: never and, or, not.

array([ True,  True,  True,  True,  True])

The parentheses are mandatory. & binds tighter than > in Python, so counts > 10 & counts < 100 parses as counts > (10 & counts) < 100 and raises a confusing error.

The Python keywords fail because and/or call bool() on their operands, and calling bool() on a multi-element array raises:

ValueError: The truth value of an array with more than one element is ambiguous.

That error message, when you see it, almost always means you wrote and where you needed &.

Any, all, and where

mask.any()               # is anything True?
mask.all()               # is everything True?
mask.any(axis=1)         # per row: see [[Axes and Reductions]]

np.where(mask)           # tuple of index arrays where True
np.where(mask, a, b)     # vectorised ternary: a where True, b where False
np.flatnonzero(mask)     # flat indices of True: cleaner than np.where(mask)[0]
np.argwhere(mask)        # (n, ndim) array of coordinate tuples

np.where(cond, x, y) is the vectorised if/else and it is worth memorising:

direction = np.where(log2fc > 0, "up", "down")
capped = np.where(counts > 1000, 1000, counts)

Both branches are evaluated eagerly, so np.where(x != 0, 1/x, 0) still emits a divide-by-zero warning. Use np.divide(1, x, out=np.zeros_like(x), where=x != 0) instead.

Masks on multi-dimensional arrays

A boolean mask of the same shape flattens the result:

expr[expr > 10]              # 1-D array of all values above 10: loses shape

To keep the matrix shape, select along one axis with a 1-D mask:

expressed = (expr > 1).sum(axis=1) >= 3      # gene passes in ≥3 samples
expr[expressed]                               # (n_passing, n_samples): shape kept
expr[:, sample_mask]                          # subset columns
expr[np.ix_(expressed, sample_mask)]          # both at once

This gene-filtering idiom: "expressed above threshold in at least N samples": is the standard first step of any RNA-seq analysis.

Boolean masking returns a copy

b = a[a > 5]
b[0] = 0        # does NOT affect a

Unlike slicing, mask selection cannot be expressed as a stride pattern, so NumPy must gather the elements into new memory. See Views vs Copies.

But assignment through a mask does write to the original:

a[a > 5] = 0    # this DOES modify a

a[mask] calls __getitem__ (gather → copy); a[mask] = x calls __setitem__ (scatter → in-place).

Bioinformatics examples

# gene filtering
keep = (counts.sum(axis=1) > 10) & ((counts > 0).sum(axis=1) >= 3)
filtered = counts[keep]
kept_gene_names = gene_names[keep]        # ← apply the SAME mask to the labels

# quality filtering on reads
good = (quals.mean(axis=1) >= 30) & (~np.isnan(quals).any(axis=1))

# significant hits
sig = (padj < 0.05) & (np.abs(log2fc) > 1)
print(f"{sig.sum()} significant of {sig.size} ({sig.mean():.1%})")

# masking NaNs before a mean
valid = ~np.isnan(x)
x[valid].mean()          # or just np.nanmean(x)

The critical habit is in line 3: whenever you filter a data matrix, apply the identical mask to the label array in the same breath. Losing the correspondence between rows and gene names is the most common: and most invisible: bug in matrix-based bioinformatics. This is precisely the problem Pandas solves with an index.

Common mistakes

• and / or instead of & / | → "truth value is ambiguous".
• Missing parentheses around comparisons combined with &.
• arr == np.nan is always False. Use np.isnan(arr). See Missing Data in NumPy.
• Filtering the matrix but not the labels. Silent, permanent misattribution of results.
• Expecting expr[expr > 10] to keep the matrix shape. It returns 1-D.
• Using a mask of the wrong length. NumPy raises for boolean masks of mismatched length: but a mask that is 1 short of the axis length used to be tolerated in old versions. Be exact.
• np.where(cond, f(x), g(x)) where f or g errors on the other branch. Both are computed.

See also

Indexing and Slicing · Fancy Indexing · Views vs Copies · Missing Data in NumPy · Axes and Reductions · Filtering and query · Applied - Expression Matrices with NumPy

Exercise

Write callable_regions(depths, min_depth, min_len), which finds the maximal runs of consecutive positions whose depth is at least min_depth and keeps only those at least min_len positions long. Return ["none"] when no run qualifies, including when depths is empty.

Otherwise the first line is runs=<n>, followed by one line per run in left-to-right order formatted <start>-<end> len=<n> mean=<value>, where start is inclusive, end is exclusive, and the mean depth over the run is rounded to two decimals. Build the mask once and locate the run boundaries from where it changes; a run that reaches the last position of the array must still be closed, which is the case a naive scan for a falling edge misses.