pythonforbio.
[WASM idle]
NumPy13/18

Reshaping and Stacking

Starting sandbox…
Beginnerlesson2 tests

Reshaping and Stacking

In one line: changing shape without changing data (reshape, transpose) is nearly free; combining arrays (concatenate, stack) always allocates.

Reshape

a = np.arange(12)

a.reshape(3, 4)          # (3, 4)
a.reshape(3, -1)         # -1 = "infer this dimension"
a.reshape(-1)            # flatten to 1-D
a.reshape(2, 2, 3)       # 3-D

Reshape reinterprets the same buffer with new strides. It succeeds only when np.prod(newshape) == a.size, and it returns a view when the new strides are expressible over the existing memory — which is always true for a C-contiguous array, and often false after a transpose. See Views vs Copies.

Order matters:

a.reshape(3, 4, order="C")     # row-major: last axis varies fastest (default)
a.reshape(3, 4, order="F")     # column-major: first axis varies fastest

If you are reading a matrix written by R or MATLAB, it is Fortran-ordered.

Flattening

a.ravel()        # view when possible — fast
a.flatten()      # always a copy — safe
a.reshape(-1)    # same as ravel

Transpose and axis moves

a.T                          # reverse all axes
a.transpose(2, 0, 1)         # explicit permutation
np.swapaxes(a, 0, 1)
np.moveaxis(a, 0, -1)        # move axis 0 to the end — clearest for >2-D
np.rollaxis(a, 2)            # legacy; prefer moveaxis

All of these are stride manipulations — instant, zero-copy, regardless of array size. The cost appears later, when a subsequent operation needs contiguous memory and silently copies. If you transpose then do heavy work, np.ascontiguousarray(a.T) up front can be faster overall.

For 1-D arrays a.T does nothing — there is only one axis to reverse. To turn a (n,) into a (1,n) or (n,1) use a[None, :] / a[:, None].

Adding and removing axes

a[None, :]                   # insert axis at front
a[:, None]                   # insert axis at position 1
np.expand_dims(a, axis=1)    # the verbose equivalent
np.squeeze(a)                # drop ALL length-1 axes
np.squeeze(a, axis=0)        # drop a specific one — safer
a.item()                     # extract the scalar from a size-1 array

Bare np.squeeze(a) is risky in library code: if a dimension happens to be length 1 for a particular input, it disappears and your shapes shift. Always name the axis.

Combining arrays

np.concatenate([a, b], axis=0)     # join along an EXISTING axis
np.stack([a, b], axis=0)           # join along a NEW axis
np.vstack([a, b])                  # concatenate axis=0 (and promote 1-D to rows)
np.hstack([a, b])                  # concatenate axis=1 (axis=0 for 1-D)
np.column_stack([a, b])            # 1-D arrays become columns
np.dstack([a, b])                  # along a third axis
np.tile(a, (2, 3))                 # repeat the whole array
np.repeat(a, 3)                    # repeat each element

concatenate vs stack is the distinction to hold onto:

a = np.zeros((3, 4)); b = np.zeros((3, 4))
np.concatenate([a, b], axis=0).shape   # (6, 4) — same rank
np.stack([a, b], axis=0).shape          # (2, 3, 4) — rank increased

vstack/hstack have special 1-D promotion rules that make them convenient interactively and ambiguous in library code. Prefer explicit concatenate(..., axis=...) when it matters.

Splitting

np.split(a, 3, axis=0)             # 3 equal parts — errors if not divisible
np.array_split(a, 3, axis=0)       # allows unequal parts
np.split(a, [2, 5])                # split at given indices
np.vsplit(a, 2); np.hsplit(a, 2)

The performance rule

Concatenation copies both operands into a fresh buffer. Doing it in a loop is quadratic:

# BAD
result = np.array([])
for chunk in chunks:
    result = np.concatenate([result, chunk])

# GOOD — one allocation
result = np.concatenate(chunks)

Same lesson as Array Creation: collect in a Python list, convert once.

Bioinformatics examples

# reshape a flat sequence into codons
codons = seq_codes[:len(seq_codes) // 3 * 3].reshape(-1, 3)

# a stack of per-sample coverage vectors → a matrix
coverage_matrix = np.stack(per_sample_vectors, axis=0)     # (n_samples, n_positions)

# combine gene panels from several files
all_expr = np.concatenate(panel_matrices, axis=0)          # more genes, same samples
all_samples = np.concatenate(batch_matrices, axis=1)       # same genes, more samples

# transpose to match a tool's expected orientation
# scanpy/AnnData want observations (cells) × variables (genes);
# bulk RNA-seq tools want genes × samples. This transpose is a constant source of bugs.
adata_style = expr.T

# tile a background profile across samples
bg_matrix = np.tile(background[:, None], (1, n_samples))
# ...though broadcasting usually makes the tile unnecessary — see [[Broadcasting]]

That last point generalises: if you find yourself using np.tile or np.repeat to make shapes match, broadcasting will probably do it for free.

Common mistakes

  • a.T on a 1-D array doing nothing, silently.
  • reshape after transpose copying when you assumed a view.
  • vstack vs hstack on 1-D input — the promotion rules surprise people. np.vstack([[1,2],[3,4]]) gives (2,2); np.hstack gives (4,).
  • concatenate in a loop.
  • Bare np.squeeze removing an axis you needed.
  • Reshaping genes × samples to samples × genes with reshape instead of .T. Reshape reinterprets the buffer in order; it does not transpose. The result is scrambled data, not an error.

That last one deserves emphasis: expr.reshape(n_samples, n_genes) and expr.T have the same output shape and completely different contents.

See also

ndarray · Memory Layout and Strides · Views vs Copies · Broadcasting · Array Creation · Concat

Test cases · 2

#viainputexpected stdout
1stdin1 2 3 4 5 6[[1, 2, 3], [4, 5, 6]]
2stdin0 0 1 1 1 2 2 2 3[[0, 0, 1], [1, 1, 2], [2, 2, 3]]

Hints · 2

01Hint

reshape(-1, 3) lets NumPy infer the row count.

02Hint

np.vstack stacks row-wise; np.hstack stacks column-wise.

starter

No output yet — run the code to populate this drawer.

NumPy dtypesStructured Arrays