Reshaping and Stacking

Core Principle: Shape manipulations (reshaping, transposing) operate as lightweight metadata updates over existing memory buffers, whereas array concatenation and stacking require new memory allocation.

Reshape

array([[[ 0,  1,  2],
        [ 3,  4,  5]],

       [[ 6,  7,  8],
        [ 9, 10, 11]]])

Reshape reinterprets the same buffer with new strides. It succeeds only when np.prod(newshape) == a.size, and it returns a view when the new strides are expressible over the existing memory: which is always true for a C-contiguous array, and often false after a transpose. See Views vs Copies.

Order matters:

array([[ 0,  3,  6,  9],
       [ 1,  4,  7, 10],
       [ 2,  5,  8, 11]])

If you are reading a matrix written by R or MATLAB, it is Fortran-ordered.

Flattening

array([ 0,  1,  2,  3,  4,  5,  6,  7,  8,  9, 10, 11])

Transpose and axis moves

a.T                          # reverse all axes
a.transpose(2, 0, 1)         # explicit permutation
np.swapaxes(a, 0, 1)
np.moveaxis(a, 0, -1)        # moves designated axis position for multidimensional tensors
np.rollaxis(a, 2)            # legacy; prefer moveaxis

These methods execute as zero-copy stride updates. When subsequent operations require memory contiguousness, enforcing layout via np.ascontiguousarray(a.T) up front prevents repeated automatic copying downstream.

For 1D arrays, a.T has no effect because rank inversion requires at least two dimensions; new axes must be introduced explicitly via a[:, None] or a[None, :].

Adding and removing axes

a[None, :]                   # insert axis at front
a[:, None]                   # insert axis at position 1
np.expand_dims(a, axis=1)    # the verbose equivalent
np.squeeze(a)                # drop ALL length-1 axes
np.squeeze(a, axis=0)        # drop a specific one: safer
a.item()                     # extract the scalar from a size-1 array

Bare np.squeeze(a) is risky in library code: if a dimension happens to be length 1 for a particular input, it disappears and your shapes shift. Always name the axis.

Combining arrays

np.concatenate([a, b], axis=0)     # join along an EXISTING axis
np.stack([a, b], axis=0)           # join along a NEW axis
np.vstack([a, b])                  # concatenate axis=0 (and promote 1-D to rows)
np.hstack([a, b])                  # concatenate axis=1 (axis=0 for 1-D)
np.column_stack([a, b])            # 1-D arrays become columns
np.dstack([a, b])                  # along a third axis
np.tile(a, (2, 3))                 # repeat the whole array
np.repeat(a, 3)                    # repeat each element

concatenate vs stack is the distinction to hold onto:

(2, 3, 4)

vstack/hstack have special 1-D promotion rules that make them convenient interactively and ambiguous in library code. Prefer explicit concatenate(..., axis=...) when it matters.

Splitting

np.split(a, 3, axis=0)             # 3 equal parts: errors if not divisible
np.array_split(a, 3, axis=0)       # allows unequal parts
np.split(a, [2, 5])                # split at given indices
np.vsplit(a, 2); np.hsplit(a, 2)

The performance rule

Concatenation copies both operands into a fresh buffer. Doing it in a loop is quadratic:

# BAD
result = np.array([])
for chunk in chunks:
    result = np.concatenate([result, chunk])

# GOOD: one allocation
result = np.concatenate(chunks)

Same lesson as Array Creation: collect in a Python list, convert once.

Bioinformatics examples

# reshape a flat sequence into codons
codons = seq_codes[:len(seq_codes) // 3 * 3].reshape(-1, 3)

# a stack of per-sample coverage vectors → a matrix
coverage_matrix = np.stack(per_sample_vectors, axis=0)     # (n_samples, n_positions)

# combine gene panels from several files
all_expr = np.concatenate(panel_matrices, axis=0)          # more genes, same samples
all_samples = np.concatenate(batch_matrices, axis=1)       # same genes, more samples

# transpose to match a tool's expected orientation
# scanpy/AnnData want observations (cells) × variables (genes);
# bulk RNA-seq tools want genes × samples. This transpose is a constant source of bugs.
adata_style = expr.T

# tile a background profile across samples
bg_matrix = np.tile(background[:, None], (1, n_samples))
# ...though broadcasting usually makes the tile unnecessary: see [[Broadcasting]]

That last point generalises: if you find yourself using np.tile or np.repeat to make shapes match, broadcasting will probably do it for free.

Common mistakes

• a.T on a 1-D array doing nothing, silently.
• reshape after transpose copying when you assumed a view.
• vstack vs hstack on 1-D input: the promotion rules surprise people. np.vstack([[1,2],[3,4]]) gives (2,2); np.hstack gives (4,).
• concatenate in a loop.
• Bare np.squeeze removing an axis you needed.
• Reshaping genes × samples to samples × genes with reshape instead of .T. Reshape reinterprets the buffer in order; it does not transpose. The result is scrambled data, not an error.

That last one deserves emphasis: expr.reshape(n_samples, n_genes) and expr.T have the same output shape and completely different contents.

See also

ndarray · Memory Layout and Strides · Views vs Copies · Broadcasting · Array Creation · Concat

Exercise

Write fold(flat, ncols, order), which folds a flat array of non-negative integers into ncols columns, padding the tail with -1 so the length divides evenly. Return ["bad spec"] when ncols is below one or order is neither "C" nor "F".

Otherwise return five lines: pad=<n> shape=<r>x<c>, the number of padding elements added and the resulting shape; rows=<nested list>, the folded matrix including its padding; col_sums=<list>, the column totals with the padding counted as zero rather than as -1; ends=<nested list>, the first and last column stacked side by side with np.column_stack; and no_pad_view=<bool>, true only when no padding was needed and reshaping the original array to (-1, ncols) returns a view rather than a copy. The order argument decides whether the array is read across the rows or down the columns, and the two give different matrices for the same input.