pythonforbio.
[WASM idle]
pandas13/21

Series

Starting sandbox…
Beginnerlesson

Series

In one line: a 1-D array plus an index of labels — and the labels participate in every operation, which is the whole difference from a NumPy array.

Construction

s = pd.Series([120, 340, 90], index=["TP53", "BRCA1", "EGFR"], name="counts")

s.values        # underlying array (NumPy or Arrow)
s.to_numpy()    # preferred — always gives you a NumPy array
s.index         # Index(['TP53', 'BRCA1', 'EGFR'], dtype='str')
s.dtype         # int64
s.name          # 'counts' — becomes the column name in a DataFrame

.to_numpy() over .values: .values returns whatever the backing store is, which for Arrow-backed or nullable dtypes is not a plain ndarray.

Access: labels and positions

s["TP53"]           # by label
s[["TP53", "EGFR"]] # multiple labels
s.loc["TP53"]       # explicit label — use this
s.iloc[0]           # explicit position — use this
s.get("MISSING", 0) # label lookup with a default

Always use .loc and .iloc explicitly. Bare s[0] is ambiguous: is 0 a label or a position? With an integer index it means label; otherwise position. That ambiguity is a genuine bug source. See loc vs iloc.

Alignment — the thing that surprises everyone

a = pd.Series([1, 2, 3], index=["x", "y", "z"])
b = pd.Series([10, 20, 30], index=["z", "y", "x"])   # reversed order!

a + b
# x    31     ← matched by LABEL, not position
# y    22
# z    13

pandas aligned on the index. NumPy would have added positionally and given you [11, 22, 33].

The failure mode:

a = pd.Series([1, 2, 3], index=["x", "y", "z"])
b = pd.Series([10, 20, 30], index=[0, 1, 2])
a + b
# 0    NaN
# 1    NaN
# ...  all NaN, six rows, no error

"Why is my result all NaN?" is almost always index misalignment. The fix is to align deliberately (b.index = a.index) or drop out of label-land (a + b.to_numpy()).

Vectorized operations

Everything from ufuncs works, and the index is carried through:

np.log2(s + 1)
s * 2
s > 100
s[s > 100]            # boolean mask, index preserved
s.clip(0, 1000)
s.round(2)

The method catalogue

s.sum(); s.mean(); s.median(); s.std()      # ddof=1 by default, unlike NumPy!
s.min(); s.max(); s.idxmax(); s.idxmin()    # idxmax gives the LABEL
s.quantile([0.25, 0.5, 0.75])
s.describe()

s.value_counts()                 # frequency table, sorted descending
s.value_counts(dropna=False)     # ← always use this; NaN counts matter
s.value_counts(normalize=True)   # proportions
s.unique(); s.nunique()
s.isna(); s.notna(); s.fillna(0); s.dropna()
s.sort_values(); s.sort_index()
s.rank(method="dense")
s.astype("float32")
s.map({"A": "Adenine"})          # elementwise via dict or function
s.replace({-9: np.nan})
s.between(0.01, 0.99)
s.isin(PANEL_GENES)
s.head(); s.tail(); s.sample(5)

s.idxmax() returning the label rather than the position is the useful difference from np.argmax — "which gene is highest" answers directly.

s.std() uses ddof=1 (sample) while np.std uses ddof=0 (population). Same data, different answer. See Axes and Reductions.

Accessors

Namespaced method groups for particular dtypes:

s.str.upper(); s.str.contains("BRCA")     # string ops — see [[String Accessor]]
s.dt.year; s.dt.dayofweek                  # datetime — see [[Datetime Handling in pandas]]
s.cat.categories; s.cat.codes              # categorical — see [[Categorical dtype]]

Calling .str on a non-string Series raises, which is a useful type check.

Series vs NumPy array — when to use which

Series ndarray
Labels yes no
Missing data first-class NaN only, floats only
Mixed types via object/Arrow no
Alignment automatic positional
Speed small overhead per op baseline
n-dimensional no yes

For a tight numeric loop over millions of elements, drop to .to_numpy(). For anything where losing the labels would be a correctness risk — which in bioinformatics is most things — stay in pandas.

Bioinformatics examples

counts = pd.Series(raw, index=gene_ids, name="counts")

cpm = counts / counts.sum() * 1e6
log_cpm = np.log2(cpm + 1)

expressed = counts[counts > 10]
counts.nlargest(20)                        # top 20 genes, labels intact
counts.rank(ascending=False).loc["TP53"]   # where does TP53 rank?

df["consequence"].value_counts(dropna=False)
df["chrom"].isin([f"chr{i}" for i in range(1, 23)])   # autosomes only

The point of counts.nlargest(20) over np.argsort(raw)[-20:] is that the gene names come along automatically. That is the whole value proposition of pandas in one line.

Common mistakes

  • Index misalignment producing all-NaN. The single most common pandas confusion.
  • Bare s[0] on an integer-indexed Series meaning label, not position.
  • Assigning a Series to a DataFrame column and getting NaN — it aligned on index. Use .to_numpy() or .reset_index(drop=True) if you meant positional.
  • s.std() vs np.std(s) disagreeing because of ddof.
  • value_counts() hiding NaN by default. Pass dropna=False.
  • .values when you wanted .to_numpy().
  • Mutating a Series extracted from a DataFrame and expecting the frame to change. Under Copy-on-Write it will not.

See also

DataFrame · Index Objects · loc vs iloc · String Accessor · Missing Data in pandas · pandas dtypes · ndarray

lesson example

No output yet — run the code to populate this drawer.