Pandas13/21
Starting sandbox…
Beginner·4 graded tests

Series

Series

Core Principle: A Series encapsulates a one-dimensional array paired with an explicit Index, ensuring that mathematical operations and aggregations align automatically by label rather than position.

Construction

[1]python
out1ms
'counts'

.to_numpy() over .values: .values returns whatever the backing store is, which for Arrow-backed or nullable dtypes is not a plain ndarray.

Access: labels and positions

[2]python
out1ms
0

Always use .loc and .iloc explicitly. Bare s[0] is ambiguous: is 0 a label or a position? With an integer index it means label; otherwise position. That ambiguity is a genuine bug source. See loc vs iloc.

Alignment: the thing that surprises everyone

[3]python
out1ms
x    31
y    22
z    13
dtype: int64

pandas aligned on the index. NumPy would have added positionally and given you [11, 22, 33].

The failure mode:

[4]python
out1ms
x   NaN
y   NaN
z   NaN
0   NaN
1   NaN
2   NaN
dtype: float64

"Why is my result all NaN?" is almost always index misalignment. The fix is to align deliberately (b.index = a.index) or drop out of label-land (a + b.to_numpy()).

Vectorized operations

Everything from ufuncs works, and the index is carried through:

[5]python
out1ms
TP53     120
BRCA1    340
EGFR      90
Name: counts, dtype: int64

The method catalogue

s.sum(); s.mean(); s.median(); s.std()      # ddof=1 by default, unlike NumPy!
s.min(); s.max(); s.idxmax(); s.idxmin()    # idxmax gives the LABEL
s.quantile([0.25, 0.5, 0.75])
s.describe()

s.value_counts()                 # frequency table, sorted descending
s.value_counts(dropna=False)     # ← always use this; NaN counts matter
s.value_counts(normalize=True)   # proportions
s.unique(); s.nunique()
s.isna(); s.notna(); s.fillna(0); s.dropna()
s.sort_values(); s.sort_index()
s.rank(method="dense")
s.astype("float32")
s.map({"A": "Adenine"})          # elementwise via dict or function
s.replace({-9: np.nan})
s.between(0.01, 0.99)
s.isin(PANEL_GENES)
s.head(); s.tail(); s.sample(5)

s.idxmax() returning the label rather than the position is the useful difference from np.argmax: "which gene is highest" answers directly.

s.std() uses ddof=1 (sample) while np.std uses ddof=0 (population). Same data, different answer. See Axes and Reductions.

Accessors

Namespaced method groups for particular dtypes:

s.str.upper(); s.str.contains("BRCA")     # string ops: see [[String Accessor]]
s.dt.year; s.dt.dayofweek                  # datetime: see [[Datetime Handling in pandas]]
s.cat.categories; s.cat.codes              # categorical: see [[Categorical dtype]]

Calling .str on a non-string Series raises, which is a useful type check.

Series vs NumPy array: when to use which

Series ndarray
Labels yes no
Missing data first-class NaN only, floats only
Mixed types via object/Arrow no
Alignment automatic positional
Speed small overhead per op baseline
n-dimensional no yes

For a tight numeric loop over millions of elements, drop to .to_numpy(). For anything where losing the labels would be a correctness risk: which in bioinformatics is most things: stay in pandas.

Bioinformatics examples

counts = pd.Series(raw, index=gene_ids, name="counts")

cpm = counts / counts.sum() * 1e6
log_cpm = np.log2(cpm + 1)

expressed = counts[counts > 10]
counts.nlargest(20)                        # top 20 genes, labels intact
counts.rank(ascending=False).loc["TP53"]   # where does TP53 rank?

df["consequence"].value_counts(dropna=False)
df["chrom"].isin([f"chr{i}" for i in range(1, 23)])   # autosomes only

The point of counts.nlargest(20) over np.argsort(raw)[-20:] is that the gene names come along automatically. That is the whole value proposition of pandas in one line.

Common mistakes

  • Index misalignment producing all-NaN. The single most common pandas confusion.
  • Bare s[0] on an integer-indexed Series meaning label, not position.
  • Assigning a Series to a DataFrame column and getting NaN: it aligned on index. Use .to_numpy() or .reset_index(drop=True) if you meant positional.
  • s.std() vs np.std(s) disagreeing because of ddof.
  • value_counts() hiding NaN by default. Pass dropna=False.
  • .values when you wanted .to_numpy().
  • Mutating a Series extracted from a DataFrame and expecting the frame to change. Under Copy-on-Write it will not.

See also

DataFrame · Index Objects · loc vs iloc · String Accessor · Missing Data in pandas · pandas dtypes · ndarray

Exercise

Write align_report(a, b), which takes two Series built for you by read_series. Stdin holds the first series as label value lines, then a line containing only --, then the second series in the same form. Labels are unique within each series but the two series need not share labels or order. Return a single string.

Add the two Series and let pandas align them on their labels: do not convert to arrays and do not reindex one onto the other. Return aligned=<n> missing=<n> sum=<total to 3 dp> top=<label>, where aligned is the number of labels present in both inputs, missing is the number of entries in the sum that came out as NaN because a label was in only one input, sum is the total of the non-NaN entries rounded to three decimal places, and top is the label of the largest non-NaN entry. Ties on the largest value are broken by taking the alphabetically first label. When the two series share no label at all, sum is 0.0 and top is the literal NONE.

exerciseEvery submission runs independently in a fresh namespace.

Test cases · 4

#viainputexpected stdout
1stdinx 1 y 2 z 3 -- z 10 y 20 x 30 aligned=3 missing=0 sum=66.0 top=x
2stdinTP53 5 BRCA1 7 EGFR 9 -- EGFR 1 KRAS 4 aligned=1 missing=3 sum=10.0 top=EGFR
3stdina 1 b 2 -- c 3 d 4 aligned=0 missing=4 sum=0.0 top=NONE
4stdinp 10 q 1 r 5 -- q 19 r 15 p 10 aligned=3 missing=0 sum=60.0 top=p

Hints · 3

01Hint

a + b produces one entry per label in the union of the two indexes, with NaN wherever a label was missing from one side.

02Hint

a.index.intersection(b.index) counts the labels that actually matched.

03Hint

idxmax returns the label of the first maximum, so sort the index before calling it to make the tiebreak alphabetical.