Ecosystem Map
Ecosystem Map
How the five libraries relate, and what sits above and below them.
The layer diagram
flowchart TD
subgraph Domain ["Domain Layer"]
HGVS["biocommons hgvs"]
Bio["Biopython"]
end
subgraph Presentation ["Presentation Layer"]
SNS["seaborn"] --> MPL["Matplotlib"]
end
subgraph Tabular ["Tabular Data Layer"]
PD["pandas"]
end
subgraph Numeric ["Numeric Array Engine"]
NP["NumPy"]
end
Bio -.-> NP
SNS --> PD
PD --> NP
MPL -.-> NP
The arrows are real dependencies, not metaphor. Pandas stores its columns as NumPy arrays (or PyArrow arrays since 3.0), Seaborn is a layer that emits Matplotlib Artists, and Biopython returns NumPy arrays from several of its modules (Bio.Align, Bio.PDB coordinates, Bio.motifs PWMs).
biocommons hgvs is the odd one out: it depends on none of the others. It is a parsing-and-coordinate-mathematics library with a database behind it, and you glue it to pandas yourself.
What each library is actually for
| Library | The one-sentence job | The thing it is not |
|---|---|---|
| NumPy | Homogeneous n-dimensional numeric arrays with fast elementwise math | Not for labelled or mixed-type data |
| Pandas | Heterogeneous tables with labelled rows and columns | Not for >memory data, not for n-d arrays |
| Matplotlib | Precise, low-level control over every mark on a figure | Not fast to write for common statistical plots |
| Seaborn | Statistical plots from tidy dataframes in one line | Not for arbitrary custom layouts |
| Biopython | Parsing bio file formats and manipulating sequences/structures | Not an aligner, not a variant caller; it wraps or parses those |
| biocommons hgvs | Correct parsing, normalization and projection of sequence variants | Not a variant annotator; it does not tell you pathogenicity |
Where the boundaries actually get blurry
pandas vs NumPy. If your data is a homogeneous matrix with meaningful row/column positions (an expression matrix, a distance matrix), NumPy is the right tool and pandas adds overhead. If you need labels, mixed dtypes, or joins, use pandas. Many pipelines carry a pandas frame for metadata alongside a NumPy array for the numbers: that is exactly what AnnData/scanpy formalises.
Matplotlib vs seaborn. Not a choice. seaborn is matplotlib. You use seaborn to get 90% of a plot in one call, then reach through to the underlying Axes for the last 10%. See Figure-level vs Axes-level.
Biopython vs hgvs. Biopython handles sequence content; hgvs handles variant nomenclature and coordinates. If you need the reference base at a position, that is SeqRepo (through hgvs) or a FASTA index (through Biopython/pysam). If you need to know whether NM_000546.6:c.215C>G and NC_000017.11:g.7676154G>C are the same event, that is hgvs and only hgvs.
The neighbours you will meet next
Worth knowing these exist, even though this vault does not cover them:
- pysam: the actual workhorse for BAM/CRAM/VCF/tabix. Biopython does not read BAM. For anything alignment-file-shaped, reach for pysam.
- scikit-bio, pyranges: interval arithmetic and ecology/diversity stats
- scanpy / AnnData: single-cell, and the canonical example of "pandas for metadata + NumPy/sparse for the matrix"
- scipy: statistics (
scipy.stats), clustering, sparse matrices. Every real analysis reaches for it. - statsmodels: regression and multiple-testing correction (
multipletestsfor FDR) - polars / DuckDB: where you go when pandas runs out of memory
- cyvcf2: fast VCF parsing when pandas-based parsing is too slow
See also
Versions and Compatibility · Learning Path · Tidy Data · Map of Content
Exercise
Write route(tasks), where tasks is a list of free-text task descriptions,
one per input line. Build a keyword table mapping a lowercase keyword to the
library that owns that job, and a second table mapping each library to its
layer in the map. Lowercase each task and split it on every run of
non-alphabetic characters, so that punctuation and digits do not hide a
keyword, then collect the distinct libraries the tokens name. Skip a task that
is empty after stripping.
Use these keywords: array, matrix, broadcast, dtype for numpy
(numeric); dataframe, merge, groupby, melt for pandas (tabular);
axes, figure, savefig for matplotlib and facet, boxplot,
clustermap for seaborn (both presentation); fasta, fastq, genbank,
translate for biopython and hgvs, normalize, project, transcript
for hgvs (both domain); bam, cram, tabix for pysam (neighbour).
Emit <task> -> <library>:<layer> when exactly one library matches,
<task> -> ambiguous:<libraries> with the names alphabetical and comma
separated when more than one does, and <task> -> unknown when none does. The
task is echoed stripped of surrounding whitespace. Finish with one line
routed=<n> ambiguous=<n> unknown=<n>.
Test cases · 4
| # | via | input | expected stdout |
|---|---|---|---|
| 1 | stdin | read a FASTQ file merge two annotation tables index a BAM file | read a FASTQ file -> biopython:domain merge two annotation tables -> pandas:tabular index a BAM file -> pysam:neighbour routed=3 ambiguous=0 unknown=0 |
| 2 | stdin | melt a dataframe then draw a boxplot normalize a transcript variant savefig the figure | melt a dataframe then draw a boxplot -> ambiguous:pandas,seaborn normalize a transcript variant -> hgvs:domain savefig the figure -> matplotlib:presentation routed=2 ambiguous=1 unknown=0 |
| 3 | stdin | count the reads compute a distance matrix | count the reads -> unknown compute a distance matrix -> numpy:numeric routed=1 ambiguous=0 unknown=1 |
| 4 | stdin | translate the CDS and write FASTA | translate the CDS and write FASTA -> biopython:domain routed=1 ambiguous=0 unknown=0 |
Hints · 3
01Hint
`re.split(r'[^a-z]+', text.lower())` turns "c. to g. projection" into tokens without punctuation attached.
02Hint
Collect the matches in a set — a task naming `array` twice must still count as one library, while `array` plus `melt` is ambiguous.
03Hint
Two keywords that resolve to the same library are not ambiguous, so decide on the number of distinct libraries, not the number of matched keywords.