Bio.Entrez

In one line: programmatic access to NCBI's databases — set your email, respect the rate limits, and cache everything.

Setup — not optional

from Bio import Entrez

Entrez.email = "you@institution.edu"      # REQUIRED by NCBI's terms
Entrez.tool = "my_analysis_pipeline"       # identifies your software
Entrez.api_key = "your_key_here"           # optional; raises the rate limit

NCBI requires an email so they can contact you before blocking your IP if a script misbehaves. Without it Biopython warns and NCBI may throttle you.

Get an API key. It is free from your NCBI account and raises your limit from 3 to 10 requests per second. Store it in an environment variable, not in the script:

import os
Entrez.api_key = os.environ.get("NCBI_API_KEY")

The main calls

Entrez.esearch(db, term, retmax)    # search → list of IDs
Entrez.efetch(db, id, rettype, retmode)   # retrieve records
Entrez.esummary(db, id)             # brief summaries
Entrez.elink(dbfrom, db, id)        # cross-database links
Entrez.einfo(db)                    # what fields does this database have?
Entrez.epost(db, id)                # upload IDs to the history server
Entrez.read(handle)                 # parse XML → nested dicts/lists

Always close the handle, or use a with block:

with Entrez.efetch(db="nucleotide", id="NM_000546.6",
                   rettype="gb", retmode="text") as h:
    rec = SeqIO.read(h, "genbank")

Common patterns

# search then fetch
with Entrez.esearch(db="nucleotide", term="TP53[Gene] AND human[Organism]",
                    retmax=20) as h:
    ids = Entrez.read(h)["IdList"]

with Entrez.efetch(db="nucleotide", id=",".join(ids),
                   rettype="gb", retmode="text") as h:
    records = list(SeqIO.parse(h, "genbank"))

# one accession as GenBank
with Entrez.efetch(db="nucleotide", id="NM_000546.6",
                   rettype="gb", retmode="text") as h:
    rec = SeqIO.read(h, "genbank")

# protein FASTA
with Entrez.efetch(db="protein", id="NP_000537.3",
                   rettype="fasta", retmode="text") as h:
    prot = SeqIO.read(h, "fasta")

# PubMed abstracts
with Entrez.efetch(db="pubmed", id="12345678", rettype="abstract",
                   retmode="text") as h:
    print(h.read())

# gene record as structured XML
with Entrez.efetch(db="gene", id="7157", retmode="xml") as h:
    gene = Entrez.read(h)

# ClinVar for a gene
with Entrez.esearch(db="clinvar", term="TP53[gene] AND pathogenic",
                    retmax=100) as h:
    ids = Entrez.read(h)["IdList"]

db="gene" with id="7157" uses the Entrez Gene ID, not the symbol. Use esearch on db="gene" to go from symbol to ID.

Batching

Never fetch one ID at a time in a loop.

def fetch_batched(ids, db="nucleotide", rettype="fasta", batch=200):
    for i in range(0, len(ids), batch):
        chunk = ids[i:i + batch]
        with Entrez.efetch(db=db, id=",".join(chunk),
                           rettype=rettype, retmode="text") as h:
            yield from SeqIO.parse(h, rettype)
        time.sleep(0.15)

For very large sets, use the history server so you never send the ID list back:

with Entrez.esearch(db="nucleotide", term=query, usehistory="y") as h:
    r = Entrez.read(h)
count, webenv, key = int(r["Count"]), r["WebEnv"], r["QueryKey"]

for start in range(0, count, 500):
    with Entrez.efetch(db="nucleotide", rettype="fasta", retmode="text",
                       retstart=start, retmax=500,
                       webenv=webenv, query_key=key) as h:
        yield from SeqIO.parse(h, "fasta")

Be a good citizen

• ≤3 requests/second without an API key, ≤10 with one. Biopython does some throttling; do not rely on it entirely.
• Bulk downloads outside US peak hours (weekends, or 21:00–06:00 Eastern).
• For anything genuinely large, use the FTP site (ftp.ncbi.nlm.nih.gov) or the datasets CLI. E-utilities is an API for queries, not a bulk transfer mechanism.
• Retry with backoff — the service returns transient 500s under load.

import time
from urllib.error import HTTPError

def robust_fetch(fn, *args, retries=4, **kw):
    for attempt in range(retries):
        try:
            return fn(*args, **kw)
        except HTTPError as e:
            if attempt == retries - 1 or e.code < 500:
                raise
            time.sleep(2 ** attempt)

Cache aggressively

Network calls are slow, rate-limited, and not reproducible — the record you fetch today may differ from the one you fetch next year, because sequences get revised.

from pathlib import Path

def cached_genbank(acc, cache_dir="ncbi_cache"):
    cache = Path(cache_dir); cache.mkdir(exist_ok=True)
    path = cache / f"{acc}.gb"
    if not path.exists():
        with Entrez.efetch(db="nucleotide", id=acc,
                           rettype="gb", retmode="text") as h:
            path.write_text(h.read())
    return SeqIO.read(path, "genbank")

Cache the accession with its version (NM_000546.6, not NM_000546). An unversioned accession returns whatever is current, so your pipeline's behaviour changes silently when NCBI updates the record. This is the same reproducibility principle as pinning your UTA snapshot — see UTA and Data Providers.

Security note

Entrez.read parses XML. Biopython 1.87 fixed CVE-2025-68463 in Bio.Entrez.Parser. Keep Biopython up to date, especially if you parse XML from anywhere other than NCBI itself.

Bioinformatics examples

# all RefSeq transcripts for a gene
with Entrez.esearch(db="nucleotide",
                    term="TP53[Gene] AND human[Organism] AND refseq[Filter] AND mRNA[Filter]",
                    retmax=50) as h:
    ids = Entrez.read(h)["IdList"]

# gene → protein via elink
with Entrez.elink(dbfrom="gene", db="protein", id="7157") as h:
    links = Entrez.read(h)
protein_ids = [l["Id"] for l in links[0]["LinkSetDb"][0]["Link"]]

# taxonomy lineage
with Entrez.efetch(db="taxonomy", id="9606", retmode="xml") as h:
    tax = Entrez.read(h)
print(tax[0]["Lineage"])

# assembly summary
with Entrez.esummary(db="assembly", id="GCF_000001405.40") as h:
    print(Entrez.read(h))

# build a local reference set for a gene panel
for gene in PANEL:
    with Entrez.esearch(db="nucleotide",
                        term=f"{gene}[Gene] AND human[Organism] AND refseq_select[Filter]") as h:
        ids = Entrez.read(h)["IdList"]
    if ids:
        rec = cached_genbank(ids[0])
        SeqIO.write(rec, f"refs/{gene}.gb", "genbank")
    time.sleep(0.15)

refseq_select[Filter] restricts to the single representative transcript per gene, which is usually what you want for a panel.

Alternatives

Need	Better tool
Bulk genome/annotation download	NCBI datasets CLI, or FTP
Ensembl data	Ensembl REST API, or pyensembl
UniProt	UniProt REST API
Variant annotation	Ensembl VEP, ClinVar VCF downloads
Transcript–genome alignments	UTA — see UTA and Data Providers
Sequences by accession	SeqRepo — see SeqRepo

For variant work specifically, SeqRepo and UTA are far better than Entrez: local, versioned, fast, and reproducible.

Common mistakes

• Not setting Entrez.email.
• No API key, so you are limited to 3 req/s.
• One request per ID in a loop.
• No caching, making the pipeline slow and irreproducible.
• Unversioned accessions.
• No retry logic for transient 500s.
• Bulk downloading through E-utilities instead of FTP.
• Not closing handles.
• Hardcoding the API key in a committed script.

See also

SeqIO · SeqRecord · Bio.Blast · SeqRepo · UTA and Data Providers · Applied - Reproducible Environment