UTA and Data Providers

In one line: the Universal Transcript Archive is a PostgreSQL database of transcript–genome alignments, and it is what makes everything except parsing possible.

Why a database is needed

Converting NM_000546.6:c.215C>G to genomic coordinates requires knowing:

• where that transcript aligns on the genome, exon by exon
• which strand it is on
• where the CDS starts and ends within it
• whether the alignment has indels relative to the genome (transcripts and the reference genome genuinely disagree in places)

None of that is derivable from the variant string. It has to be looked up. UTA is that lookup.

Connecting

import hgvs.dataproviders.uta

hdp = hgvs.dataproviders.uta.connect()          # public remote instance by default
hdp = hgvs.dataproviders.uta.connect(
    "postgresql://anonymous@localhost:5432/uta/uta_20210129"
)

Or via the environment, which is the better pattern:

export UTA_DB_URL=postgresql://anonymous@localhost:5432/uta/uta_20210129

The URL's final path segment (uta_20210129) is the schema name, which is the dataset version. Different schemas are different snapshots of RefSeq/Ensembl alignments, and they can give different answers.

Build hdp once and pass it around. Each connect() opens a database connection; doing it per variant is catastrophic for performance.

Remote vs local

The default remote instance (uta.biocommons.org) is a free public service.

Use it for: learning, exploration, a handful of variants, testing.

Do not use it for: any pipeline, any batch job, anything time-sensitive. It is a shared courtesy resource — slow, rate-limited, and its availability is not guaranteed. Running 50,000 variants through it is both painful for you and antisocial.

Running UTA locally

Docker is the practical route:

docker run -d --name uta -p 5432:5432 biocommons/uta:uta_20210129
export UTA_DB_URL=postgresql://anonymous@localhost:5432/uta/uta_20210129

Roughly 5–10 GB on disk. The speed difference is 10–100× on batch work — a job that takes hours remotely takes minutes locally.

For production, restore the SQL dump into your own PostgreSQL instance rather than relying on the container.

What the provider offers

hdp.data_version()                                  # schema version
hdp.schema_version()
hdp.get_tx_info(tx_ac, alt_ac, alt_aln_method)      # CDS bounds, transcript info
hdp.get_tx_exons(tx_ac, alt_ac, alt_aln_method)     # exon structure
hdp.get_tx_for_gene("TP53")                          # all transcripts for a gene
hdp.get_tx_for_region(alt_ac, method, start, end)    # transcripts overlapping a locus
hdp.get_acs_for_protein_seq(seq)
hdp.get_gene_info("TP53")
hdp.get_similar_transcripts(tx_ac)
hdp.get_pro_ac_for_tx_ac(tx_ac)                      # transcript → protein accession

Most of the time you use these indirectly through AssemblyMapper, but they are useful for building transcript-selection logic.

import pandas as pd
tx = pd.DataFrame(hdp.get_tx_for_gene("TP53"),
                  columns=["hgnc", "cds_start_i", "cds_end_i", "tx_ac",
                           "alt_ac", "alt_aln_method"])
tx[tx["alt_ac"].str.startswith("NC_0000")]      # only primary assembly alignments

Alignment methods

alt_aln_method="splign"      # NCBI's aligner — for RefSeq transcripts. THE DEFAULT.
alt_aln_method="blat"        # UCSC
alt_aln_method="genebuild"   # Ensembl
alt_aln_method="transcript"  # transcript as its own reference

Use splign for RefSeq (NM_/NR_) transcripts. Mixing methods for the same transcript can give different genomic coordinates, because the aligners place indels differently in ambiguous regions. Pick one and record it.

Caching for speed and reproducibility

from hgvs.dataproviders.seqfetcher import SeqFetcher
import hgvs.dataproviders.uta

hdp = hgvs.dataproviders.uta.connect(pooling=True)

pooling=True reuses connections — a straightforward win for batch work.

For a fully offline and reproducible setup, biocommons provides caching wrappers, and hgvs's own test suite uses a cached dataset so tests need no network. In a pipeline, wrapping hdp in your own memoisation of get_tx_info/get_tx_exons is cheap and effective, since the same handful of transcripts is queried thousands of times.

Pin your data version

print(hdp.data_version())        # e.g. 'uta_20210129'

Record this in every output. Two facts make it essential:

1. Normalization results depend on the sequence data, which depends on the UTA snapshot.
2. Transcript sets change. A transcript present in one snapshot may be suppressed in the next, and CDS boundaries occasionally get corrected.

An analysis that does not record its UTA and SeqRepo versions is not reproducible, no matter how carefully you pinned your Python packages. See Applied - Reproducible Environment.

Common connection errors

Error	Cause
HGVSDataNotAvailableError: No transcript definition for ...	transcript not in this UTA snapshot — often a version mismatch
psycopg2.OperationalError	database unreachable; check UTA_DB_URL and that the container is running
Very slow projection	you are on the remote instance, or reconnecting per variant
No alignment for tx X on Y with method splign	wrong alignment method, or the transcript is unplaced on that assembly

HGVSDataNotAvailableError with a versioned accession usually means that exact version is absent. Query get_tx_for_gene to see which versions the snapshot has, and decide deliberately whether to fall back to a different version — do not strip the version silently.

Alternative providers

The provider interface is pluggable:

from hgvs.dataproviders.interface import Interface

Alternatives exist — including implementations backed by Ensembl, and by SeqRepo-only setups for reference lookups. The openvar/vv_hgvs fork (VariantValidator) uses its own provider. Unless you have a specific need, UTA is the well-trodden path.

Common mistakes

• Using the public remote instance for a pipeline.
• Reconnecting per variant.
• Not recording the UTA version.
• Mixing alt_aln_method values.
• Stripping accession versions when a lookup fails.
• Assuming every transcript is present. UTA covers what RefSeq/Ensembl aligned; novel and suppressed transcripts are not there.
• Ignoring HGVSDataNotAvailableError instead of reporting which variants could not be processed.

See also

SeqRepo · AssemblyMapper · hgvs easy · biocommons hgvs · hgvs Normalizer · Applied - Reproducible Environment