← Về thư mục
📄 / / profiles / r-and-d / raw_evidence / OpenAI4S / skills / bioskills / bio-database-access-entrez-fetch / SKILL.md

name: bio-entrez-fetch description: Retrieve records from NCBI databases using Biopython Bio.Entrez (EFetch, ESummary). Use when downloading sequences, fetching GenBank/GenPept records, getting document summaries, parsing nested XML, navigating GI deprecation, choosing between rettype+retmode combinations, and parsing into Biopython SeqRecord/SwissProt objects. Covers nucleotide, protein, gene, pubmed, sra, gds, taxonomy, snp, clinvar. origin: openai4s category: bioskills/database-access metadata: tool_type: python primary_tool: Bio.Entrez third_party: name: GPTomics/bioSkills repository: https://github.com/GPTomics/bioSkills commit: d91ed3d563019e649dc854c56ccd62551359488a license: MIT

Version Compatibility

Reference examples tested with: BioPython 1.83+, Entrez Direct 21.0+

Before using code patterns, verify installed versions match. If versions differ: - Python: pip show biopython then help(Bio.Entrez.efetch) to check signatures - CLI: efetch -version then efetch -help to confirm flags

If code throws ImportError, AttributeError, or TypeError, introspect the installed package and adapt the example to match the actual API rather than retrying.

Entrez Fetch

"Download a record by accession from NCBI" -> EFetch returns the full record content in a chosen format (FASTA, GenBank, XML, MEDLINE, etc.). ESummary returns a lightweight "docsum" object — much faster when only metadata is needed.

The agent's first decision is always: does this workflow need the full record, or just metadata? ESummary is 5-10x cheaper than EFetch for the equivalent record set. For "tell me the organism, length, and definition line for 10,000 accessions", ESummary wins by an order of magnitude.

Required Setup

from Bio import Entrez, SeqIO
Entrez.email = 'researcher@institution.edu'
Entrez.api_key = 'optional_api_key'  # raises rate to 10 req/sec

Decision matrix: rettype + retmode per database

The combinations are not orthogonal — each (db, rettype, retmode) triple is enabled or disabled by NCBI server-side. Wrong combinations return either silent empty responses or HTTP 400. The triples below are the safe, current set.

nucleotide / protein

rettype retmode Returns Use when
fasta text FASTA Just need sequence + defline
gb (nuc) / gp (prot) text Full flat file Need annotations, features, references
gbwithparts text GB with CONTIG sequences inlined Whole-genome shotgun assemblies; default gb returns CONTIG records requiring a chase to resolve
fasta_cds_na text CDS-only nucleotide Extract coding regions from annotated GB
fasta_cds_aa text CDS-translated AA Get translated proteins from GB record in one call
xml (== gb XML) xml INSDSeq XML Programmatic parsing; the schema is unversioned and shifts
acc text Accession.version per line Just resolve UID -> accession
seqid text Internal seq-id Rarely needed

pubmed

rettype retmode Returns Use when
abstract text Title + authors + abstract Reading abstracts
medline text MEDLINE flat Parsing with Bio.Medline
xml xml Full PubMed XML Programmatic — get MeSH, grants, PMC link
(omitted) (omitted) Defaults to XML EFetch default for pubmed is XML — pass retmode='xml' explicitly for clarity

gene

rettype retmode Returns Use when
gene_table text Tabular per-transcript layout Exon coordinates
xml xml Full Entrez Gene XML Everything else — name, synonyms, GeneRIFs, locus

sra

rettype retmode Returns Use when
runinfo text CSV of run metadata Convert SRA UID -> SRR accession + Run metrics
xml xml Full SRA XML hierarchy Need BioSample/BioProject linkage in one call

taxonomy

rettype retmode Returns Use when
xml xml (default) TaxNode XML Lineage, parent, common name

gds (GEO)

rettype retmode Returns Use when
(default — no rettype) text Plaintext SOFT-style summary Quick metadata; for full series matrix go to FTP

EFetch for GDS records is intentionally minimal — full GEO downloads go via the FTP mirror or GEOparse. See geo-data skill.

GI deprecation (still bites in 2026)

NCBI stopped issuing new GI numbers for major nucleotide/protein submissions starting 2017. Records submitted after the cutoff have only accession.version identifiers. Many older scripts assume id=<numeric_gi>; passing a modern accession string also works, but mixing the two in one comma-separated id list is the bug.

Rules: - For modern code, always pass accession.version strings. - A bare accession without .version resolves to the latest version — fine for exploratory work, dangerous for reproducibility. - Old id=12345 GI lookups still work for records issued before 2017, but a search returning a UID that looks like a GI may actually be the legacy GI for an old record — assume UID is an opaque identifier. - EFetch accepts comma-separated IDs of mixed types but the URL has a ~2000 char practical limit; chunk large ID lists into batches.

ESummary vs EFetch triage

Need ESummary EFetch (text) EFetch (xml)
Title, organism, length yes overkill overkill
Authors of a PubMed article yes yes yes
Full abstract text no rettype=abstract better — structured
MeSH terms, grant info, PMC ID no no yes
Sequence no rettype=fasta overkill
Sequence features (CDS, exons) no rettype=gb yes
Cross-references (xref) partial yes (in GB) yes
Bulk metadata for 10K records best (1 call per ~500) slow slow

ESummary's documented hard limit is 10,000 docsums per call, but the practical sweet spot is ~500 (keeps the URL under length limits when IDs are comma-joined; for >500 use EPost to push IDs server-side first). Per-record payload is much smaller than EFetch. Use ESummary as the default for any metadata-only workflow.

XML schema brittleness

Entrez.read() parses INSDSeq XML, PubmedArticle XML, Gene XML, etc. The schemas are NOT versioned; NCBI adds and renames fields without notice. Real-world consequence: a parser that worked in 2022 may KeyError in 2026 because a nested field moved.

Defensive patterns: - Use .get(key, default) not [key] for every nested field - For sequence content, prefer SeqIO.read() over Entrez.read() — the SeqIO parsers are versioned with BioPython - Pin BioPython version in production code; expect to update the parser when NCBI changes the XML - For PubMed, Bio.Medline.parse(handle) (against rettype='medline') is more stable than the XML route

Code patterns

Single sequence by accession

Goal: Fetch one nucleotide record as a SeqRecord with features.

Approach: EFetch with rettype='gb', retmode='text'; parse with SeqIO.read().

Reference (BioPython 1.83+):

def fetch_genbank(accession):
    h = Entrez.efetch(db='nucleotide', id=accession, rettype='gb', retmode='text')
    record = SeqIO.read(h, 'genbank'); h.close()
    return record

gb = fetch_genbank('NM_007294.4')
for feat in gb.features:
    if feat.type == 'CDS':
        print(feat.location, feat.qualifiers.get('product', ['?'])[0])

Bulk metadata via ESummary

Goal: Get organism + length + title for 1,000 UIDs without downloading sequences.

Approach: ESummary on a comma-joined ID batch (max 500 per call by convention; supports 10K hard limit).

Reference (BioPython 1.83+):

def bulk_summaries(db, ids, chunk=500):
    out = []
    for i in range(0, len(ids), chunk):
        h = Entrez.esummary(db=db, id=','.join(ids[i:i+chunk]))
        out.extend(Entrez.read(h)); h.close()
        time.sleep(0.1 if Entrez.api_key else 0.34)
    return out

records = bulk_summaries('nucleotide', uid_list)

Extract CDS in one round-trip

Goal: Download the CDS-only translated protein sequences from a GenBank record without manually walking features.

Approach: Use rettype='fasta_cds_aa' — NCBI server-side extracts and translates every CDS in the record.

Reference (BioPython 1.83+):

def cds_proteins(accession):
    h = Entrez.efetch(db='nucleotide', id=accession, rettype='fasta_cds_aa', retmode='text')
    return list(SeqIO.parse(h, 'fasta'))

proteins = cds_proteins('NC_000913.3')  # E. coli K-12 genome
print(f'{len(proteins)} CDS-translated proteins')

Pull PubMed with structured MeSH

Goal: Get MeSH terms and grant information that aren't in the abstract format.

Approach: rettype='xml' and walk the PubmedArticle structure defensively.

Reference (BioPython 1.83+):

def pubmed_full(pmid):
    h = Entrez.efetch(db='pubmed', id=pmid, retmode='xml')
    records = Entrez.read(h); h.close()
    article = records['PubmedArticle'][0]
    citation = article['MedlineCitation']
    mesh = [m['DescriptorName'] for m in citation.get('MeshHeadingList', [])]
    title = citation['Article']['ArticleTitle']
    return {'pmid': pmid, 'title': title, 'mesh': mesh}

History-server fetch (post-ESearch)

Goal: Pull a 50,000-record result set without re-sending UIDs.

Approach: ESearch with usehistory='y'; iterate EFetch with webenv/query_key and retstart. See batch-downloads for the production pattern.

h = Entrez.esearch(db='nucleotide', term='Homo sapiens[ORGN] AND srcdb_refseq[PROP] AND biomol_mrna[PROP]',
                   usehistory='y', retmax=0)
r = Entrez.read(h); h.close()
total = int(r['Count'])

with open('out.fasta', 'w') as out:
    for start in range(0, total, 500):
        h = Entrez.efetch(db='nucleotide', rettype='fasta', retmode='text',
                          retstart=start, retmax=500,
                          webenv=r['WebEnv'], query_key=r['QueryKey'])
        out.write(h.read()); h.close()
        time.sleep(0.1 if Entrez.api_key else 0.34)

SRA UID -> SRR accession + run metrics

Goal: Convert an opaque SRA UID into the SRR run accession plus Bases/Spots metrics, in one EFetch.

Approach: rettype='runinfo' returns a CSV row per run.

def sra_runinfo(uids):
    h = Entrez.efetch(db='sra', id=','.join(uids), rettype='runinfo', retmode='text')
    text = h.read(); h.close()
    lines = text.strip().split('\n')
    header = lines[0].split(',')
    return [dict(zip(header, row.split(','))) for row in lines[1:]]

Taxonomy lineage by TXID

def lineage(txid):
    h = Entrez.efetch(db='taxonomy', id=str(txid), retmode='xml')
    record = Entrez.read(h)[0]; h.close()
    return record['Lineage'], record['ScientificName']

Failure modes

Mixed-format batch silently truncates

gb returns CONTIG instead of sequence

XML parse fails on schema drift

Silent empty response on bad rettype

Accession without .version returns wrong record later

EFetch returns HTML error page

Common errors

Error / symptom Cause Solution
HTTPError 400 Invalid id/db/rettype combo Verify against decision matrix; check accession exists
HTTPError 429 Rate limit exceeded Add time.sleep(0.34) or use API key
Empty SeqRecord.seq WGS record with rettype='gb' Use rettype='gbwithparts'
ValueError: Sequence too short Wrong format declared to SeqIO Match rettype to SeqIO format string
ExpatError Got HTML where XML expected Sniff response start; retry
KeyError on nested XML field Schema drift Use .get() defensively; pin BioPython

References