User guide

Pipeline concepts

genome_entropy connects several distinct operations:

  1. ORF discovery calls candidate coding regions in six reading frames using get_orfs. This is gene calling, not functional annotation.

  2. Translation uses the selected NCBI genetic code (table 11 by default).

  3. Structural-state prediction converts protein sequence into 3Di and, with multitask ModernProst, a 12-state encoding.

  4. Entropy calculation measures observed symbol diversity at each available representation.

  5. GenBank matching, when annotations are supplied, assigns in_genbank through a strand- and C-terminus-aware heuristic.

ORFs and genetic codes

--min-nt controls the minimum nucleotide length for orf; run exposes the equivalent --min-aa and multiplies it by three. Table 11 is the bacterial, archaeal, and plant-plastid code. Select a different table only when appropriate for the source organism.

Coordinates and start/stop flags follow the exact implementation conventions in Data formats, coordinates, and entropy; do not reinterpret them as BED coordinates.

Structural-state encodings

3Di is a 20-state structural alphabet introduced by Foldseek. It describes local tertiary interactions inferred here from protein sequence; it is not a set of atomic coordinates and does not replace full structure prediction.

The multitask ModernProst head also emits 12 classes serialised as A through L. Legacy encoders expose missing 12-state values as None/JSON null. See Models and structural-state encodings for model selection, provenance, and security.

Shannon entropy

For observed symbol frequencies p_i, raw Shannon entropy in bits is:

\[H = -\sum_i p_i \log_2(p_i)\]

Zero describes a sequence containing one observed symbol. Larger values reflect more even use of more symbols, but do not by themselves establish sequence quality, function, or biological complexity. Short sequences cannot realise the theoretical maximum reliably, and comparisons across lengths or alphabets need caution.

Standard output stores raw entropy only. Normalised entropy divides by the theoretical maximum and should be derived downstream. The formula, alphabet sizes, helpers, and missing-value semantics are documented in Raw and normalised entropy.

Choosing an input workflow

Use DNA FASTA when only sequence-derived features are required. Use GenBank when in_genbank labels are required. Use protein FASTA with encode3di when ORF calling and nucleotide entropy are outside the analysis. Gzip support and each intermediate JSON format are listed in Data formats, coordinates, and entropy.

Performance and reproducibility

The encoding budget is an approximate sum of amino-acid lengths per batch, not a tokenizer token count guaranteed across models. Measure it on representative hardware. Pre-cache model artefacts for offline jobs and record package, model, PyTorch, Transformers, device, and parameter versions in an analysis provenance record.

Multi-GPU mode parallelises batches across one encoder per visible accelerator. It does not split an individual sequence across devices. See GPU and HPC operation for scheduler and ROCm details.

Interpretation limits

in_genbank=False means the current matching heuristic did not find a CDS; it does not prove the ORF is non-coding. Conversely, True is a heuristic annotation match, not a functional assignment. ML predictions learn this label and inherit its biases. Use genome- or file-level splits to estimate transfer to unseen records and read Machine-learning workflow before reporting results.

The GenBank heuristic normalises ORF and Biopython coordinates, then requires the same parent, strand, codon phase, at least 90% overlap of the shorter coding interval, and at least 98% compatibility across coordinate-aligned amino acids. This permits alternative starts and small terminal differences but not arbitrary local protein similarity. An aligned X is compatible with a specific residue. CDS-specific transl_table and codon_start qualifiers are respected, including when a missing /translation must be generated from the feature nucleotides. See Data formats, coordinates, and entropy for compound-location and partial-CDS handling.