Python API reference

The CLI is the primary stable interface. Package __init__ modules export the objects shown below; implementation modules may expose additional semi-public helpers. Type annotations and generated signatures come from the current code.

ORF discovery

ORF finder wrapper using get_orfs binary.

genome_entropy.orf.finder.find_orfs(sequences, table_id=11, min_nt_length=90, binary_path='get_orfs')

Find ORFs in DNA sequences using get_orfs binary.

This function wraps the external get_orfs binary (https://github.com/linsalrob/get_orfs). The binary must be installed and available in PATH or specified via binary_path.

Parameters:

• sequences (Dict[str, str]) – Dictionary mapping sequence IDs to DNA sequences

• table_id (int) – NCBI genetic code table ID (default: 11, bacterial)

• min_nt_length (int) – Minimum ORF length in nucleotides (default: 90)

• binary_path (str) – Path to get_orfs binary (default: from config/environment)

Returns:

List of OrfRecord objects

Raises:

OrfFinderError – If get_orfs binary is not found or fails

Return type:

List[OrfRecord]

genome_entropy.orf.finder.reverse_complement(seq)

Return the reverse complement of a DNA sequence.

Parameters:

seq (str)

Return type:

str

Data types for ORF representation.

class genome_entropy.orf.types.OrfRecord(parent_id, orf_id, start, end, strand, frame, nt_sequence, aa_sequence, table_id, has_start_codon, has_stop_codon, in_genbank=False)

Represents a single Open Reading Frame (ORF).

Variables:

• parent_id (str) – ID of the parent DNA sequence

• orf_id (str) – Unique identifier for this ORF

• start (int) – One-based inclusive coordinate from get_orfs output

• end (int) – One-based inclusive coordinate from get_orfs output

• strand (Literal['+', '-']) – Strand orientation (‘+’ or ‘-‘)

• frame (int) – Absolute reading-frame number (0, 1, 2, or 3)

• nt_sequence (str) – Nucleotide sequence of the ORF

• aa_sequence (str) – Amino acid sequence of the ORF

• table_id (int) – NCBI genetic code table ID used

• has_start_codon (bool) – Whether the source amino-acid string contains M

• has_stop_codon (bool) – Whether the source amino-acid string contains *

• in_genbank (bool) – Whether the coordinate-anchored GenBank CDS matcher matched this ORF

Parameters:

• parent_id (str)

• orf_id (str)

• start (int)

• end (int)

• strand (Literal['+', '-'])

• frame (int)

• nt_sequence (str)

• aa_sequence (str)

• table_id (int)

• has_start_codon (bool)

• has_stop_codon (bool)

• in_genbank (bool)

parent_id: str

orf_id: str

start: int

end: int

strand: Literal['+', '-']

frame: int

nt_sequence: str

aa_sequence: str

table_id: int

has_start_codon: bool

has_stop_codon: bool

in_genbank: bool = False

__post_init__()

Validate ORF attributes.

Return type:

None

__init__(parent_id, orf_id, start, end, strand, frame, nt_sequence, aa_sequence, table_id, has_start_codon, has_stop_codon, in_genbank=False)

Parameters:

• parent_id (str)

• orf_id (str)

• start (int)

• end (int)

• strand (Literal['+', '-'])

• frame (int)

• nt_sequence (str)

• aa_sequence (str)

• table_id (int)

• has_start_codon (bool)

• has_stop_codon (bool)

• in_genbank (bool)

Return type:

None

Translation

Translation of nucleotide sequences to amino acids.

class genome_entropy.translate.translator.ProteinRecord(orf, aa_sequence, aa_length)

Represents a translated protein from an ORF.

Variables:

• orf (genome_entropy.orf.types.OrfRecord) – The OrfRecord that was translated

• aa_sequence (str) – The amino acid sequence

• aa_length (int) – Length of the amino acid sequence

Parameters:

• orf (OrfRecord)

• aa_sequence (str)

• aa_length (int)

orf: OrfRecord

aa_sequence: str

aa_length: int

__post_init__()

Validate protein attributes.

Return type:

None

__init__(orf, aa_sequence, aa_length)

Parameters:

• orf (OrfRecord)

• aa_sequence (str)

• aa_length (int)

Return type:

None

genome_entropy.translate.translator.translate_orf(orf, table_id=11)

Translate an ORF to a protein sequence.

Uses pygenetic-code for unambiguous DNA and Biopython for sequences that contain IUPAC ambiguity codes. This prevents a multiply-resolvable codon such as AAN or NNN from being assigned an arbitrary amino acid while preserving specific translations for resolvable codons such as GCN.

Parameters:

• orf (OrfRecord) – OrfRecord to translate

• table_id (int) – NCBI genetic code table ID (default: from config)

Returns:

ProteinRecord with translated sequence

Raises:

TranslationError – If translation fails

Return type:

ProteinRecord

genome_entropy.translate.translator.translate_orfs(orfs, table_id=11)

Translate multiple ORFs to protein sequences.

Parameters:

• orfs (List[OrfRecord]) – List of OrfRecord objects to translate

• table_id (int) – NCBI genetic code table ID

Returns:

List of ProteinRecord objects

Return type:

List[ProteinRecord]

Structural-state encoding

Data types for structural-state encoding.

class genome_entropy.encode3di.types.ThreeDiRecord(protein, three_di, method, model_name, inference_device, twelve_state=None)

Structural-state encodings predicted for a protein.

Variables:

• protein (genome_entropy.translate.translator.ProteinRecord) – The ProteinRecord that was encoded

• three_di (str) – The 3Di token sequence

• method (str) – Encoder method identifier

• model_name (str) – Canonical model identifier used for inference

• inference_device (str) – Device string, such as cuda, mps, or cpu

• twelve_state (str | None) – Optional 12-state sequence; None for 3Di-only models

Parameters:

• protein (ProteinRecord)

• three_di (str)

• method (str)

• model_name (str)

• inference_device (str)

• twelve_state (str | None)

protein: ProteinRecord

three_di: str

method: str

model_name: str

inference_device: str

twelve_state: str | None = None

__init__(protein, three_di, method, model_name, inference_device, twelve_state=None)

Parameters:

• protein (ProteinRecord)

• three_di (str)

• method (str)

• model_name (str)

• inference_device (str)

• twelve_state (str | None)

Return type:

None

class genome_entropy.encode3di.types.StructuralEncoding(three_di, twelve_state)

Associated structural encodings produced by one model forward pass.

Parameters:

• three_di (str)

• twelve_state (str | None)

three_di: str

twelve_state: str | None

__init__(three_di, twelve_state)

Parameters:

• three_di (str)

• twelve_state (str | None)

Return type:

None

class genome_entropy.encode3di.types.IndexedSeq(idx, seq)

A sequence paired with its original position in the input list.

Parameters:

• idx (int)

• seq (str)

idx: int

seq: str

__init__(idx, seq)

Parameters:

• idx (int)

• seq (str)

Return type:

None

ProstT5-based encoder for amino acid to 3Di structural token conversion.

class genome_entropy.encode3di.encoder.ProstT5ThreeDiEncoder(model_name='Rostlab/ProstT5_fp16', device=None)

Encoder for converting amino acid sequences to 3Di structural tokens.

Uses the ProstT5 model from HuggingFace to predict 3Di tokens directly from protein sequences without requiring 3D structures.

Parameters:

• model_name (str)

• device (str | None)

__init__(model_name='Rostlab/ProstT5_fp16', device=None)

Initialize the ProstT5 encoder.

Parameters:

• model_name (str) – HuggingFace model identifier

• device (str | None) – Device to use (“cuda”, “mps”, “cpu”, or None for auto-detect)

Raises:

• ModelError – If PyTorch or Transformers are not installed

• DeviceError – If specified device is not available

token_budget_batches(aa_sequences, token_budget)

Yield batches of sequences (with original indices) under an approximate token budget.

Optimized strategy to address the problem of isolated long sequences:

1. Keep original indices.

2. Sort by length to minimize padding within each batch.

3. For each batch: - Start with long sequences from the end (largest first) - Add long sequences until adding another would exceed budget - Fill remaining budget with short sequences from the beginning

4. This approach avoids ending up with long proteins that can’t be combined, resulting in better token budget utilization and fewer iterations.

Parameters

aa_sequences : Sequence[str] Unordered amino acid sequences. token_budget : int Maximum approximate “tokens” per batch

Yields

List[IndexedSeq] A batch of (original_index, sequence) records.

Parameters:

• aa_sequences (Sequence[str])

• token_budget (int)

Return type:

Iterator[List[IndexedSeq]]

encode(aa_sequences, encoding_size=10000, use_multi_gpu=False, gpu_ids=None, multi_gpu_encoder=None)

Encode amino acid sequences to 3Di tokens.

Parameters:

• aa_sequences (List[str]) – List of amino acid sequences. note: Amino acid sequences are expected to be upper-case, while 3Di sequences need to be lower-case.

• encoding_size (int) – Maximum size (approx. amino acids) to encode per gpu

• use_multi_gpu (bool) – If True, use multi-GPU parallel encoding when available

• gpu_ids (List[int] | None) – Optional list of GPU IDs to use for multi-GPU encoding. If None and use_multi_gpu=True, auto-discover available GPUs.

• multi_gpu_encoder (Any | None) – Optional pre-initialized MultiGPUEncoder instance. If provided, this encoder will be reused instead of creating a new one. This is important for efficiency when processing multiple sequences.

Returns:

List of 3Di token sequences (one per input sequence)

Raises:

EncodingError – If encoding fails

Return type:

List[str]

encode_proteins(proteins, encoding_size=10000, use_multi_gpu=False, gpu_ids=None, multi_gpu_encoder=None)

Encode protein records to 3Di records.

Parameters:

• proteins (List[ProteinRecord]) – List of ProteinRecord objects

• encoding_size (int) – Maximum size (approx. amino acids) to encode per batch

• use_multi_gpu (bool) – If True, use multi-GPU parallel encoding when available

• gpu_ids (List[int] | None) – Optional list of GPU IDs to use for multi-GPU encoding

• multi_gpu_encoder (Any | None) – Optional pre-initialized MultiGPUEncoder instance. If provided, this encoder will be reused instead of creating a new one. This is important for efficiency when processing multiple sequences.

Returns:

List of ThreeDiRecord objects

Return type:

List[ThreeDiRecord]

ModernProst encoder for amino acid to 3Di structural token conversion.

This module implements an encoder for gbouras13/modernprost models, adapted from the phold implementation.

Note: The multitask ModernProst models require transformers >= 5.14.1. Multi-GPU support uses HuggingFace accelerate library.

class genome_entropy.encode3di.modernprost.ModernProstThreeDiEncoder(model_name, device=None, use_accelerate=False)

Encoder for converting proteins to structural-state tokens.

Multitask models predict paired 3Di and 12-state sequences, while deprecated models retain the tensor-only 3Di output API.

Based on implementation from phold: https://github.com/gbouras13/phold/blob/main/src/phold/features/predict_3Di.py

Parameters:

• model_name (str)

• device (str | None)

• use_accelerate (bool)

__init__(model_name, device=None, use_accelerate=False)

Initialize the ModernProst encoder.

Parameters:

• model_name (str) – A supported Hugging Face model identifier.

• device (str | None) – Device to use (“cuda”, “mps”, “cpu”, or None for auto-detect)

• use_accelerate (bool) – If True, use HuggingFace accelerate for multi-GPU support

Raises:

• ModelError – If PyTorch or Transformers are not installed

• DeviceError – If specified device is not available

token_budget_batches(aa_sequences, token_budget)

Yield batches of sequences (with original indices) under an approximate token budget.

Optimized strategy to address the problem of isolated long sequences:

1. Keep original indices.

2. Sort by length to minimize padding within each batch.

3. For each batch: - Start with long sequences from the end (largest first) - Add long sequences until adding another would exceed budget - Fill remaining budget with short sequences from the beginning

4. This approach avoids ending up with long proteins that can’t be combined, resulting in better token budget utilization and fewer iterations.

Parameters

aa_sequences : Sequence[str] Unordered amino acid sequences. token_budget : int Maximum approximate “tokens” per batch

Yields

List[IndexedSeq] A batch of (original_index, sequence) records.

Parameters:

• aa_sequences (Sequence[str])

• token_budget (int)

Return type:

Iterator[List[IndexedSeq]]

encode(aa_sequences, encoding_size=10000, use_multi_gpu=False, gpu_ids=None, multi_gpu_encoder=None)

Encode amino acid sequences to 3Di tokens.

Parameters:

• aa_sequences (List[str]) – List of amino acid sequences (upper-case).

• encoding_size (int) – Maximum size (approx. amino acids) to encode per batch

• use_multi_gpu (bool) – If True, use accelerate for multi-GPU parallel encoding

• gpu_ids (List[int] | None) – Optional list of GPU IDs (currently unused with accelerate)

• multi_gpu_encoder (Any | None) – Optional pre-initialized encoder (for backward compatibility)

Returns:

List of 3Di token sequences (one per input sequence)

Raises:

EncodingError – If encoding fails

Return type:

List[StructuralEncoding]

encode_proteins(proteins, encoding_size=10000, use_multi_gpu=False, gpu_ids=None, multi_gpu_encoder=None)

Encode protein records to 3Di records.

Parameters:

• proteins (List[ProteinRecord]) – List of ProteinRecord objects

• encoding_size (int) – Maximum size (approx. amino acids) to encode per batch

• use_multi_gpu (bool) – If True, use multi-GPU parallel encoding when available

• gpu_ids (List[int] | None) – Optional list of GPU IDs to use for multi-GPU encoding

• multi_gpu_encoder (Any | None) – Optional pre-initialized MultiGPUEncoder instance.

Returns:

List of ThreeDiRecord objects

Return type:

List[ThreeDiRecord]

Multi-GPU asynchronous encoding for protein to 3Di conversion.

class genome_entropy.encode3di.multi_gpu.MultiGPUEncoder(model_name, encoder_class, gpu_ids=None)

Manages multi-GPU encoding of amino acid sequences to 3Di tokens.

This class distributes encoding batches across multiple GPUs using asyncio for parallel processing. It handles GPU allocation, load balancing, and error recovery.

Parameters:

• model_name (str)

• encoder_class (type)

• gpu_ids (List[int] | None)

__init__(model_name, encoder_class, gpu_ids=None)

Initialize multi-GPU encoder.

Parameters:

• model_name (str) – HuggingFace model identifier

• encoder_class (type) – Encoder class to instantiate (e.g., ProstT5ThreeDiEncoder)

• gpu_ids (List[int] | None) – List of GPU IDs to use. If None, auto-discover available GPUs. If empty list or None after discovery, falls back to single GPU.

property num_gpus: int

Number of GPUs being used.

is_multi_gpu()

Check if using multiple GPUs.

Return type:

bool

async encode_batch_async(encoder_idx, batch)

Encode a single batch on a specific GPU asynchronously.

Parameters:

• encoder_idx (int) – Index of encoder/GPU to use

• batch (List[IndexedSeq]) – List of IndexedSeq objects to encode

Returns:

Tuple of (original_indices, encoded_3di_sequences)

Return type:

Tuple[List[int], List[Any]]

async encode_all_batches_async(batches, total_sequences)

Encode all batches across multiple GPUs asynchronously.

Parameters:

• batches (List[List[IndexedSeq]]) – List of batches to encode

• total_sequences (int) – Total number of sequences

Returns:

List of encoded 3Di sequences in original input order

Raises:

EncodingError – If encoding fails

Return type:

List[Any]

encode_multi_gpu(aa_sequences, token_budget_batches_fn, encoding_size, skip_model_loading=False)

Encode sequences using multiple GPUs.

This is a synchronous wrapper around the async encoding method.

Parameters:

• aa_sequences (List[str]) – List of preprocessed amino acid sequences

• token_budget_batches_fn (Callable[[List[str], int], Iterator[Any]]) – Function to create batches under token budget

• encoding_size (int) – Maximum size (approx. amino acids) per batch

• skip_model_loading (bool) – If True, skip model loading (assumes models already loaded). This is useful when the encoder is being reused across multiple calls.

Returns:

List of 3Di token sequences (one per input sequence)

Return type:

List[Any]

GPU discovery and management utilities for multi-GPU encoding.

genome_entropy.encode3di.gpu_utils.discover_available_gpus()

Discover available GPU devices from environment variables and CUDA.

Checks multiple sources in order of priority: 1. SLURM_JOB_GPUS - SLURM allocated GPU IDs 2. SLURM_GPUS - Alternative SLURM GPU specification 3. CUDA_VISIBLE_DEVICES - User-specified visible devices 4. torch.cuda - Query CUDA directly if available

Returns:

List of GPU device IDs available for use. Empty list if no GPUs found.

Return type:

List[int]

Examples

>>> # With SLURM_JOB_GPUS="0,1,2"
>>> discover_available_gpus()
[0, 1, 2]

>>> # With CUDA_VISIBLE_DEVICES="2,3"
>>> discover_available_gpus()
[0, 1]  # Remapped to local indices

genome_entropy.encode3di.gpu_utils.select_device_for_gpu(gpu_id)

Get the device string for a specific GPU.

Parameters:

gpu_id (int) – GPU device ID

Returns:

0”, “cuda:1”)

Return type:

Device string (e.g., “cuda

genome_entropy.encode3di.gpu_utils.validate_gpu_availability(gpu_ids)

Validate that specified GPUs are actually available.

Parameters:

gpu_ids (List[int]) – List of GPU IDs to validate

Returns:

List of valid GPU IDs (subset of input)

Return type:

List[int]

Token size estimation for optimal GPU memory usage in 3Di encoding.

genome_entropy.encode3di.token_estimator.generate_random_protein(length, seed=None)

Generate a random protein sequence of specified length.

Parameters:

• length (int) – Length of the protein sequence

• seed (int | None) – Random seed for reproducibility (optional)

Returns:

Random protein sequence using the 20 standard amino acids

Return type:

str

genome_entropy.encode3di.token_estimator.generate_combined_proteins(target_length, base_length=100, seed=None)

Generate multiple shorter proteins that combine to target length.

Parameters:

• target_length (int) – Total target length across all proteins

• base_length (int) – Approximate length of each individual protein

• seed (int | None) – Random seed for reproducibility (optional)

Returns:

List of protein sequences that total approximately target_length

Return type:

List[str]

genome_entropy.encode3di.token_estimator.estimate_token_size(encoder, start_length=3000, end_length=10000, step=1000, num_trials=3, base_protein_length=100)

Estimate optimal token size for GPU encoding by testing increasing lengths.

This function generates random protein sequences of increasing total length and attempts to encode them. It catches OutOfMemoryError to find the maximum length that can be encoded on the available GPU.

Parameters:

• encoder (Any) – ProstT5ThreeDiEncoder instance to use for encoding

• start_length (int) – Starting total length to test (default: 3000)

• end_length (int) – Maximum total length to test (default: 10000)

• step (int) – Increment between test lengths (default: 1000)

• num_trials (int) – Number of trials per length for robustness (default: 3)

• base_protein_length (int) – Approximate length of individual proteins (default: 100)

Returns:

• ‘max_length’: Maximum length successfully encoded

• ’recommended_token_size’: Recommended token budget (90% of max)

• ’trials_per_length’: Dictionary of successful trials per length

• ’device’: Device used for testing

Return type:

Dictionary with estimation results

Raises:

ValueError – If encoder doesn’t have required attributes or torch not available

Entropy

The normalisation helpers are intended for downstream use and are not invoked by standard serialisation.

Shannon entropy calculation for sequences.

genome_entropy.entropy.shannon.normalise_entropy(entropy, alphabet_size)

Normalise a raw Shannon entropy using its theoretical alphabet size.

This helper is intended for downstream analysis. Normalised values are derived from raw entropy and are therefore not stored in standard output.

Parameters:

• entropy (float | None) – Raw Shannon entropy in bits, or None for missing data.

• alphabet_size (int) – The theoretical number of symbols in the representation.

Returns:

Entropy divided by log2(alphabet_size), or None when entropy is None.

Raises:

ValueError – If alphabet_size is not greater than one.

Return type:

float | None

genome_entropy.entropy.shannon.normalise_dna_entropy(entropy)

Normalise raw DNA entropy using the theoretical four-symbol alphabet.

Parameters:

entropy (float | None)

Return type:

float | None

genome_entropy.entropy.shannon.normalise_protein_entropy(entropy)

Normalise raw protein entropy using the theoretical 20-symbol alphabet.

Parameters:

entropy (float | None)

Return type:

float | None

genome_entropy.entropy.shannon.normalise_three_di_entropy(entropy)

Normalise raw 3Di entropy using the theoretical 20-symbol alphabet.

Parameters:

entropy (float | None)

Return type:

float | None

genome_entropy.entropy.shannon.normalise_twelve_state_entropy(entropy)

Normalise raw 12-state entropy using its theoretical alphabet.

Parameters:

entropy (float | None)

Return type:

float | None

class genome_entropy.entropy.shannon.EntropyReport(dna_entropy_global, orf_nt_entropy, protein_aa_entropy, three_di_entropy, alphabet_sizes, twelve_state_entropy=None)

Report containing entropy values at different representation levels.

Variables:

• dna_entropy_global (float) – Entropy of the entire input DNA sequence

• orf_nt_entropy (Dict[str, float]) – Dictionary mapping ORF IDs to their nucleotide entropy

• protein_aa_entropy (Dict[str, float]) – Dictionary mapping ORF IDs to their amino acid entropy

• three_di_entropy (Dict[str, float]) – Dictionary mapping ORF IDs to their 3Di token entropy

• alphabet_sizes (Dict[str, int]) – Dictionary with alphabet sizes for each representation

• twelve_state_entropy (Dict[str, float] | None) – Optional mapping of ORF IDs to 12-state entropy

Parameters:

• dna_entropy_global (float)

• orf_nt_entropy (Dict[str, float])

• protein_aa_entropy (Dict[str, float])

• three_di_entropy (Dict[str, float])

• alphabet_sizes (Dict[str, int])

• twelve_state_entropy (Dict[str, float] | None)

dna_entropy_global: float

orf_nt_entropy: Dict[str, float]

protein_aa_entropy: Dict[str, float]

three_di_entropy: Dict[str, float]

alphabet_sizes: Dict[str, int]

twelve_state_entropy: Dict[str, float] | None = None

__init__(dna_entropy_global, orf_nt_entropy, protein_aa_entropy, three_di_entropy, alphabet_sizes, twelve_state_entropy=None)

Parameters:

• dna_entropy_global (float)

• orf_nt_entropy (Dict[str, float])

• protein_aa_entropy (Dict[str, float])

• three_di_entropy (Dict[str, float])

• alphabet_sizes (Dict[str, int])

• twelve_state_entropy (Dict[str, float] | None)

Return type:

None

genome_entropy.entropy.shannon.shannon_entropy(sequence, alphabet=None, normalize=False)

Calculate Shannon entropy of a sequence.

Shannon entropy: \(H = -\sum_i p_i \log_2(p_i)\), where \(p_i\) is the frequency of symbol \(i\).

Parameters:

• sequence (str) – String to calculate entropy for

• alphabet (Set[str] | None) – Optional set of symbols in the alphabet for normalization

• normalize (bool) – Legacy explicit in-memory normalisation switch. Standard pipeline output never enables it; prefer normalise_entropy for downstream analysis.

Returns:

Shannon entropy value (bits) - Returns 0.0 for empty sequences - Returns normalized entropy in [0, 1] if normalize=True

Return type:

float

Examples

>>> shannon_entropy("AAAA")
0.0
>>> shannon_entropy("ACGT")
2.0

genome_entropy.entropy.shannon.calculate_sequence_entropy(sequence, alphabet=None, normalize=False)

Calculate entropy for a biological sequence.

Convenience wrapper around shannon_entropy that handles common preprocessing (e.g., converting to uppercase).

Parameters:

• sequence (str) – Biological sequence (DNA, protein, 3Di tokens)

• alphabet (Set[str] | None) – Optional alphabet for the legacy normalisation switch

• normalize (bool) – Legacy explicit normalisation switch; standard output is raw

Returns:

Shannon entropy in bits, or a legacy explicitly normalised value

Return type:

float

genome_entropy.entropy.shannon.calculate_entropies_for_sequences(sequences, alphabet=None, normalize=False)

Calculate entropy for multiple sequences.

Parameters:

• sequences (Dict[str, str]) – Dictionary mapping IDs to sequences

• alphabet (Set[str] | None) – Optional alphabet for normalization

• normalize (bool) – Whether to normalize by alphabet size

Returns:

Dictionary mapping IDs to entropy values

Return type:

Dict[str, float]

Pipeline and schemas

End-to-end pipeline orchestration for DNA to 3Di with entropy calculation.

class genome_entropy.pipeline.runner.PipelineResult(input_id, input_dna_length, orfs, proteins, three_dis, entropy)

Result of running the complete DNA to 3Di pipeline.

Variables:

• input_id (str) – ID of the input DNA sequence

• input_dna_length (int) – Length of the input DNA sequence

• orfs (List[genome_entropy.orf.types.OrfRecord]) – List of ORFs found in the sequence

• proteins (List[genome_entropy.translate.translator.ProteinRecord]) – List of translated proteins

• three_dis (List[genome_entropy.encode3di.types.ThreeDiRecord]) – List of 3Di encoded structures

• entropy (genome_entropy.entropy.shannon.EntropyReport) – Entropy report for all representations

Parameters:

• input_id (str)

• input_dna_length (int)

• orfs (List[OrfRecord])

• proteins (List[ProteinRecord])

• three_dis (List[ThreeDiRecord])

• entropy (EntropyReport)

input_id: str

input_dna_length: int

orfs: List[OrfRecord]

proteins: List[ProteinRecord]

three_dis: List[ThreeDiRecord]

entropy: EntropyReport

__init__(input_id, input_dna_length, orfs, proteins, three_dis, entropy)

Parameters:

• input_id (str)

• input_dna_length (int)

• orfs (List[OrfRecord])

• proteins (List[ProteinRecord])

• three_dis (List[ThreeDiRecord])

• entropy (EntropyReport)

Return type:

None

genome_entropy.pipeline.runner.run_pipeline(input_fasta=None, table_id=11, min_aa_len=30, model_name='gbouras13/modernprost-50M', compute_entropy=True, output_json=None, device=None, use_multi_gpu=False, gpu_ids=None, genbank_file=None, encoding_size=None)

Run the complete DNA to 3Di pipeline with entropy calculation.

Pipeline steps: 1. Read FASTA file or GenBank file 2. Find ORFs in all 6 reading frames 3. Translate ORFs to proteins 4. Encode proteins to 3Di structural tokens 5. Calculate entropy at all levels 6. Optionally match ORFs to GenBank CDS annotations 7. Optionally write results to JSON

Parameters:

• input_fasta (str | Path | None) – Path to input FASTA file. Optional if genbank_file is provided.

• table_id (int) – NCBI genetic code table ID

• min_aa_len (int) – Minimum protein length in amino acids

• model_name (str) – ProstT5 model name

• compute_entropy (bool) – Whether to compute entropy values

• output_json (str | Path | None) – Optional path to save results as JSON

• device (str | None) – Device for 3Di encoding (“cuda”, “mps”, “cpu”, or None for auto) Ignored if use_multi_gpu is True.

• use_multi_gpu (bool) – If True, use multi-GPU parallel encoding when available

• gpu_ids (List[int] | None) – Optional list of GPU IDs for multi-GPU encoding. If None and use_multi_gpu=True, auto-discover available GPUs.

• genbank_file (str | Path | None) – Optional path to GenBank file. If provided alone, extracts DNA sequences from it. Can be combined with input_fasta to use FASTA sequences with GenBank CDS annotations.

• encoding_size (int | None) – Maximum size (approx. amino acids) to encode per batch. If None, uses DEFAULT_ENCODING_SIZE from config.

Returns:

List of PipelineResult objects (one per input sequence)

Raises:

• PipelineError – If any pipeline step fails

• ValueError – If neither input_fasta nor genbank_file is provided

Return type:

List[PipelineResult]

genome_entropy.pipeline.runner.calculate_pipeline_entropy(dna_sequence, orfs, proteins, three_dis)

Calculate entropy at all representation levels.

Parameters:

• dna_sequence (str) – Original DNA sequence

• orfs (List[OrfRecord]) – List of ORF records

• proteins (List[ProteinRecord]) – List of protein records

• three_dis (List[ThreeDiRecord]) – List of 3Di records

Returns:

EntropyReport with entropy values

Return type:

EntropyReport

Unified data types for pipeline output format.

This module defines the unified feature structure that eliminates redundancy by consolidating ORF, protein, and 3Di data into a single hierarchical format.

The unified structure addresses the problem where: - The old proteins list duplicated entire ORF objects - The old three_dis list duplicated entire protein objects (which contained ORFs) - Each level repeated sequences, coordinates, and metadata

The new structure stores each piece of biological information exactly once, organized hierarchically by biological concept.

class genome_entropy.pipeline.types.FeatureLocation(start, end, strand, frame)

Genomic location of a feature (ORF).

Variables:

• start (int) – One-based inclusive coordinate from get_orfs

• end (int) – One-based inclusive coordinate from get_orfs

• strand (Literal['+', '-']) – Strand orientation (‘+’ or ‘-‘)

• frame (int) – Reading frame (0, 1, 2, or 3)

Parameters:

• start (int)

• end (int)

• strand (Literal['+', '-'])

• frame (int)

start: int

end: int

strand: Literal['+', '-']

frame: int

__init__(start, end, strand, frame)

Parameters:

• start (int)

• end (int)

• strand (Literal['+', '-'])

• frame (int)

Return type:

None

class genome_entropy.pipeline.types.FeatureDNA(nt_sequence, length)

DNA-level information for a feature.

Variables:

• nt_sequence (str) – Nucleotide sequence

• length (int) – Length of nucleotide sequence

Parameters:

• nt_sequence (str)

• length (int)

nt_sequence: str

length: int

__init__(nt_sequence, length)

Parameters:

• nt_sequence (str)

• length (int)

Return type:

None

class genome_entropy.pipeline.types.FeatureProtein(aa_sequence, length)

Protein-level information for a feature.

Variables:

• aa_sequence (str) – Amino acid sequence

• length (int) – Length of amino acid sequence

Parameters:

• aa_sequence (str)

• length (int)

aa_sequence: str

length: int

__init__(aa_sequence, length)

Parameters:

• aa_sequence (str)

• length (int)

Return type:

None

class genome_entropy.pipeline.types.FeatureThreeDi(encoding, length, method, model_name, inference_device)

3Di structural encoding for a feature.

Variables:

• encoding (str) – 3Di token sequence

• length (int) – Length of 3Di sequence

• method (str) – Method used for encoding (e.g., “prostt5_aa2fold”)

• model_name (str) – Name of the model used

• inference_device (str) – Device used for inference (“cuda”, “mps”, or “cpu”)

Parameters:

• encoding (str)

• length (int)

• method (str)

• model_name (str)

• inference_device (str)

encoding: str

length: int

method: str

model_name: str

inference_device: str

__init__(encoding, length, method, model_name, inference_device)

Parameters:

• encoding (str)

• length (int)

• method (str)

• model_name (str)

• inference_device (str)

Return type:

None

class genome_entropy.pipeline.types.FeatureTwelveState(encoding, length)

Twelve-state encoding, serialised as deterministic symbols A–L.

Parameters:

• encoding (str)

• length (int)

encoding: str

length: int

__init__(encoding, length)

Parameters:

• encoding (str)

• length (int)

Return type:

None

class genome_entropy.pipeline.types.FeatureMetadata(parent_id, table_id, has_start_codon, has_stop_codon, in_genbank)

Metadata about a feature.

Variables:

• parent_id (str) – ID of the parent DNA sequence

• table_id (int) – NCBI genetic code table ID used

• has_start_codon (bool) – Whether the source amino-acid string contains M

• has_stop_codon (bool) – Whether the source amino-acid string contains *

• in_genbank (bool) – Whether the C-terminal GenBank CDS heuristic matched

Parameters:

• parent_id (str)

• table_id (int)

• has_start_codon (bool)

• has_stop_codon (bool)

• in_genbank (bool)

parent_id: str

table_id: int

has_start_codon: bool

has_stop_codon: bool

in_genbank: bool

__init__(parent_id, table_id, has_start_codon, has_stop_codon, in_genbank)

Parameters:

• parent_id (str)

• table_id (int)

• has_start_codon (bool)

• has_stop_codon (bool)

• in_genbank (bool)

Return type:

None

class genome_entropy.pipeline.types.FeatureEntropy(dna_entropy, protein_entropy, three_di_entropy, twelve_state_entropy=None)

Entropy values at different representation levels for a feature.

Variables:

• dna_entropy (float) – Shannon entropy of nucleotide sequence

• protein_entropy (float) – Shannon entropy of amino acid sequence

• three_di_entropy (float) – Shannon entropy of 3Di encoding

• twelve_state_entropy (float | None) – Shannon entropy of 12-state encoding, or None

Parameters:

• dna_entropy (float)

• protein_entropy (float)

• three_di_entropy (float)

• twelve_state_entropy (float | None)

dna_entropy: float

protein_entropy: float

three_di_entropy: float

twelve_state_entropy: float | None = None

__init__(dna_entropy, protein_entropy, three_di_entropy, twelve_state_entropy=None)

Parameters:

• dna_entropy (float)

• protein_entropy (float)

• three_di_entropy (float)

• twelve_state_entropy (float | None)

Return type:

None

class genome_entropy.pipeline.types.UnifiedFeature(orf_id, location, dna, protein, three_di, metadata, entropy, twelve_state=None)

Unified representation of a biological feature (ORF and derived data).

This structure consolidates all information about a single ORF into one hierarchical object, eliminating the redundancy present in the old format where ORF data was duplicated in proteins list and protein data was duplicated in three_dis list.

Variables:

• orf_id (str) – Unique identifier for this feature

• location (genome_entropy.pipeline.types.FeatureLocation) – Genomic coordinates

• dna (genome_entropy.pipeline.types.FeatureDNA) – DNA sequence information

• protein (genome_entropy.pipeline.types.FeatureProtein) – Protein sequence information

• three_di (genome_entropy.pipeline.types.FeatureThreeDi) – 3Di structural encoding

• metadata (genome_entropy.pipeline.types.FeatureMetadata) – Additional metadata

• entropy (genome_entropy.pipeline.types.FeatureEntropy) – Entropy values at all representation levels

• twelve_state (genome_entropy.pipeline.types.FeatureTwelveState | None) – Optional 12-state encoding; None for 3Di-only models

Parameters:

• orf_id (str)

• location (FeatureLocation)

• dna (FeatureDNA)

• protein (FeatureProtein)

• three_di (FeatureThreeDi)

• metadata (FeatureMetadata)

• entropy (FeatureEntropy)

• twelve_state (FeatureTwelveState | None)

orf_id: str

location: FeatureLocation

dna: FeatureDNA

protein: FeatureProtein

three_di: FeatureThreeDi

metadata: FeatureMetadata

entropy: FeatureEntropy

twelve_state: FeatureTwelveState | None = None

__init__(orf_id, location, dna, protein, three_di, metadata, entropy, twelve_state=None)

Parameters:

• orf_id (str)

• location (FeatureLocation)

• dna (FeatureDNA)

• protein (FeatureProtein)

• three_di (FeatureThreeDi)

• metadata (FeatureMetadata)

• entropy (FeatureEntropy)

• twelve_state (FeatureTwelveState | None)

Return type:

None

class genome_entropy.pipeline.types.UnifiedPipelineResult(schema_version, input_id, input_dna_length, dna_entropy_global, alphabet_sizes, features)

Unified DNA-to-structural-state pipeline result.

This is the new format that eliminates redundancy by using a single dictionary of features keyed by orf_id, instead of separate parallel lists for orfs, proteins, and three_dis.

Variables:

• schema_version (str) – Version of the output schema (for compatibility tracking)

• input_id (str) – ID of the input DNA sequence

• input_dna_length (int) – Length of the input DNA sequence

• dna_entropy_global (float) – Entropy of the entire input DNA sequence

• alphabet_sizes (Dict[str, int]) – Dictionary with alphabet sizes for each representation

• features (Dict[str, genome_entropy.pipeline.types.UnifiedFeature]) – Dictionary mapping orf_id to UnifiedFeature objects

Parameters:

• schema_version (str)

• input_id (str)

• input_dna_length (int)

• dna_entropy_global (float)

• alphabet_sizes (Dict[str, int])

• features (Dict[str, UnifiedFeature])

schema_version: str

input_id: str

input_dna_length: int

dna_entropy_global: float

alphabet_sizes: Dict[str, int]

features: Dict[str, UnifiedFeature]

__init__(schema_version, input_id, input_dna_length, dna_entropy_global, alphabet_sizes, features)

Parameters:

• schema_version (str)

• input_id (str)

• input_dna_length (int)

• dna_entropy_global (float)

• alphabet_sizes (Dict[str, int])

• features (Dict[str, UnifiedFeature])

Return type:

None

I/O

FASTA file reading and writing utilities.

genome_entropy.io.fasta.read_fasta(fasta_path)

Read a FASTA file and return a dictionary of sequence_id -> sequence.

Automatically detects and handles gzipped files (ending in .gz).

Parameters:

fasta_path (str | Path) – Path to FASTA file (plain text or gzipped)

Returns:

Dictionary mapping sequence IDs to sequences

Raises:

• FileNotFoundError – If the FASTA file doesn’t exist

• ValueError – If the FASTA file is malformed

Return type:

Dict[str, str]

genome_entropy.io.fasta.read_fasta_iter(fasta_path)

Read a FASTA file and yield (sequence_id, sequence) tuples.

Memory-efficient iterator for large FASTA files. Automatically detects and handles gzipped files (ending in .gz).

Parameters:

fasta_path (str | Path) – Path to FASTA file (plain text or gzipped)

Yields:

Tuples of (sequence_id, sequence)

Raises:

• FileNotFoundError – If the FASTA file doesn’t exist

• ValueError – If the FASTA file is malformed

Return type:

Iterator[Tuple[str, str]]

genome_entropy.io.fasta.write_fasta(sequences, output_path, line_width=80)

Write sequences to a FASTA file.

Automatically compresses output if filename ends with .gz.

Parameters:

• sequences (Dict[str, str]) – Dictionary mapping sequence IDs to sequences

• output_path (str | Path) – Path to output FASTA file (plain text or .gz for compressed)

• line_width (int) – Maximum line width for sequence lines (default: 80)

Return type:

None

GenBank file reading and parsing utilities.

class genome_entropy.io.genbank.CodingInterval(start, end, strand)

A coding interval in zero-based, half-open genomic coordinates.

Parameters:

• start (int)

• end (int)

• strand (Literal['+', '-'])

start: int

end: int

strand: Literal['+', '-']

property length: int

Return the genomic interval length in nucleotides.

__init__(start, end, strand)

Parameters:

• start (int)

• end (int)

• strand (Literal['+', '-'])

Return type:

None

class genome_entropy.io.genbank.CdsMatchResult(matched, overlap_nt=0, overlap_fraction=0.0, compared_aa=0, compatible_aa=0, wildcard_aa=0, identity=0.0, phase_compatible=False, reason='')

Diagnostics from one coordinate-anchored ORF/CDS comparison.

Parameters:

• matched (bool)

• overlap_nt (int)

• overlap_fraction (float)

• compared_aa (int)

• compatible_aa (int)

• wildcard_aa (int)

• identity (float)

• phase_compatible (bool)

• reason (str)

matched: bool

overlap_nt: int = 0

overlap_fraction: float = 0.0

compared_aa: int = 0

compatible_aa: int = 0

wildcard_aa: int = 0

identity: float = 0.0

phase_compatible: bool = False

reason: str = ''

__init__(matched, overlap_nt=0, overlap_fraction=0.0, compared_aa=0, compatible_aa=0, wildcard_aa=0, identity=0.0, phase_compatible=False, reason='')

Parameters:

• matched (bool)

• overlap_nt (int)

• overlap_fraction (float)

• compared_aa (int)

• compatible_aa (int)

• wildcard_aa (int)

• identity (float)

• phase_compatible (bool)

• reason (str)

Return type:

None

class genome_entropy.io.genbank.GenBankCDS(parent_id, start, end, strand, protein_sequence, record_length=None, feature_id='', translation_table=11, codon_start=1, partial=False, skip_reason='')

Represents a CDS (Coding Sequence) feature from GenBank.

Variables:

• parent_id (str) – ID of the parent sequence

• start (int) – 0-based start position (inclusive)

• end (int) – 0-based end position (exclusive)

• strand (Literal['+', '-']) – Strand orientation (‘+’ or ‘-‘)

• protein_sequence (str) – Translated protein sequence

• record_length (int | None) – Length of the parent sequence, needed to convert reverse-complement ORF coordinates to genomic coordinates

• feature_id (str) – Stable CDS identifier used in diagnostics

• translation_table (int) – NCBI genetic code used by this CDS

• codon_start (int) – One-based offset of the first complete CDS codon

• partial (bool) – Whether either Biopython location boundary is partial

• skip_reason (str) – Why this feature cannot safely be matched, if applicable

Parameters:

• parent_id (str)

• start (int)

• end (int)

• strand (Literal['+', '-'])

• protein_sequence (str)

• record_length (int | None)

• feature_id (str)

• translation_table (int)

• codon_start (int)

• partial (bool)

• skip_reason (str)

parent_id: str

start: int

end: int

strand: Literal['+', '-']

protein_sequence: str

record_length: int | None = None

feature_id: str = ''

translation_table: int = 11

codon_start: int = 1

partial: bool = False

skip_reason: str = ''

__init__(parent_id, start, end, strand, protein_sequence, record_length=None, feature_id='', translation_table=11, codon_start=1, partial=False, skip_reason='')

Parameters:

• parent_id (str)

• start (int)

• end (int)

• strand (Literal['+', '-'])

• protein_sequence (str)

• record_length (int | None)

• feature_id (str)

• translation_table (int)

• codon_start (int)

• partial (bool)

• skip_reason (str)

Return type:

None

genome_entropy.io.genbank.read_genbank(genbank_path)

Read a GenBank file and return a dictionary of sequence_id -> DNA sequence.

Automatically detects and handles gzipped files (ending in .gz).

Parameters:

genbank_path (str | Path) – Path to GenBank file (plain text or gzipped)

Returns:

Dictionary mapping sequence IDs to DNA sequences

Raises:

• FileNotFoundError – If the GenBank file doesn’t exist

• ValueError – If the GenBank file is malformed

Return type:

Dict[str, str]

genome_entropy.io.genbank.extract_cds_features(genbank_path, pipeline_table_id=11)

Extract CDS features from a GenBank file.

Automatically detects and handles gzipped files (ending in .gz).

Parameters:

• genbank_path (str | Path) – Path to GenBank file (plain text or gzipped)

• pipeline_table_id (int)

Returns:

List of GenBankCDS objects

Raises:

• FileNotFoundError – If the GenBank file doesn’t exist

• ValueError – If the GenBank file is malformed

Return type:

List[GenBankCDS]

genome_entropy.io.genbank.normalise_protein_sequence(sequence)

Normalise a protein for GenBank matching.

Whitespace is removed, residues are upper-cased, and one terminal stop marker is stripped. An internal stop marker makes the sequence invalid for matching and is represented by an empty result.

Parameters:

sequence (str)

Return type:

str

genome_entropy.io.genbank.amino_acids_are_compatible(residue_a, residue_b)

Return whether two aligned protein residues are compatible.

Equal valid residues match. X is an unknown-residue wildcard, but the more specific ambiguity symbols B, Z, and J are not themselves wildcards. U and O are also treated as specific residues.

Parameters:

• residue_a (str)

• residue_b (str)

Return type:

bool

genome_entropy.io.genbank.normalise_orf_coordinates(orf, record_length)

Convert get_orfs one-based inclusive coordinates to genomic coordinates.

Positive-strand coordinates index the source sequence. Negative-strand coordinates index its reverse complement and therefore require the parent record length to map them back to the genomic axis.

Parameters:

• orf (OrfRecord)

• record_length (int | None)

Return type:

CodingInterval

genome_entropy.io.genbank.normalise_genbank_coordinates(cds)

Return a CDS’s already-normalised Biopython genomic interval.

Parameters:

cds (GenBankCDS)

Return type:

CodingInterval

genome_entropy.io.genbank.coding_phase_is_compatible(orf_interval, cds_interval)

Return whether biological translation starts share a codon phase.

Parameters:

• orf_interval (CodingInterval)

• cds_interval (CodingInterval)

Return type:

bool

genome_entropy.io.genbank.calculate_interval_overlap(first, second)

Return overlap length and its fraction of the shorter interval.

Parameters:

• first (CodingInterval)

• second (CodingInterval)

Return type:

tuple[int, float]

genome_entropy.io.genbank.compare_shared_translation(orf_sequence, cds_sequence, orf_offset, cds_offset, shared_codons)

Compare coordinate-aligned amino acids without gaps or local alignment.

Parameters:

• orf_sequence (str)

• cds_sequence (str)

• orf_offset (int)

• cds_offset (int)

• shared_codons (int)

Return type:

CdsMatchResult

genome_entropy.io.genbank.evaluate_orf_genbank_cds_match(orf, cds)

Evaluate one genomic, strand, phase, overlap, and translation match.

Parameters:

• orf (OrfRecord)

• cds (GenBankCDS)

Return type:

CdsMatchResult

genome_entropy.io.genbank.orf_matches_genbank_cds(orf, cds)

Return whether an ORF and CDS represent the same coordinate-anchored gene.

Parameters:

• orf (OrfRecord)

• cds (GenBankCDS)

Return type:

bool

genome_entropy.io.genbank.match_orf_to_genbank_cds(orf, genbank_cds_list)

Return whether an ORF represents any annotated GenBank CDS.

Parameters:

• orf (OrfRecord)

• genbank_cds_list (List[GenBankCDS])

Return type:

bool

JSON serialization for data models.

genome_entropy.io.jsonio.to_json_dict(obj)

Convert a dataclass object to a JSON-serializable dictionary.

Recursively handles nested dataclasses, lists, and dictionaries.

Parameters:

obj (Any) – Object to convert (typically a dataclass instance)

Returns:

JSON-serializable dictionary

Return type:

Any

genome_entropy.io.jsonio.convert_pipeline_result_to_unified(pipeline_result)

Convert pipeline results to schema-versioned unified records.

Each ORF becomes one feature containing its location, DNA, protein, 3Di, optional 12-state representation, metadata, and raw entropy values. This removes the duplicated objects used by the legacy parallel-list format.

Parameters:

pipeline_result – A PipelineResult or a list of pipeline results.

Returns:

A UnifiedPipelineResult, or a list of unified results when the input is a list.

genome_entropy.io.jsonio.write_json(data, output_path, indent=2)

Write data to a JSON file.

Automatically handles dataclass objects by converting them to dictionaries. If data contains PipelineResult objects, they are automatically converted to the new unified format to eliminate redundancy. Automatically compresses output if filename ends with .gz.

AUTOMATIC CONVERSION:

This function transparently converts old-format PipelineResult objects to the new unified format. This means:

1. Users don’t need to manually call convert_pipeline_result_to_unified()

2. All JSON output from the pipeline automatically uses the new format

3. The conversion happens only once during serialization

4. No changes needed to pipeline code or user scripts

MAPPING: Old Keys → New Structure

OLD FORMAT:

• orfs[i].orf_id → features[orf_id].orf_id

• orfs[i].start → features[orf_id].location.start

• orfs[i].nt_sequence → features[orf_id].dna.nt_sequence

• proteins[i].aa_sequence → features[orf_id].protein.aa_sequence

• three_dis[i].three_di → features[orf_id].three_di.encoding

• entropy.orf_nt_entropy[id] → features[id].entropy.dna_entropy

NEW FORMAT adds:

• schema_version: “2.1.0” (for compatibility tracking)

• features: dict (replaces orfs, proteins, three_dis lists)

• Hierarchical organization (location, dna, protein, three_di, metadata, entropy)

param data:

Data to write (dataclass, dict, list, etc.)

param output_path:

Path to output JSON file (plain text or .gz for compressed)

param indent:

Indentation level for pretty printing (default: 2)

Parameters:

• data (Any)

• output_path (str | Path)

• indent (int)

Return type:

None

genome_entropy.io.jsonio.read_json(input_path)

Read JSON data from a file.

Automatically detects and handles gzipped files (ending in .gz).

Parameters:

input_path (str | Path) – Path to input JSON file (plain text or gzipped)

Returns:

Parsed JSON data (dict, list, etc.)

Raises:

• FileNotFoundError – If the JSON file doesn’t exist

• json.JSONDecodeError – If the file contains invalid JSON

Return type:

Any

Machine learning

Install the ml extra before importing these modules.

Main classifier for predicting GenBank ORF annotations.

This module provides functionality to train machine learning models that predict whether an ORF was annotated in the original GenBank file (in_genbank: True/False) based on various sequence features including entropy values, length, position, etc.

genome_entropy.ml.classifier.load_json_data(json_dir)

Load all JSON files from a directory.

Handles both old PipelineResult format and new unified format. Automatically handles gzipped JSON files (ending in .gz).

Parameters:

json_dir (Path) – Directory containing JSON output files

Returns:

List of lists of parsed JSON data (each file’s content wrapped in a list)

Raises:

ValueError – If no JSON files found or if files are invalid

Return type:

List[List[Dict[str, Any]]]

genome_entropy.ml.classifier.load_json_file(json_file)

Load records from one pipeline JSON file as independent groups.

Keeping each top-level record separate allows train/test splitting by genome or sequence instead of mixing ORFs from one record across splits.

Parameters:

json_file (Path)

Return type:

List[List[Dict[str, Any]]]

genome_entropy.ml.classifier.split_json_records(json_data, test_split=0.1, random_seed=42)

Split top-level JSON records into reproducible train and test groups.

Parameters:

• json_data (List[List[Dict[str, Any]]])

• test_split (float)

• random_seed (int)

Return type:

Tuple[List[List[Dict[str, Any]]], List[List[Dict[str, Any]]]]

genome_entropy.ml.classifier.extract_features(json_data, include_sequences=False, return_metadata=False)

Extract features and labels from JSON data.

Extracts numerical and categorical features from the unified JSON format to predict the in_genbank boolean target.

Features extracted: - Numerical: dna_entropy, protein_entropy, three_di_entropy - Numerical: dna_length, protein_length, three_di_length - Numerical: start, end (genomic position) - Categorical (encoded): strand (+/-), frame (0-3) - Boolean (encoded): has_start_codon, has_stop_codon

Parameters:

• json_data (List[List[Dict[str, Any]]]) – List of lists of parsed JSON dictionaries from load_json_data()

• include_sequences (bool) – If True, include sequence-based features (default: False) This can make feature vectors very large

• return_metadata (bool) – If True, return metadata for each ORF including orf_id and actual in_genbank value (default: False)

Returns:

• features: numpy array of shape (n_samples, n_features)

• labels: numpy array of shape (n_samples,) with 0/1 labels

• feature_names: list of feature names in order

• metadata: list of dicts with orf_id and in_genbank (if return_metadata=True), else None

Return type:

Tuple of (features, labels, feature_names, metadata)

Raises:

ValueError – If data format is invalid or no features found

genome_entropy.ml.classifier.filter_json_records_with_features(json_data)

Return only record groups containing at least one extractable ORF.

Parameters:

json_data (List[List[Dict[str, Any]]])

Return type:

List[List[Dict[str, Any]]]

class genome_entropy.ml.classifier.GenbankClassifier(model_type='xgboost', device=None, **model_kwargs)

Machine learning classifier for predicting GenBank ORF annotations.

This classifier trains a model to predict whether an ORF was annotated in the original GenBank file based on various sequence features.

Supports multiple model types: - “xgboost”: Gradient boosted trees (default, recommended) - “neural_net”: Simple neural network using PyTorch

Example

>>> classifier = GenbankClassifier(model_type="xgboost")
>>> data = load_json_data(Path("results/"))
>>> X, y, feature_names = extract_features(data)
>>> classifier.fit(X, y, feature_names)
>>> metrics = classifier.evaluate(X, y)
>>> print(f"Accuracy: {metrics['accuracy']:.3f}")

Parameters:

• model_type (str)

• device (str | None)

• model_kwargs (Any)

__init__(model_type='xgboost', device=None, **model_kwargs)

Initialize the classifier.

Parameters:

• model_type (str) – Type of model to use (“xgboost” or “neural_net”)

• device (str | None) – Device for computation (None for auto-detect, “cuda”, “cpu”)

• **model_kwargs (Any) – Additional arguments passed to the model

fit(X, y, feature_names=None, validation_split=0.2)

Train the classifier on the provided data.

Parameters:

• X (ndarray) – Feature matrix of shape (n_samples, n_features)

• y (ndarray) – Label array of shape (n_samples,)

• feature_names (List[str] | None) – Optional list of feature names

• validation_split (float) – Fraction of data to use for validation

Returns:

Dictionary with training metrics

Return type:

Dict[str, Any]

predict(X)

Make predictions on new data.

Parameters:

X (ndarray) – Feature matrix of shape (n_samples, n_features)

Returns:

Predicted labels (0 or 1)

Return type:

ndarray

predict_proba(X)

Predict class probabilities.

Parameters:

X (ndarray) – Feature matrix of shape (n_samples, n_features)

Returns:

Predicted probabilities of shape (n_samples, 2)

Return type:

ndarray

evaluate(X, y)

Evaluate the model on test data.

Parameters:

• X (ndarray) – Feature matrix of shape (n_samples, n_features)

• y (ndarray) – True labels

Returns:

Dictionary with evaluation metrics

Return type:

Dict[str, float]

get_feature_importance()

Get feature importance scores.

Returns:

Dictionary mapping feature names to importance scores, or None if model doesn’t support feature importance

Return type:

Dict[str, float] | None

save(path)

Save the trained model to disk.

Parameters:

path (Path) – Path to save the model

Return type:

None

load(path)

Load a trained model from disk.

Parameters:

path (Path) – Path to the saved model

Return type:

None

File-based train/test splitting for ML classifier.

This module provides functionality to randomly split JSON files into training and test sets, train a classifier on the training set, and evaluate on the test set.

genome_entropy.ml.file_split.split_json_files(directory, train_ratio=0.8, random_seed=42)

Split JSON files in directory into train and test sets.

Handles both .json and .json.gz files.

Parameters:

• directory (Path) – Path to directory containing JSON files

• train_ratio (float) – Fraction of files to use for training (default: 0.8)

• random_seed (int) – Random seed for reproducible splits

Returns:

Tuple of (train_files, test_files) as lists of Path objects

Raises:

ValueError – If no JSON files found, invalid train_ratio, or insufficient files for splitting (need at least 2)

Return type:

Tuple[List[Path], List[Path]]

genome_entropy.ml.file_split.load_json_files(file_list)

Load JSON data from a list of files.

Automatically handles gzipped JSON files (ending in .gz).

Parameters:

file_list (List[Path]) – List of paths to JSON files

Returns:

List of lists of parsed JSON data (same format as load_json_data)

Raises:

ValueError – If no valid JSON files could be loaded

Return type:

List[List[Dict[str, Any]]]

genome_entropy.ml.file_split.train_with_file_split(split_dir, output, model_type='xgboost', device=None, validation_split=0.2, random_seed=42, json_output=None)

Train classifier with file-based train/test split.

This function: 1. Randomly splits JSON files in directory 80/20 2. Trains classifier on training files 3. Evaluates on test files 4. Returns/saves detailed results

Parameters:

• split_dir (Path) – Directory containing JSON files to split

• output (Path) – Path to save trained model

• model_type (str) – “xgboost” or “neural_net”

• device (str | None) – Device for training (None for auto-detect)

• validation_split (float) – Fraction of training data for validation

• random_seed (int) – Random seed for reproducible splits

• json_output (Path | None) – Optional path to save detailed JSON report

Returns:

Dictionary with training results, test results, and file lists

Return type:

Dict[str, Any]

Machine learning model implementations.

This module provides wrapper classes for different ML model types that can be used for predicting GenBank annotations.

class genome_entropy.ml.models.BaseModel

Bases: ABC

Abstract base class for ML models.

abstractmethod fit(X, y, validation_split=0.2)

Train the model.

Parameters:

• X (ndarray)

• y (ndarray)

• validation_split (float)

Return type:

Dict[str, Any]

abstractmethod predict(X)

Make predictions.

Parameters:

X (ndarray)

Return type:

ndarray

abstractmethod predict_proba(X)

Predict class probabilities.

Parameters:

X (ndarray)

Return type:

ndarray

abstractmethod evaluate(X, y)

Evaluate the model.

Parameters:

• X (ndarray)

• y (ndarray)

Return type:

Dict[str, float]

abstractmethod save(path)

Save the model.

Parameters:

path (Path)

Return type:

None

abstractmethod load(path)

Load the model.

Parameters:

path (Path)

Return type:

None

get_feature_importance()

Get feature importance. Returns None if not supported.

Return type:

ndarray | None

class genome_entropy.ml.models.XGBoostModel(device=None, n_estimators=100, max_depth=6, learning_rate=0.1, **kwargs)

Bases: BaseModel

XGBoost gradient boosted tree classifier.

Uses xgboost.train() with binary logistic output and histogram trees; it is not an XGBRFClassifier random forest. GPU training requires a CUDA-enabled XGBoost build. Auto-detection uses PyTorch visibility and can therefore select cuda even when XGBoost lacks a compatible backend.

Parameters:

• device (str | None)

• n_estimators (int)

• max_depth (int)

• learning_rate (float)

• kwargs (Any)

__init__(device=None, n_estimators=100, max_depth=6, learning_rate=0.1, **kwargs)

Initialize XGBoost model.

Parameters:

• device (str | None) – "cuda", "cpu", or None for PyTorch-based auto-detection. Pass "cpu" when XGBoost GPU support is not independently available.

• n_estimators (int) – Number of boosting rounds

• max_depth (int) – Maximum tree depth

• learning_rate (float) – Learning rate (eta)

• **kwargs (Any) – Additional XGBoost parameters

fit(X, y, validation_split=0.2)

Train XGBoost model.

Parameters:

• X (ndarray) – Feature matrix

• y (ndarray) – Labels

• validation_split (float) – Fraction for validation

Returns:

Dictionary with training metrics

Return type:

Dict[str, Any]

predict(X)

Make predictions.

Parameters:

X (ndarray) – Feature matrix

Returns:

Predicted labels (0 or 1)

Return type:

ndarray

predict_proba(X)

Predict class probabilities.

Parameters:

X (ndarray) – Feature matrix

Returns:

Probabilities of shape (n_samples, 2)

Return type:

ndarray

evaluate(X, y)

Evaluate the model.

Parameters:

• X (ndarray) – Feature matrix

• y (ndarray) – True labels

Returns:

Dictionary with metrics

Return type:

Dict[str, float]

get_feature_importance()

Return normalised gain importance in feature-index order.

Returns:

Array summing to one when any split has non-zero gain, or None before training. Importance is associative, not causal.

Return type:

ndarray | None

save(path)

Save model to disk.

Parameters:

path (Path) – Path to save model

Return type:

None

load(path)

Load model from disk.

Parameters:

path (Path) – Path to saved model

Return type:

None

class genome_entropy.ml.models.NeuralNetModel(input_dim, device=None, hidden_dim=64, dropout=0.3, learning_rate=0.001, epochs=100, batch_size=32)

Bases: BaseModel

Simple neural network classifier using PyTorch.

Alternative to XGBoost. Uses a simple feedforward network with: - 2 hidden layers with ReLU activation - Dropout for regularization - Binary cross-entropy loss - GPU support via PyTorch

Generally less suitable than XGBoost for this task because: - Requires more data and careful tuning - Less interpretable (no feature importance) - More prone to overfitting on small datasets

However, it provides GPU acceleration and can model complex non-linear relationships if sufficient data is available.

Parameters:

• input_dim (int)

• device (str | None)

• hidden_dim (int)

• dropout (float)

• learning_rate (float)

• epochs (int)

• batch_size (int)

__init__(input_dim, device=None, hidden_dim=64, dropout=0.3, learning_rate=0.001, epochs=100, batch_size=32)

Initialize neural network model.

Parameters:

• input_dim (int) – Number of input features

• device (str | None) – Device to use (None for auto-detect)

• hidden_dim (int) – Hidden layer dimension

• dropout (float) – Dropout rate

• learning_rate (float) – Learning rate

• epochs (int) – Number of training epochs

• batch_size (int) – Batch size for training

fit(X, y, validation_split=0.2)

Train neural network.

Parameters:

• X (ndarray) – Feature matrix

• y (ndarray) – Labels

• validation_split (float) – Fraction for validation

Returns:

Dictionary with training metrics

Return type:

Dict[str, Any]

predict(X)

Make predictions.

Parameters:

X (ndarray) – Feature matrix

Returns:

Predicted labels (0 or 1)

Return type:

ndarray

predict_proba(X)

Predict class probabilities.

Parameters:

X (ndarray) – Feature matrix

Returns:

Probabilities of shape (n_samples, 2)

Return type:

ndarray

evaluate(X, y)

Evaluate the model.

Parameters:

• X (ndarray) – Feature matrix

• y (ndarray) – True labels

Returns:

Dictionary with metrics

Return type:

Dict[str, float]

save(path)

Save model to disk.

Parameters:

path (Path) – Path to save model

Return type:

None

load(path)

Load model from disk.

Parameters:

path (Path) – Path to saved model

Return type:

None

Configuration, errors, and logging

Configuration, model capabilities, and constants for genome_entropy.

class genome_entropy.config.ModelCapabilities(model_name, family, supports_3di, supports_12st, supports_profiles=False, deprecated=False, description='')

Capabilities and provenance for one supported Hugging Face model.

Parameters:

• model_name (str)

• family (Literal['modernprost_multitask', 'modernprost_legacy', 'prostt5'])

• supports_3di (bool)

• supports_12st (bool)

• supports_profiles (bool)

• deprecated (bool)

• description (str)

model_name: str

family: Literal['modernprost_multitask', 'modernprost_legacy', 'prostt5']

supports_3di: bool

supports_12st: bool

supports_profiles: bool = False

deprecated: bool = False

description: str = ''

__init__(model_name, family, supports_3di, supports_12st, supports_profiles=False, deprecated=False, description='')

Parameters:

• model_name (str)

• family (Literal['modernprost_multitask', 'modernprost_legacy', 'prostt5'])

• supports_3di (bool)

• supports_12st (bool)

• supports_profiles (bool)

• deprecated (bool)

• description (str)

Return type:

None

genome_entropy.config.resolve_model_name(model_name, *, warn=True)

Resolve old repository aliases and validate a supported model identifier.

Parameters:

• model_name (str)

• warn (bool)

Return type:

str

genome_entropy.config.get_model_capabilities(model_name, *, warn=True)

Return central capability metadata for a model or legacy alias.

Parameters:

• model_name (str)

• warn (bool)

Return type:

ModelCapabilities

genome_entropy.config.supported_models_help()

Return CLI help text generated from the central model registry.

Return type:

str

Custom exceptions for genome_entropy.

exception genome_entropy.errors.OrfEntropyError

Bases: Exception

Base exception for genome_entropy package.

exception genome_entropy.errors.ConfigurationError

Bases: OrfEntropyError

Raised when there’s a configuration error.

exception genome_entropy.errors.InputError

Bases: OrfEntropyError

Raised when input data is invalid or cannot be processed.

exception genome_entropy.errors.OrfFinderError

Bases: OrfEntropyError

Raised when ORF finding fails.

exception genome_entropy.errors.TranslationError

Bases: OrfEntropyError

Raised when translation fails.

exception genome_entropy.errors.EncodingError

Bases: OrfEntropyError

Raised when 3Di encoding fails.

exception genome_entropy.errors.ModelError

Bases: OrfEntropyError

Raised when model loading or inference fails.

exception genome_entropy.errors.DeviceError

Bases: OrfEntropyError

Raised when device selection or initialization fails.

exception genome_entropy.errors.PipelineError

Bases: OrfEntropyError

Raised when the pipeline orchestration fails.

Centralized logging configuration for genome_entropy.

This module provides a single source for configuring logging throughout the application. It supports: - Multiple log levels (DEBUG, INFO, WARNING, ERROR, CRITICAL) - Output to file or STDOUT - Consistent format across all modules

genome_entropy.logging_config.configure_logging(level=20, log_file=None, log_format='%(asctime)s - %(name)s - %(levelname)s - %(message)s', date_format='%Y-%m-%d %H:%M:%S', force=False)

Configure logging for the entire application.

This should be called once at application startup (e.g., in CLI main).

Parameters:

• level (int | str) – Logging level (DEBUG, INFO, WARNING, ERROR, CRITICAL) as int or string

• log_file (str | Path | None) – Optional path to log file. If None, logs to STDOUT

• log_format (str) – Format string for log messages

• date_format (str) – Format string for timestamps

• force (bool) – If True, reconfigure even if already configured

Return type:

None

Examples

>>> configure_logging(level=logging.DEBUG, log_file="app.log")
>>> configure_logging(level="INFO")  # Log to STDOUT
>>> configure_logging(level="DEBUG", log_file=None)  # Debug to STDOUT

genome_entropy.logging_config.get_logger(name)

Get a logger instance for a module.

This is the preferred way to get loggers in the application.

Parameters:

name (str) – Name of the logger (usually __name__ of the module)

Returns:

Configured logger instance

Return type:

Logger

Example

>>> logger = get_logger(__name__)
>>> logger.info("Processing started")

genome_entropy.logging_config.is_configured()

Check if logging has been configured.

Returns:

True if configure_logging() has been called

Return type:

bool

genome_entropy.logging_config.get_log_file()

Get the current log file path.

Returns:

Path to log file, or None if logging to STDOUT

Return type:

Path | None

genome_entropy.logging_config.get_log_level()

Get the current logging level.

Returns:

Current logging level as integer

Return type:

int

genome_entropy.logging_config.set_log_level(level)

Change the logging level at runtime.

Parameters:

level (int | str) – New logging level (DEBUG, INFO, WARNING, ERROR, CRITICAL)

Return type:

None

Example

>>> set_log_level("DEBUG")
>>> set_log_level(logging.WARNING)