Tasks

Tasks define data and model usage. They provide a simple interface for swapping backbones, adapters, and data without any code changes, enabling rapid and reproducible experimentation. They are specified with the --model argument in the CLI or in the model section of a configuration file.

Tasks automatically configure backbones and adapters for training with mgen fit, evaluation with mgen test/validate, and inference with mgen predict. They cover a range of use-cases for information extraction, domain adaptation, supervised prediction, generative modeling, and zero-shot applications.

This reference overviews the available no-code tasks for finetuning and inference. If you would like to develop new tasks, see Experiment Design.

# Example Task Configuration
model:
  class_path: SequenceClassification
  init_args:
    backbone:
      class_path: aido_dna_7b
      init_args:
        use_peft: true
        lora_r: 16
        lora_alpha: 32
        lora_dropout: 0.1
    adapter:
      class_path: modelgenerator.adapters.MLPPoolAdapter
      init_args:
        pooling: mean_pooling
        hidden_sizes:
        - 512
        - 256
        bias: true
        dropout: 0.1
        dropout_in_middle: false
    optimizer:
      class_path: torch.optim.AdamW
      init_args:
        lr: 1e-4
    lr_scheduler:
      class_path: torch.optim.lr_scheduler.StepLR
      init_args:
        step_size: 1
        gamma: 0.1
data:
  ...
trainer:
  ...

Note: Adapters and Backbones are typed as Callables, since some args are reserved to be automatically configured within the task. As a general rule, positional arguments are reserved while keyword arguments are free to use. For example, the backbone, adapter, optimizer, and lr_scheduler can be configured as

Extract

`modelgenerator.tasks.Embed`

Bases: TaskInterface

Task for getting embeddings from a pretrained backbone. This task is used only for inference.

Note

Must be used with PredictionWriter. Embeddings are stored under "predictions".

Parameters:

Name	Type	Description	Default
`backbone`	`Callable`	A pretrained backbone from the modelgenerator library.	required
`optimizer`	`Callable`	The optimizer to use for training.	`<class 'torch.optim.adamw.AdamW'>`
`lr_scheduler`	`Optional`	The learning rate scheduler to use for training.	`None`
`batch_size`	`Optional`	The batch size to use for training.	`None`
`use_legacy_adapter`	`bool`	Whether to use the adapter from the backbone (HF head support). Warning: This is not supported for all tasks and will be depreciated in the future.	`False`
`strict_loading`	`bool`	Whether to strictly load the model from the checkpoint. If False, replaces missing weights with pretrained weights. Should be enabled when loading a checkpoint from a run with `use_peft` and `save_peft_only`.	`True`
`reset_optimizer_states`	`bool`	Whether to reset the optimizer states. Set it to True if you want to replace the adapter (e.g. for continued pretraining).	`False`
`data_module`	`Optional`	A data module from the modelgenerator library. Managed argument, do not override. Links `--data` argument to tasks for auto-configuration of backbones, adapters, and losses.	`None`

`modelgenerator.tasks.Inference`

Bases: MLM

Task for performing inference with a pretrained backbone end-to-end, including the backbone's original adapter.

Note

Must be used with PredictionWriter. Model outputs are stored under "predictions".

Parameters:

Name	Type	Description	Default
`backbone`	`Callable`	A pretrained backbone from the modelgenerator library.	required
`use_legacy_adapter`	`bool`	Whether to use the adapter from the backbone (HF head support). Warning: This is not supported for all tasks and will be depreciated in the future.	`True`
`optimizer`	`Callable`	The optimizer to use for training.	`<class 'torch.optim.adamw.AdamW'>`
`lr_scheduler`	`Optional`	The learning rate scheduler to use for training.	`None`
`batch_size`	`Optional`	The batch size to use for training.	`None`
`strict_loading`	`bool`	Whether to strictly load the model from the checkpoint. If False, replaces missing weights with pretrained weights. Should be enabled when loading a checkpoint from a run with `use_peft` and `save_peft_only`.	`True`
`reset_optimizer_states`	`bool`	Whether to reset the optimizer states. Set it to True if you want to replace the adapter (e.g. for continued pretraining).	`False`
`data_module`	`Optional`	A data module from the modelgenerator library. Managed argument, do not override. Links `--data` argument to tasks for auto-configuration of backbones, adapters, and losses.	`None`

Adapt

`modelgenerator.tasks.MLM`

Bases: TaskInterface

Task for performing masked language modeling (MLM) with a pretrained backbone. Can be used to train from scratch or for domain adaptation. Uses the MLMDataModule. Evaluates in terms of reconstruction accuracy on all tokens and cross-entropy loss.

Parameters:

Name	Type	Description	Default
`backbone`	`Callable`	A pretrained backbone from the modelgenerator library.	required
`use_legacy_adapter`	`bool`	Whether to use the adapter from the backbone (HF head support). Warning: This is not supported for all tasks and will be depreciated in the future.	`True`
`optimizer`	`Callable`	The optimizer to use for training.	`<class 'torch.optim.adamw.AdamW'>`
`lr_scheduler`	`Optional`	The learning rate scheduler to use for training.	`None`
`batch_size`	`Optional`	The batch size to use for training.	`None`
`strict_loading`	`bool`	Whether to strictly load the model from the checkpoint. If False, replaces missing weights with pretrained weights. Should be enabled when loading a checkpoint from a run with `use_peft` and `save_peft_only`.	`True`
`reset_optimizer_states`	`bool`	Whether to reset the optimizer states. Set it to True if you want to replace the adapter (e.g. for continued pretraining).	`False`
`data_module`	`Optional`	A data module from the modelgenerator library. Managed argument, do not override. Links `--data` argument to tasks for auto-configuration of backbones, adapters, and losses.	`None`

`modelgenerator.tasks.ConditionalMLM`

Bases: TaskInterface

Task for masked language modeling with extra condition inputs. Evaluates in terms of reconstruction accuracy on all tokens and cross-entropy loss.

Parameters:

Name	Type	Description	Default
`backbone`	`Callable`	A pretrained backbone from the modelgenerator library.	required
`adapter`	`Optional`	A ConditionalGenerationAdapter for the model.	`<class 'modelgenerator.adapters.adapters.ConditionalLMAdapter'>`
`use_legacy_adapter`	`bool`	Whether to use the pre-trained legacy adapter within the conditional decoder.	`True`
`condition_dim`	`int`	The dimension of the condition.	`1`
`optimizer`	`Callable`	The optimizer to use for training.	`<class 'torch.optim.adamw.AdamW'>`
`lr_scheduler`	`Optional`	The learning rate scheduler to use for training.	`None`
`batch_size`	`Optional`	The batch size to use for training.	`None`
`strict_loading`	`bool`	Whether to strictly load the model from the checkpoint. If False, replaces missing weights with pretrained weights. Should be enabled when loading a checkpoint from a run with `use_peft` and `save_peft_only`.	`True`
`reset_optimizer_states`	`bool`	Whether to reset the optimizer states. Set it to True if you want to replace the adapter (e.g. for continued pretraining).	`False`
`data_module`	`Optional`	A data module from the modelgenerator library. Managed argument, do not override. Links `--data` argument to tasks for auto-configuration of backbones, adapters, and losses.	`None`

Predict

`modelgenerator.tasks.SequenceClassification`

Bases: TaskInterface

Task for fine-tuning a sequence model for classification. Evaluates in terms of accuracy, F1 score, Matthews correlation coefficient (MCC), and AUROC.

Note

Supports binary, multiclass, and binary multi-label classification tasks.

Parameters:

Name	Type	Description	Default
`backbone`	`Callable`	A pretrained backbone from the modelgenerator library.	required
`adapter`	`Optional`	A SequenceAdapter for the model.	`<class 'modelgenerator.adapters.adapters.LinearCLSAdapter'>`
`n_classes`	`int`	The number of classes in the classification task.	`2`
`multilabel`	`bool`	Indicate whether multiple labels can be positive, turning this into a multi-way binary classification task.	`False`
`weighted_loss`	`bool`	The description is missing.	`False`
`optimizer`	`Callable`	The optimizer to use for training.	`<class 'torch.optim.adamw.AdamW'>`
`lr_scheduler`	`Optional`	The learning rate scheduler to use for training.	`None`
`batch_size`	`Optional`	The batch size to use for training.	`None`
`use_legacy_adapter`	`bool`	Whether to use the adapter from the backbone (HF head support). Warning: This is not supported for all tasks and will be depreciated in the future.	`False`
`strict_loading`	`bool`	Whether to strictly load the model from the checkpoint. If False, replaces missing weights with pretrained weights. Should be enabled when loading a checkpoint from a run with `use_peft` and `save_peft_only`.	`True`
`reset_optimizer_states`	`bool`	Whether to reset the optimizer states. Set it to True if you want to replace the adapter (e.g. for continued pretraining).	`False`
`data_module`	`Optional`	A data module from the modelgenerator library. Managed argument, do not override. Links `--data` argument to tasks for auto-configuration of backbones, adapters, and losses.	`None`

`modelgenerator.tasks.TokenClassification`

Bases: SequenceClassification

Task for fine-tuning a model for token-wise classification. Evaluates in terms of accuracy, F1 score, Matthews correlation coefficient (MCC), and AUROC.

Parameters:

Name	Type	Description	Default
`backbone`	`Callable`	A pretrained backbone from the modelgenerator library.	required
`adapter`	`Optional`	A TokenAdapter for the model.	`<class 'modelgenerator.adapters.adapters.LinearAdapter'>`
`n_classes`	`int`	The number of classes in the classification task.	`2`
`multilabel`	`bool`	Indicate whether multiple labels can be positive, turning this into a multi-way binary classification task.	`False`
`weighted_loss`	`bool`	The description is missing.	`False`
`optimizer`	`Callable`	The optimizer to use for training.	`<class 'torch.optim.adamw.AdamW'>`
`lr_scheduler`	`Optional`	The learning rate scheduler to use for training.	`None`
`batch_size`	`Optional`	The batch size to use for training.	`None`
`use_legacy_adapter`	`bool`	Whether to use the adapter from the backbone (HF head support). Warning: This is not supported for all tasks and will be depreciated in the future.	`False`
`strict_loading`	`bool`	Whether to strictly load the model from the checkpoint. If False, replaces missing weights with pretrained weights. Should be enabled when loading a checkpoint from a run with `use_peft` and `save_peft_only`.	`True`
`reset_optimizer_states`	`bool`	Whether to reset the optimizer states. Set it to True if you want to replace the adapter (e.g. for continued pretraining).	`False`
`data_module`	`Optional`	A data module from the modelgenerator library. Managed argument, do not override. Links `--data` argument to tasks for auto-configuration of backbones, adapters, and losses.	`None`

Attributes:

Name	Type	Description
`legacy_adapter_type`	`LegacyAdapterType`	The LegacyAdapterType.TOKEN_CLS legacy adapter.

`modelgenerator.tasks.PairwiseTokenClassification`

Bases: SequenceClassification

Task for fine-tuning a model for pairwise token classification. Evaluates in terms of accuracy, F1 score, Matthews correlation coefficient (MCC), AUROC, and top-k accuracy for k=2,5,10.

Parameters:

Name	Type	Description	Default
`backbone`	`Callable`	A pretrained backbone from the modelgenerator library.	required
`adapter`	`Optional`	A TokenAdapter for the model.	`<class 'modelgenerator.adapters.adapters.LinearAdapter'>`
`adapter_dim_multiplier`	`int`	The multiplier for the adapter dimension.	`2`
`n_classes`	`int`	The number of classes in the classification task.	`2`
`multilabel`	`bool`	Indicate whether multiple labels can be positive, turning this into a multi-way binary classification task.	`False`
`weighted_loss`	`bool`	The description is missing.	`False`
`optimizer`	`Callable`	The optimizer to use for training.	`<class 'torch.optim.adamw.AdamW'>`
`lr_scheduler`	`Optional`	The learning rate scheduler to use for training.	`None`
`batch_size`	`Optional`	The batch size to use for training.	`None`
`use_legacy_adapter`	`bool`	Whether to use the adapter from the backbone (HF head support). Warning: This is not supported for all tasks and will be depreciated in the future.	`False`
`strict_loading`	`bool`	Whether to strictly load the model from the checkpoint. If False, replaces missing weights with pretrained weights. Should be enabled when loading a checkpoint from a run with `use_peft` and `save_peft_only`.	`True`
`reset_optimizer_states`	`bool`	Whether to reset the optimizer states. Set it to True if you want to replace the adapter (e.g. for continued pretraining).	`False`
`data_module`	`Optional`	A data module from the modelgenerator library. Managed argument, do not override. Links `--data` argument to tasks for auto-configuration of backbones, adapters, and losses.	`None`

Attributes:

Name	Type	Description
`legacy_adapter_type`	`LegacyAdapterType`	The LegacyAdapterType.TOKEN_CLS legacy adapter.

`modelgenerator.tasks.MMSequenceRegression`

Bases: TaskInterface

Task for fine-tuning multiple models on single-/multi-task regression. Evaluates in terms of mean absolute error, mean squared error, R2 score, Pearson correlation, and Spearman correlation.

Note

Supports any combination of DNA, RNA and protein backbones Requires data column names to be prefixed by backbone_order list. For example, if backbone_order is ["dna_seq", "rna_seq"], the sequence columns should be named "dna_seq_sequences" and "rna_seq_sequences". Please use rename_cols in the data module to rename the columns if necessary.

Parameters:

Name	Type	Description	Default
`backbone`	`Callable`	A pretrained backbone from the modelgenerator library.	required
`backbone1`	`Callable`	A second pretrained backbone from the modelgenerator library.	required
`backbone2`	`Optional`	An optional third pretrained backbone from the modelgenerator library.	`None`
`backbone_order`	`list`	A list of data columns in order of the backbones.	`['dna_seq', 'rna_seq']`
`adapter`	`Optional`	A callable that returns a FusionAdapter.	`<class 'modelgenerator.adapters.fusion.MMFusionTokenAdapter'>`
`num_outputs`	`int`	The number of outputs for the regression task.	`1`
`loss_func`	`Callable`	A callable that returns a loss function.	`<class 'torch.nn.modules.loss.MSELoss'>`
`optimizer`	`Callable`	The optimizer to use for training.	`<class 'torch.optim.adamw.AdamW'>`
`lr_scheduler`	`Optional`	The learning rate scheduler to use for training.	`None`
`batch_size`	`Optional`	The batch size to use for training.	`None`
`use_legacy_adapter`	`bool`	Whether to use the adapter from the backbone (HF head support). Warning: This is not supported for all tasks and will be depreciated in the future.	`False`
`strict_loading`	`bool`	Whether to strictly load the model from the checkpoint. If False, replaces missing weights with pretrained weights. Should be enabled when loading a checkpoint from a run with `use_peft` and `save_peft_only`.	`True`
`reset_optimizer_states`	`bool`	Whether to reset the optimizer states. Set it to True if you want to replace the adapter (e.g. for continued pretraining).	`False`
`data_module`	`Optional`	A data module from the modelgenerator library. Managed argument, do not override. Links `--data` argument to tasks for auto-configuration of backbones, adapters, and losses.	`None`

Generate

`modelgenerator.tasks.Diffusion`

Bases: TaskInterface

Masked Diffusion Language Modeling training and generation on sequences. Evaluates in terms of reconstruction accuracy on masked tokens and cross-entropy loss.

Parameters:

Name	Type	Description	Default
`backbone`	`Callable`	A pretrained backbone from the modelgenerator library.	required
`adapter`	`Optional`	A TokenAdapter for the model.	`None`
`use_legacy_adapter`	`bool`	Whether to use the adapter from the backbone (HF head support). Warning: This is not supported for all tasks and will be depreciated in the future.	`True`
`sample_seq`	`bool`	Whether to sample tokens during denoising, instead of always using the most likely token.	`False`
`num_denoise_steps`	`int`	Granularity of the denoising process. Less steps makes fewer forward passes and denoises more aggressively.	`4`
`sampling_temperature`	`float`	The temperature for sampling tokens, if sample_seq is True.	`1.0`
`normalize_posterior_weights`	`bool`	Whether to normalize posterior weights. Experimental feature to help training stability.	`False`
`verbose`	`bool`	Print while denoising (warning: fun to watch).	`False`
`optimizer`	`Callable`	The optimizer to use for training.	`<class 'torch.optim.adamw.AdamW'>`
`lr_scheduler`	`Optional`	The learning rate scheduler to use for training.	`None`
`batch_size`	`Optional`	The batch size to use for training.	`None`
`strict_loading`	`bool`	Whether to strictly load the model from the checkpoint. If False, replaces missing weights with pretrained weights. Should be enabled when loading a checkpoint from a run with `use_peft` and `save_peft_only`.	`True`
`reset_optimizer_states`	`bool`	Whether to reset the optimizer states. Set it to True if you want to replace the adapter (e.g. for continued pretraining).	`False`
`data_module`	`Optional`	A data module from the modelgenerator library. Managed argument, do not override. Links `--data` argument to tasks for auto-configuration of backbones, adapters, and losses.	`None`

`modelgenerator.tasks.ConditionalDiffusion`

Bases: Diffusion

Task for masked diffusion language modeling with extra condition inputs. Evaluates in terms of reconstruction accuracy on masked tokens and cross-entropy loss.

Parameters:

Name	Type	Description	Default
`backbone`	`Callable`	A pretrained backbone from the modelgenerator library.	required
`adapter`	`Optional`	A ConditionalGenerationAdapter for the model.	`<class 'modelgenerator.adapters.adapters.ConditionalLMAdapter'>`
`use_legacy_adapter`	`bool`	Whether to use the pre-trained legacy adapter within the conditional decoder.	`True`
`condition_dim`	`int`	The dimension of the condition.	`1`
`sample_seq`	`bool`	Whether to sample tokens during denoising, instead of always using the most likely token.	`False`
`num_denoise_steps`	`int`	Granularity of the denoising process. Less steps makes fewer forward passes and denoises more aggressively.	`4`
`sampling_temperature`	`float`	The temperature for sampling tokens, if sample_seq is True.	`1.0`
`normalize_posterior_weights`	`bool`	Whether to normalize posterior weights. Experimental feature to help training stability.	`False`
`verbose`	`bool`	Print while denoising (warning: fun to watch).	`False`
`optimizer`	`Callable`	The optimizer to use for training.	`<class 'torch.optim.adamw.AdamW'>`
`lr_scheduler`	`Optional`	The learning rate scheduler to use for training.	`None`
`batch_size`	`Optional`	The batch size to use for training.	`None`
`strict_loading`	`bool`	Whether to strictly load the model from the checkpoint. If False, replaces missing weights with pretrained weights. Should be enabled when loading a checkpoint from a run with `use_peft` and `save_peft_only`.	`True`
`reset_optimizer_states`	`bool`	Whether to reset the optimizer states. Set it to True if you want to replace the adapter (e.g. for continued pretraining).	`False`
`data_module`	`Optional`	A data module from the modelgenerator library. Managed argument, do not override. Links `--data` argument to tasks for auto-configuration of backbones, adapters, and losses.	`None`

Zero-Shot

`modelgenerator.tasks.ZeroshotPredictionDiff`

Bases: TaskInterface

Task for zero-shot prediction with a languange model that produces token logits. Computes the log-likelihood difference between probability of ref and alt at the mutated position. Evaluates in terms of AUROC and AUPRC.

Parameters:

Name	Type	Description	Default
`backbone`	`Callable`	A pretrained backbone from the modelgenerator library.	required
`optimizer`	`Callable`	The optimizer to use for training.	`<class 'torch.optim.adamw.AdamW'>`
`lr_scheduler`	`Optional`	The learning rate scheduler to use for training.	`None`
`batch_size`	`Optional`	The batch size to use for training.	`None`
`use_legacy_adapter`	`bool`	Whether to use the adapter from the backbone (HF head support). Warning: This is not supported for all tasks and will be depreciated in the future.	`False`
`strict_loading`	`bool`	Whether to strictly load the model from the checkpoint. If False, replaces missing weights with pretrained weights. Should be enabled when loading a checkpoint from a run with `use_peft` and `save_peft_only`.	`True`
`reset_optimizer_states`	`bool`	Whether to reset the optimizer states. Set it to True if you want to replace the adapter (e.g. for continued pretraining).	`False`
`data_module`	`Optional`	A data module from the modelgenerator library. Managed argument, do not override. Links `--data` argument to tasks for auto-configuration of backbones, adapters, and losses.	`None`

`modelgenerator.tasks.ZeroshotPredictionDistance`

Bases: TaskInterface

Task for zero-shot prediction with a model that produces embeddings. Computes the L1 and L2 distance between the reference and mutated sequence embeddings. Evaluates in terms of AUROC and AUPRC of the embedding distance.

Parameters:

Name	Type	Description	Default
`backbone`	`Callable`	A pretrained backbone from the modelgenerator library.	required
`all_hidden_states`	`bool`	Whether to run the test on all available hidden layers.	`False`
`shared_ref`	`bool`	Whether to use a shared reference sequence to accelerate zero-shot computation. Uses a separate reference sequence for each mutated sequence by default.	`False`
`optimizer`	`Callable`	The optimizer to use for training.	`<class 'torch.optim.adamw.AdamW'>`
`lr_scheduler`	`Optional`	The learning rate scheduler to use for training.	`None`
`batch_size`	`Optional`	The batch size to use for training.	`None`
`use_legacy_adapter`	`bool`	Whether to use the adapter from the backbone (HF head support). Warning: This is not supported for all tasks and will be depreciated in the future.	`False`
`strict_loading`	`bool`	Whether to strictly load the model from the checkpoint. If False, replaces missing weights with pretrained weights. Should be enabled when loading a checkpoint from a run with `use_peft` and `save_peft_only`.	`True`
`reset_optimizer_states`	`bool`	Whether to reset the optimizer states. Set it to True if you want to replace the adapter (e.g. for continued pretraining).	`False`
`data_module`	`Optional`	A data module from the modelgenerator library. Managed argument, do not override. Links `--data` argument to tasks for auto-configuration of backbones, adapters, and losses.	`None`