Adapters

Adapters work with Backbones and Tasks to adapt pretrained models to new objectives. They are specified with the --model.adapter argument in the CLI or in the model.adapter section of a configuration file.

Adapters are the focal point for architecture design on top of backbones, and can be swapped with other adapters of the same type to benchmark different architectures.

This reference overviews the available no-code adapters. If you would like to develop new adapters, see Experiment Design.

# Example Adapter Configuration
model:
  class_path: modelgenerator.tasks.SequenceRegression
  init_args:
    adapter:
      class_path: modelgenerator.adapters.MLPPoolAdapter
      init_args:
        pooling: mean_pooling
        hidden_sizes:
        - 512
        bias: true
        dropout: 0.1
        dropout_in_middle: false
data:
  ...
trainer:
  ...

Sequence Adapters

These adapters make a single prediction for the entire input.

modelgenerator.adapters.MLPAdapter

Bases: Sequential, TokenAdapter

Multi-layer perceptron (MLP) adapter.

Parameters:

Name	Type	Description	Default
in_features	int	Number of features of the input	required
out_features	int	Number of features of the output	required
hidden_sizes	List[int]	List of the hidden feature dimensions. Defaults to [].	[]
activation_layer	Callable[..., Module]	Activation function. Defaults to torch.nn.Tanh.	Tanh
bias	bool	Whether to use bias in the linear layer. Defaults to True	True
dropout	float	The probability for the dropout layer. Defaults to 0.0	0.0
dropout_in_middle	bool	Whether to use dropout in the middle layers. Defaults to True	True

modelgenerator.adapters.LinearCLSAdapter

Bases: Module, SequenceAdapter

Simple linear adapter for a 1D embedding

Parameters:

Name	Type	Description	Default
in_features	int	Number of input features	required
out_features	int	Number of output features	required

modelgenerator.adapters.LinearMeanPoolAdapter

Bases: Module, SequenceAdapter

Mean pooling adapter for hidden_states of shape (n, seq_len, in_features)

Parameters:

Name	Type	Description	Default
in_features	int	Number of input features	required
out_features	int	Number of output features	required

modelgenerator.adapters.LinearMaxPoolAdapter

Bases: Module, SequenceAdapter

Simple max pooling adapter for [batch,seq_len,dim] embeddings

Parameters:

Name	Type	Description	Default
in_features	int	Number of input features	required
out_features	int	Number of output features	required

modelgenerator.adapters.LinearTransformerAdapter

Bases: Module, SequenceAdapter

Transformer adapter

Note: Support cls_pooling only.

Parameters:

Name	Type	Description	Default
embed_dim	int	Hidden size	required
out_features	int	Number of output features	required

modelgenerator.adapters.ResNet2DAdapter

Bases: Module, SequenceAdapter

Adapter that applies ResNet2DModule to input embeddings.

Parameters:

Name	Type	Description	Default
in_channels	int	Input matrix channels.	required
num_res_blocks	int	Number of residual blocks in the ResNet2DModule.	2
conv_channels	int	Intermediate convolution channels.	64
kernel_size	int	Kernel size for convolutions.	3

modelgenerator.adapters.ResNet1DAdapter

Bases: Module, SequenceAdapter

Adapter module that applies a ResNet1DModule to sequence data.

Parameters:

Name	Type	Description	Default
input_dim	int	Input feature dimension.	required
channels	int	Number of channels for ResNet.	256
num_blocks	int	Number of residual blocks in ResNet1DModule.	9
dropout	float	Dropout rate.	0.2

Token Adapters

These adapters make one prediction per token.

modelgenerator.adapters.LinearAdapter

Bases: MLPAdapter

Simple linear adapter for a 1D embedding

Parameters:

Name	Type	Description	Default
in_features	int	Number of input features	required
out_features	int	Number of output features	required

modelgenerator.adapters.MLPAdapter

Bases: Sequential, TokenAdapter

Multi-layer perceptron (MLP) adapter.

Parameters:

Name	Type	Description	Default
in_features	int	Number of features of the input	required
out_features	int	Number of features of the output	required
hidden_sizes	List[int]	List of the hidden feature dimensions. Defaults to [].	[]
activation_layer	Callable[..., Module]	Activation function. Defaults to torch.nn.Tanh.	Tanh
bias	bool	Whether to use bias in the linear layer. Defaults to True	True
dropout	float	The probability for the dropout layer. Defaults to 0.0	0.0
dropout_in_middle	bool	Whether to use dropout in the middle layers. Defaults to True	True

modelgenerator.adapters.MLPAdapterWithoutOutConcat

Bases: Module, TokenAdapter

Multi-layer perceptron (MLP) adapter without outer concatenate

This class is generally used in PairwiseTokenClassification. The following two implementations are equivalent: 1. hidden_states -> outer_concat -> MLPAdapter 2. hidden_states -> MLPAdapterWithoutOutConcat MLPAdapterWithoutOutConcat avoids the large memory consumption of outer_concat

Parameters:

Name	Type	Description	Default
in_features	int	Number of features of the input	required
out_features	int	Number of features of the output	required
hidden_sizes	List[int]	List of the hidden feature dimensions. Defaults to [].	[]
activation_layer	Callable[..., Module]	Activation function. Defaults to torch.nn.Tanh.	Tanh
bias	bool	Whether to use bias in the linear layer. Defaults to True	True
dropout	float	The probability for the dropout layer. Defaults to 0.0	0.0
dropout_in_middle	bool	Whether to use dropout in the middle layers. Defaults to True	True

Conditional Generation Adapters

These adapters are used for conditional generation tasks.

modelgenerator.adapters.ConditionalLMAdapter

Bases: Module, ConditionalGenerationAdapter

Conditional sequence adapter

Parameters:

Name	Type	Description	Default
in_features	int	Number of input features	required
embed_dim	int	Hidden size	required
seq_len	int	Sequence length	required

Fusion Adapters

These adapters are used for multi-modal fusion to combine multiple backbones.

modelgenerator.adapters.MMFusionSeqAdapter

Bases: Module, FusionAdapter

Multimodal embeddings fusion with SequenceAdapter.

Note

Accepts 2-3 sequence embeddings as input and fuses them into a multimodal embedding for the adapter.

Parameters:

Name	Type	Description	Default
out_features	int	Number of output features.	required
input_size	int	number of input features for the first modality.	required
input_size_1	int	number of input features for the second modality.	required
input_size_2	int	number of input features for the third modality.	None
fusion	Callable[[int, int, int], CrossAttentionFusion]	The callable that returns a fusion module.	CrossAttentionFusion
adapter	Callable[[int, int], SequenceAdapter]	The callable that returns an adapter.	LinearCLSAdapter

modelgenerator.adapters.MMFusionTokenAdapter

Bases: Module, FusionAdapter

Multimodal embeddings fusion with TokenAdapter. Fuses embeddings into a single token embedding.

Note

Accepts 2-3 sequence embeddings as input and fuse them into a multimodal embedding for the adapter

Parameters:

Name	Type	Description	Default
out_features	int	Number of output features.	required
input_size	int	number of input features for the first modality.	required
input_size_1	int	number of input features for the second modality.	required
input_size_2	int	number of input features for the third modality.	None
fusion	Callable[[int, int, int], ConcatFusion]	The callable that returns a fusion module.	ConcatFusion
adapter	Callable[[int, int], TokenAdapter]	The callable that returns an adapter.	MLPAdapter