--bert_layers N [N1 N2 ... Nk]


Specifies the BERT layers to aggregate over. One or more layers may be specified.

Argument and Default Value

Default: the last four layers (8-11).


Layers should be specified by index, beginning from the 0th index. 11 is the index of the final (12th) layer.

Other Switches

Required Switches:

Optional Switches:

Example Commands

Creates BERT features aggregated over the last two layers.

dlatkInterface.py -d dla_tutorial -t msgs -c user_id --add_bert --bert_layers 10 11