--lex_anscombe

Switch

--lex_anscombe

Description

Applies an Anscombe transformation to the lexicon/category value produced by --add_lex_table.

Argument and Default Value

No argument. Boolean switch, default: False (no transform applied to the lexicon value).

Details

Applies the Anscombe transformation (2*sqrt(x + 3/8)) to the final lexicon/category value produced by --add_lex_table (or --add_corp_lex_table), as opposed to --anscombe which transforms the underlying word/ngram frequencies that feed into that value. This is useful when the lexicon's raw counts are sparse/low, since the Anscombe transform stabilizes variance better than a plain square root in that regime.

Verified: folds a suffix into the lexicon name portion of the table, joined with an underscore rather than a $, e.g. feat$cat_met_a30_2000_cp_w_16to8$msgs$user_id for a weighted lexicon (int(2*sqrt(16 + 3/8)) == 8). See --lex_sqrt for the general naming mechanism.

Other Switches

Required Switches:

• --add_lex_table

• -l <LEX_TABLE>

Optional Switches:

None

Example Commands

dlatkInterface.py -d dla_tutorial -t msgs -c user_id --add_lex_table -l met_a30_2000_cp --weighted_lexicon --lex_anscombe