--add_message_id

Switch

--add_message_id MESSAGE_FILE STATE_FILE_GZ

Description

Rewrites a Mallet LDA state file's numeric document index back into real group ids, by cross-referencing the joined-feature-lines file used to build the LDA model.

Argument and Default Value

Two file paths: the joined-feature-lines file (as written by --print_joined_feature_lines) and the gzipped Mallet state file. Default: None (not used directly).

Details

Verified: this is normally triggered automatically by --estimate_lda_topics (which sets args.addmessageid = [joined_feature_lines_file, state_file] itself right after calling Mallet) — you don't need to pass it by hand in the common workflow. Despite the name, the id substituted in is whichever group id the feature table used (e.g. user_id), not necessarily a literal message id — confirmed by running with a user-level feature table and seeing real user_id values appear in the rewritten state file's source column, in place of Mallet's internal sequential document index.

Other Switches

Optional Switches:

Example Commands

Using the packaged dla_tutorial dataset (see Differential Language Analysis (DLA) Tutorial and DLATK LDA Interface) — verified end-to-end with real Mallet:

dlatkInterface.py -d dla_tutorial -t msgs -c user_id -f 'feat$1gram$msgs$user_id' \
--estimate_lda_topics --num_topics 20 --no_lda_lexicon \
--mallet_path /opt/homebrew/bin/mallet --save_lda_files ./lda_out --num_lda_threads 2