Access to Large Language Model Predictions

---

Documentation for package ‘pangoling’ version 1.0.3

• DESCRIPTION file.
• User guides, package vignettes and other documentation.
• Package NEWS.

Help Pages

causal_config	Returns the configuration of a causal model
causal_next_tokens_pred_tbl	Generate next tokens after a context and their predictability using a causal transformer model
causal_pred_mats	Generate a list of predictability matrices using a causal transformer model
causal_preload	Preloads a causal language model
causal_targets_pred	Compute predictability using a causal transformer model
causal_tokens_pred_lst	Compute predictability using a causal transformer model
causal_words_pred	Compute predictability using a causal transformer model
df_jaeger14	Self-Paced Reading Dataset on Chinese Relative Clauses
df_sent	Example dataset: Two word-by-word sentences
installed_py_pangoling	Check if the required Python dependencies for 'pangoling' are installed
install_py_pangoling	Install the Python packages needed for 'pangoling'
masked_config	Returns the configuration of a masked model
masked_preload	Preloads a masked language model
masked_targets_pred	Get the predictability of a target word (or phrase) given a left and right context
masked_tokens_pred_tbl	Get the possible tokens and their log probabilities for each mask in a sentence
ntokens	The number of tokens in a string or vector of strings
perplexity_calc	Calculates perplexity
set_cache_folder	Set cache folder for HuggingFace transformers
tokenize_lst	Tokenize an input
transformer_vocab	Returns the vocabulary of a model