SMC Steering with Learned Twisting Functions
Probabilistic language-model steering with contrastive twist learning and sequential Monte Carlo.
A research prototype exploring inference-time language-model steering with learned twisting functions and sequential Monte Carlo (SMC).
This implementation studies the approach introduced by Zhao et al. in Probabilistic Inference in Language Models via Twisted Sequential Monte Carlo.
The implementation trains a twisting function through contrastive twist learning (CTL) while keeping the base language model frozen. PyTorch/Pyro experiments use GPT-2 Medium and Qwen3-0.6B, with terminal review-scoring objectives.
The experiments compare unsteered generation, terminal-score weighting, twisted sequential importance sampling (SIS), and twisted SMC. Fixed-length and end-of-sequence variants track effective sample size, particle ancestry, and terminal scores to examine weight concentration and resampling behavior.
This is experimental research on sampling and steering; the current validation does not establish an improvement in text quality.