Overview
Architecture
Map Python orchestration and the C++ runtime from model/autograd through compiler, lowering, generic programmed composition, serving, and backends.
Read guideConcepts
Understand the design boundaries, lifecycle rules, data representation, and optimization model.
Concept
Module and Parameter Lifecycle
Understand parameter registration, ownership, recursive names, nonmoving modules, and transactional backend transfer.
Read guideConcept
Tokenization and Next-Token Batches
Understand byte and byte-pair tokenization strategies, token IDs, UTF-8 boundaries, and shifted next-token windows.
Read guideConcept
Adam Optimizer
Understand bias-corrected Adam, global gradient clipping, contiguous or bounded-page state, and transactional updates.
Read guide