Cajal Compiler to Transformer Heads
Compiling typed programs into programmed linear-attention heads.
Compiles typed Cajal-lite programs into programmed linear-attention heads and combines them with learned transformer components.
The compiler checks typed symbolic programs, lowers them into multilinear maps, and turns those maps into differentiable modules. This work is part of Riftco Transformer.
Read about compiling programs to transformers or view the source code.