inference

Foundry Local enables local execution of Small Language Models using the hardware on your device.

Desktop GUI for BitLlama LLM inference engine with Soul learning and model management

Pure Rust LLM inference engine with 1.58-bit ternary support and Test-Time Training