6 commits

Author	SHA1	Message	Date
George Hotz	b5a9465b13	llm: add support for moonlight (deepseek MLA) (#15466) ... * add gguf Q5_0 * it works * rebase * simpler test * class * less diff * dicts * normal names * simplify * this * simpler * work * work	2026-04-11 10:32:48 +08:00
b1tg	9ab1415937	llm: fix streaming UTF-8 decode (#15653)	2026-04-10 17:01:02 +08:00
George Hotz	fe2690399b	llm: support assistant prefill + refactor to TransformerConfig (#15457) ... * llm: support assistant prefill * refactor to ModelConfig * TransformerConfig * more	2026-03-25 10:50:48 +08:00
George Hotz	a33ac869aa	llm server: temperature + test client (#15444) ... * improvements to the llm server * eval script * eval llm * better eval gets 58.71 * cleanups * add temperature, but multinomial is absurdly slow * claude is so smart * lint * remove slop * no more stop	2026-03-24 21:07:15 +08:00
George Hotz	8a82b26522	llm: print the prefill cache size (#15146) ... * print the llm prefill cache size * mock that too	2026-03-05 12:13:28 +08:00
George Hotz	d59e6e7a37	move more tests to test/null, split some existing ones (#14512) ... * move more tests to test/null, split some existing ones * null work * null work * move more * fixes * move PIL * PIL in CLIP * don't move that	2026-02-03 20:20:20 +08:00