3 commits

Author	SHA1	Message	Date
kevvz	e2873a3a41	[bounty] Muon optim (#11414) ... * newton schulz * add muon + move newton schulz to tensor * compact newton schulz * better tests * cleanup * add comments for muon * cleanup * add export with tests * match muon optim with test optim * cleanup * unsed import * correct comment * whitespace * move export * muon test fix * match reference impl + tests * remove export by moving muon device * add credit * cleanup * remove print * spacing * spacing * comma * cleanup * removal * fix tests + optim momentum * consistent is not/ not * more consistency * fix test * cleanup * fix the nones * remove comment * cast * comment * comment * muon teeny test * muon flag beautiful mnist * set steps * steps as hyperparam * match default test steps * name * large cleanup * dont care about steps * nesterov false default * match each other impl * steps * switch nest * swap defaults * update docstring * add no nesterov test * ban fuse_optim * prints * classical momentum * alternative condition * recon * pre + post wd * false default * detach * signature changes * context * swap order * big cleanup * 0 step instead * parity * remove fuse * remove fused * better paper * assert message * correct shape check + eps * multidim * add eps * cleanup * correct assert message * lint * better tests * naming * ns_steps,ns_params * update docstring * docstring * match sgd and muon together * sandwich * add back fused * parity --------- Co-authored-by: George Hotz <72895+geohot@users.noreply.github.com>	2025-08-13 14:27:55 -04:00
kevvz	ef7e01cadf	Fix SVD shape bug + Fix batched SVD bug (#11477) ... * failing test case * fix * better test * space	2025-08-02 09:47:41 -07:00
kevvz	3b041d188f	[bounty] Singular Value Decomposition (#10875) ... * inital commit * add qr + expand svd to full matrix * add odd number support * add linalg tests * qr supports dims of arbitrary size * add qr tests * svd supports dims of arbitrary size * small cleanip * improvements over svd batch handling * improve linalg tests * make u_pad match q shape * add nonfull matrix tests * little less verbose nonfull svd test * added dtypes on svd + return vt instead of vt * lint * more lint * lint + set seed * small fix * small lint * lint * add int casting to indices and shapes * remove int from shape tuple in svd * small cleanup * add return types * reuse inverse_permute * refactoring * whitespace * remove regularization term to prevent bad outputs on ill conditioned matrices * remove seed * refactor * lint * refactor * spacing * remove clone * line reduction * smarter heuristic for iterations_per_round * add big test * lint * turns out no constant needed? * wrap tests * some small matrices need the constant * remove realize --------- Co-authored-by: George Hotz <72895+geohot@users.noreply.github.com>	2025-07-02 09:06:03 -07:00