437 commits

Author	SHA1	Message	Date
qazal	26d0265d66	test schedule of LazyBuffers [run_process_replay] (#5859)	2024-08-01 19:06:29 +03:00
qazal	1b53207b4f	revert isolated dags scheduling (#5724)	2024-07-25 19:45:12 -04:00
qazal	9ceb3a3d1f	beautiful_mnist -4.3% kernels (#5709) ... * add is_complete * partially delete forced_realized * p2 * start * refactor to can_group * remove steps * _get_inputs is nicer * fix the cache * cache is dict now * rename to group	2024-07-25 20:30:49 +03:00
George Hotz	dc21e63bd2	test: put conv in one reduce (#4441) ... * test: put conv in one reduce * put reduce at the end * more expand * generic, and that expand was breaking things * ratio * don't undo the expand * arg 1 * strides * warning, for resnet * warning removed * disable cast * handle cast * op * err, that's right * fixup * fix that * a test to play with * add double_reduces * working up to final reshape * fold the last reshape * moved to schedule * fix axis * ci, need to bring arange back * FUSE_CONV_BW maybe * valid in 3.9 * test_expand_reduce_is_folded_on_different_axes * add FUSE_CONV_BW=1 * test_fold_batchnorm_backward * test_sgd_4convs_fuse --------- Co-authored-by: qazal <qazal.software@gmail.com>	2024-07-22 12:16:13 +03:00
kormann	2c4add6844	pretty print lazy op per default (#5505) ... * pretty lop * min diff * walrus * fix * min diff * simplify * pretty helper function * ws * pretty uop upat * tests * stricter tests * test passes * ws * stronger upat test * delete print_tree * min diff * stricter exp test * fix merge * stronger uops eval test * +readable and deep upat test * +readable and deep upat test * sort inv fix * fix * revert allowed_len	2024-07-18 09:34:08 -07:00
George Hotz	fa7e734b49	MetaOps.KERNEL (#5543)	2024-07-17 19:41:23 -07:00
wozeparrot	90f0e2fc49	db in wal mode (#5388)	2024-07-12 20:43:36 -07:00
chenyu	4df63da190	clean up rest of the loadop [run_process_replay] (#5440) ... to metaop and filter_sink	2024-07-12 23:38:51 -04:00
George Hotz	03c2dc8bd7	lowerer is kernel [run_process_replay] (#5437)	2024-07-12 18:50:55 -07:00
George Hotz	870dc8c350	s/Linearizer/Lowerer [run_process_replay] (#5428)	2024-07-12 15:54:07 -07:00
George Hotz	6707c778d0	scheduleitem is not Tuple [run_process_replay] (#5425) ... * scheduleitem is not Tuple [run_process_replay] * fix tests * fix op + fuzzers * fix mop test	2024-07-12 15:13:19 -07:00
George Hotz	f6ef283e6a	s/loadops/metaops [run_process_replay] (#5421)	2024-07-12 13:26:50 -07:00
qazal	e22b377839	generalize FUSE_AS_ONE_KERNEL in the scheduler (#5397) ... * test: use const * hotfix: base * asserts * dont push through reshape * cleanup * dont need the cache * test_reduceop_reshape_dont_push and test_index_fused are next	2024-07-12 10:23:16 +03:00
Timmy	bb7746985f	multireduce scheduler tests (#5141) ... Co-authored-by: qazal <77887910+Qazalin@users.noreply.github.com>	2024-07-08 20:28:55 +03:00
qazal	5292d37db6	LoadOps.VIEW in the scheduler spec (#5296) ... * refactor to allow_buffer_view * tests * fix multi	2024-07-05 19:43:50 +03:00
hikettei	1ab7a4cff0	Handling Multiple UnaryOps.BITCAST in Function for Proper Kernel Fusion [run_process_replay] (#5172) ... * [Patch] added an option not to ignore view replacing when doing bitcast * added the testcase * [Add] reproduced bitcast cannot be fused into a single kernel in the unittest --------- Co-authored-by: qazal <77887910+Qazalin@users.noreply.github.com>	2024-07-05 19:16:44 +03:00
chenyu	ee0c6dfc15	build Tensor._tri with movements only (#5110) ... * build Tensor._tri with movements only doesn't need arange, saved a kernel in attention mask * simpler, more tests	2024-06-23 00:07:36 -04:00
chenyu	2b2488f2e2	revert creating Tensor from a list without numpy (#5041) ... the change was incomplete and broke creating Tensor from a list of np array	2024-06-18 17:31:22 -04:00
qazal	04feeb37e6	look for unsafe pad ops in multiview ShapeTrackers (#5002)	2024-06-17 00:28:12 +03:00
Timmy	01b26756d6	Multireduce Scheduler Tests (#4972) ... * scheduler tests * linters * cleaning up tests * fixing tests * syntax * fixing metal	2024-06-16 16:30:22 +03:00
chenyu	5eee974b2a	construct Tensor from python list/tuple directly (#4947) ... * construct Tensor from python list/tuple directly no numpy. annoying that half memoryview is 3.12 feature... * simpler, and test * flat already * simpler * cute * 10% faster * 5%	2024-06-14 11:36:05 -04:00
chenyu	286b4dbdf2	compile raise CompileError and skip only RuntimeError in multiprocess… (#4646) ... * compile raise CompileError and skip only RuntimeError in multiprocess beam renderer error with multiprocess should not be skipped by beam * use `==` for dtype to dtype comparison * that needs to be is * typo	2024-05-19 00:25:25 -04:00
qazal	bf8f855838	assert kernel counts in unsupported fusions (#4643) ... * replace with comments * not relevant * update comment * custom exception maybe * fix LoadOps.VIEW	2024-05-18 20:14:37 +03:00
qazal	f3f2b96583	pick schedule tests from external_test_opt (#4615) ... * conv tests * misc * that shouldnt const fold	2024-05-16 15:43:41 +03:00
qazal	13200c6894	check simple_pads in all views (#4614)	2024-05-16 14:34:39 +03:00
qazal	0b464df605	base change scheduling spec (#4613) ... * spec and kernel cnt * dont use half * skip half	2024-05-16 13:30:49 +03:00
George Hotz	ff64bcab69	move graph/search to engine (#4596)	2024-05-14 23:12:59 -07:00
George Hotz	fd02ab1e8b	move disassemblers and openpilot (#4592) ... * move disassemblers and openpilot * delete junk * put that in pre-commit * fixup readme	2024-05-14 19:30:02 -07:00
qazal	355e1c135c	pad fusion tests (#4570) ... * what breaks * Revert "what breaks" This reverts commit e79f679283. * simplest case * one unsafe op * expand+pad, shrink+pad * safe case * refactor	2024-05-14 20:34:46 +03:00
chenyu	7afca52796	replace pow in LAMB by tracking b1**t and b2**t per step (#4582) ... * replace pow in LAMB by tracking b1**t and b2**t per step * remove t, add [self.b1_t, self.b2_t] to return * adam has one less kernel	2024-05-14 13:08:22 -04:00
George Hotz	17faae091b	optimizer shouldn't be run without training (#4460) ... * optimizer shouldn't be run without training * set training in relevant tests * fix multitensor * that too	2024-05-06 15:34:12 -07:00
qazal	6dbe5585b0	batchnorm + conv backward in test_schedule (#4420) ... * test both optims * batchnorm_backward	2024-05-06 16:40:17 +03:00
George Hotz	595a6e3069	test_fold_conv_relu_backward test	2024-05-05 11:13:43 -07:00
David Hou	c0a048c044	batchnorm d(var)/d(mean) = 0 (#4430) ... * d(var)/d(mean) = 0 * drop the number in test_schedule!	2024-05-05 00:25:45 -04:00
qazal	5f3bae378f	search children in fusion (#4322) ... * scheduler diff * tests diff * new changes * realizes * chores * assign * kind of r3 * forced_realize wont do it * with forced_realize * start with children * test search * r3 with parents * diff cleanup * add children * crossing assign * late fuse descendants * update kernel counts * assign diff doesnt belong here	2024-05-04 17:22:15 +03:00
qazal	0deaaf2bc8	partial fusion spec (#4398)	2024-05-03 04:14:23 +03:00
qazal	0b47818e0f	simpler reduceop children chasing (#4350) ... * simplest case * midreduce case * all tests * pending things * unify tests	2024-05-02 15:15:30 +03:00
qazal	ea06f657df	fusion tests from test_opt (#4357) ... * opt tests * more sgd * batchnorm * models stay in external	2024-05-01 16:44:12 +03:00
qazal	cc1797673e	all fusion opportunities (#4348)	2024-04-29 19:32:23 +03:00
qazal	3372bea322	reduce children fusion tests (#4321) ... * base tests * real-world tests	2024-04-28 11:14:02 -04:00
qazal	74a1be88f5	test reduce graph permutations (#4291)	2024-04-25 11:34:44 +03:00
George Hotz	967638f0d5	update docs, remove corealize (#4264) ... * update docs, remove corealize * handle 0 line count * tensor schedule	2024-04-23 12:05:29 +04:00
qazal	77a3780005	assert reduce recompute (#4250)	2024-04-22 16:12:39 +03:00
qazal	f75020a903	minimal diff for multioutput reduce pairs (#4030) ... * simple fusion * compiler cache patch * Revert "compiler cache patch" This reverts commit fa18049597. * Revert "Revert "compiler cache patch"" This reverts commit 57f8d41f98. * delete that * early sort * teeny renames * spec * .empty is great * delete sort * Update test_schedule.py * this is one kernel now --------- Co-authored-by: George Hotz <72895+geohot@users.noreply.github.com>	2024-04-17 10:55:44 -04:00
David Hou	97d846dd67	in forced_realize, unchase last op if it is upcast (#4185) ... * in forced_realize, unchase last op if it is upcast * start on test * flesh out test * more test * comment * comment out parallel reduce test * reorder * unused	2024-04-16 17:15:17 -04:00
George Hotz	ae849d12d7	numpy device + pickle it (#4120)	2024-04-09 13:19:30 -07:00
chenyu	f61ed869f5	Use exec_alu for lazy const folding (#4039)	2024-04-02 20:52:05 -04:00
chenyu	0e02d074bd	fix Tensor.pow folding for exponent 0 and 1 (#4025)	2024-03-31 19:57:23 -04:00
George Hotz	68ca4d4276	split to schedule.py (#3949) ... * split to schedule.py * split	2024-03-26 21:02:46 -07:00
George Hotz	150ea2eb76	create engine folder and move code (#3948) ... * retry * older tf * that	2024-03-26 20:38:03 -07:00