mirrors/tinygrad
2
0
Fork 0
mirror of https://github.com/tinygrad/tinygrad.git synced 2026-06-24 02:14:17 +00:00
Code Issues Projects Releases Packages Wiki Activity
8,871 commits 280 branches 15 tags 325 MiB
Commit graph

8,871 commits

This branch
This branch
All branches
Author SHA1 Message Date
qazal
e55ee28b29
 		little smaller viz/worker.js [pr] (#10402) 2025-05-18 23:44:46 +03:00
qazal
8a6fb37560
 		move viz /prof to extra [pr] (#10401) 2025-05-18 23:25:59 +03:00
George Hotz
411392dfb7
 		move files into uop dir (#10399) 
* move files into uop dir [pr]

* tinygrad.uop is a thing

* fix uop docs, no pr

* fix viz
 2025-05-18 11:38:28 -07:00
uuuvn
0f825e12f2
 		Remote fixedvars (#10371) 
* amd mockgpu graph support

For testing remote graph stuff (prompted by #10371) in ci

* Remote fixedvars

Somehow none of existing tests failed when fixedvars were added, looking
what to add as an regression test for this

---------

Co-authored-by: George Hotz <72895+geohot@users.noreply.github.com>
 2025-05-18 09:57:13 -07:00
uuuvn
27c12be471
 		amd mockgpu graph support (#10385) 
For testing remote graph stuff (prompted by #10371) in ci
 2025-05-18 09:43:16 -07:00
George Hotz
a3308e145d hotfix: remote print -> DEBUG=3 2025-05-18 09:09:04 -07:00
qazal
04b23087d8
 		grouper tests from fuse_arange_default [pr] (#10394) 2025-05-18 18:42:43 +03:00
qazal
17f0f5e764
 		add v_rcp_f32_e64 to remu (#10393) 
* tests from the box

* add v_rcp_f32_e64 to remu

* f32::from_bits utils

* v_cndmask_b32 tests
 2025-05-18 17:08:21 +03:00
qazal
9e2089dcd4
 		don't raise Exception in process replay [pr] (#10392) 
* don't raise Exception in process replay [pr]

* continue generating diffs unless [pr] is set, exit(1) otherwise

* change

* works
 2025-05-18 11:23:23 +03:00
chenyu
9b4e2a75cd
 		symlink datasets in mlperf workflow (#10391) 2025-05-18 03:26:05 -04:00
uuuvn
f20c5aac1f
 		Use itertools.count instead of manual increment in remote (#10389) 
Similar to how it's done with `UOp.unique_num`, looks a bit nicer
 2025-05-18 00:15:37 -07:00
qazal
0294bfe507
 		simpler can_pad (#10364) 
* simpler can_pad [pr]

* 3 kernels

* tests

* less kernels
 2025-05-18 10:00:07 +03:00
George Hotz
c91f2c4580
 		use float32 for sgd momentum (#10387) 2025-05-17 21:56:44 -07:00
George Hotz
305a3231c4
 		fix beam none if buf is optimized out (#10388) 2025-05-17 21:50:33 -07:00
George Hotz
6f77b938d7
 		Move getbits tests into test_helpers (#10382) 2025-05-17 17:04:00 -07:00
George Hotz
6ebfb505e9
 		docs: fix crossentropy name (#10377) 2025-05-17 16:39:14 -07:00
George Hotz
0b733ba75e
 		multi device training with GPT2 [pr] (#10375) 
* multi device training with GPT2 [pr]

* Update grouper.py
 2025-05-17 15:33:56 -07:00
George Hotz
6ec88d94df
 		add tests for multi ram usage [pr] (#10376) 2025-05-17 15:33:40 -07:00
uuuvn
5a18eab908
 		Fix __del__ in remote program (#10372) 
Similar to #10341, broke after hypotesis unpin
 2025-05-17 21:29:44 +03:00
वेदांत
2453d99050
 		rms matching pytorch implementation (#10319) 
* rms matching pytorch implementation

* pre commit fix

---------

Co-authored-by: chenyu <chenyu@fastmail.com>
 2025-05-17 08:23:11 -07:00
nimlgen
da2b1834b4
 		hotfix: metal graph var vals (#10370) 2025-05-17 17:22:55 +03:00
qazal
e054b53a75
 		kernel count tests for pad [pr] (#10369) 
* kernel count tests for pads

* handcoded rand one kernel

* comment

* prerealize device rng counter

* test_rand_handcoded generates /0

* remove track_rewrites
 2025-05-17 17:20:46 +03:00
nimlgen
90c4bb10c0
 		fixedvars in all graphs (#10365) 
* cuda fixedvars

* metal: fixevars

* f

* ups

* count fixedvars
 2025-05-17 16:18:52 +03:00
chenyu
efa8dfe7fb
 		test cron job to run resnet (#10368) 2025-05-17 08:57:02 -04:00
uuuvn
2c706d363e
 		Remote higher timeout and overridable via REMOTE_TIMEOUT (#10367) 
sometimes a minute is not enough, 5 minutes should be but if it isn't
for some huge workload it can be overridden
 2025-05-17 15:30:49 +03:00
nimlgen
4fa1837916
 		metal: do not require icb fix on m3+ (#10366) 2025-05-17 15:30:40 +03:00
Xingyu
286b0f4051
 		Add equal function implementation and corresponding test (#10351) 
- Implemented a new function `equal` in the torch backend to compare two tensors for equality.
- Added unit tests for the `equal` function to verify its correctness with different tensor inputs.
 2025-05-16 23:39:49 -07:00
George Hotz
e13f2a3092
 		multi is O(1) (#10183) 
* multi is O(1)

* allreduce

* no new uops needed

* junk

* something

* simple

* that's really what i want

* closer

* inject _device_num

* pretty print

* cleanups

* this

* early dnum

* ops allreduce is good

* ish

* device is the tuple and this is fine

* simpler

* progress

* copy_multi

* work

* more tests

* more tests pass

* work

* no None axis

* tests

* no none multi

* type fixes

* pre commit passes

* lil

* remove this

* mlperf dataloader on mac

* that test was wrong

* unbind

* support DEBUG=2

* realize

* only unbind bound vars

* don't include fixedvars

* graph test

* one test

* fixedvars in hcq

* new ring reduce

* ring reduce

* simpler ring

* mselect

* mselect doesn't work

* Revert "mselect doesn't work"

This reverts commit c78b77bd7d.

* Revert "mselect"

This reverts commit bb2e430ac3.

* simpler

* fixups

* no optional

* fix jit

* move things around

* cleanup multi

* simpler multi

* simpler reshape
 2025-05-16 23:14:23 -07:00
George Hotz
e1a40e8040
 		add hcq fixedvars support [pr] (#10356) 
* add hcq fixedvars support [pr]

* different test

* fixedvars are only for comp_queues

* fix hcq varvals
 2025-05-16 22:05:53 -07:00
George Hotz
11b5895c85 hotfix: schedule timing in tensor.py 2025-05-16 20:10:32 -07:00
uuuvn
64409a8bda
 		Remote beam (#10357) 
* Use renderer properties instead of `.device`

* Remote beam
 2025-05-16 18:59:22 -07:00
George Hotz
7cc35a031b
 		don't use UOp.multi in Tensor.rand (#10362) 2025-05-16 16:09:36 -07:00
George Hotz
7703dbef99
 		view substitute [pr] (#10360) 2025-05-16 15:08:24 -07:00
Elnur Rakhmatullin
de2b323d97
 		Fixed a typo in "simplify" (#10358) 2025-05-16 14:45:14 -07:00
Harald Schäfer
ee5258328a
 		You never want multiple backends (#10354) 2025-05-16 13:10:39 -07:00
George Hotz
876d2275a1
 		changes from new multi (#10353) 
* changes from new multi

* revert hcq change
 2025-05-16 13:07:29 -07:00
wozeparrot
66e00c04dd
 		fix: skip kernel timing tests on ci cuda (#10348) 2025-05-16 11:48:06 -07:00
Ignacio Sica
a54fd745c3
 		simpler barrier match in remu (#10339) 
* s_barrier

* remove s_barrier from syncs
 2025-05-16 14:40:58 +03:00
qazal
e9e5b54e43
 		grouper cleanups and merge with insert_kernels [pr] (#10349) 
* grouper cleanups and merge with insert_kernels [pr]

* remove that
 2025-05-16 14:39:56 +03:00
b1tg
caded2f413
 		llvm diagnostic error (#10267) 
* llvm diagnostic info

* use decorator

* better error reporting

* fix mypy

* collect all diag msgs

* test diag error

---------

Co-authored-by: b1tg <b1tg@users.noreply.github.com>
Co-authored-by: chenyu <chenyu@fastmail.com>
 2025-05-16 02:03:20 -04:00
George Hotz
a4a25720b2
 		add test_multitensor_jit_input [pr] (#10347) 2025-05-15 20:47:57 -07:00
chenyu
c798f2f427
 		brew --quiet to suppress already installed warnings (#10346) 
example https://github.com/tinygrad/tinygrad/actions/runs/15057000247
 2025-05-15 23:31:18 -04:00
wozeparrot
12a1ccc680
 		clean: double import (#10345) 2025-05-15 20:15:09 -07:00
wozeparrot
1ed04f993b
 		move benchmark stat tracking to influxdb (#10185) 2025-05-15 16:14:56 -07:00
wozeparrot
f59ecf2116
 		fix: mockgpu cuda timing (#10343) 2025-05-15 14:14:14 -07:00
nimlgen
a825608dc2
 		hcq: fix progs' __del__ when shutdown (#10341) 
* debug ci

* better?

* and mute this?

* revrt that
 2025-05-15 23:26:48 +03:00
Ignacio Sica
47b3055fe2
 		set fail-fast behavior (#10336) 2025-05-15 11:24:45 -07:00
uuuvn
c2bf2c6bb0
 		Remote offset (#10311) 
For memory savings from memory planner. Also for some reason it makes hlb
cifar on mac noticeably faster.

master:
```
  3  210.12 ms run,    4.34 ms python,  205.78 ms REMOTE, 2075.90 loss, 0.002698 LR, 2.07 GB used,   1558.41 GFLOPS,    327.45 GOPS
  4  210.40 ms run,    4.33 ms python,  206.07 ms REMOTE, 2481.94 loss, 0.002262 LR, 2.07 GB used,   1556.34 GFLOPS,    327.45 GOPS
  5  188.08 ms run,    4.41 ms python,  183.67 ms REMOTE, 1967.49 loss, 0.001827 LR, 2.07 GB used,   1741.00 GFLOPS,    327.45 GOPS
  6  211.19 ms run,    4.26 ms python,  206.93 ms REMOTE, 1511.62 loss, 0.001392 LR, 2.07 GB used,   1550.51 GFLOPS,    327.45 GOPS
```

this:
```
  3  189.05 ms run,    4.50 ms python,  184.55 ms REMOTE, 2075.90 loss, 0.002698 LR, 1.60 GB used,   1732.08 GFLOPS,    327.45 GOPS
  4  187.81 ms run,    4.11 ms python,  183.71 ms REMOTE, 2481.94 loss, 0.002262 LR, 1.60 GB used,   1743.49 GFLOPS,    327.45 GOPS
  5  186.70 ms run,    4.09 ms python,  182.62 ms REMOTE, 1967.49 loss, 0.001827 LR, 1.60 GB used,   1753.89 GFLOPS,    327.45 GOPS
  6  187.18 ms run,    4.06 ms python,  183.12 ms REMOTE, 1511.62 loss, 0.001392 LR, 1.60 GB used,   1749.36 GFLOPS,    327.45 GOPS
```

(`PYTHONPATH=. REMOTE=1 REMOTEDEV=METAL BS=256 STEPS=10 python examples/hlb_cifar10.py`)

Clouldn't reliably reproduce the faster thing on tinybox though.
 2025-05-15 11:20:01 -07:00
Ignacio Sica
3c453e96a9
 		add ds_load_b96 and ds_store_b96 instructions (#10338) 2025-05-15 18:11:08 +03:00
qazal
be8202b293
 		add s_abs_i32 instruction to remu (#10334) 2025-05-15 16:47:58 +03:00