mirrors/tinygrad
2
0
Fork 0
mirror of https://github.com/tinygrad/tinygrad.git synced 2026-06-24 02:14:17 +00:00
Code Issues Projects Releases Packages Wiki Activity
11,106 commits 280 branches 15 tags 325 MiB
Commit graph

11,106 commits

This branch
This branch
All branches
Author SHA1 Message Date
George Hotz
6ebfb505e9
 		docs: fix crossentropy name (#10377) 2025-05-17 16:39:14 -07:00
George Hotz
0b733ba75e
 		multi device training with GPT2 [pr] (#10375) 
* multi device training with GPT2 [pr]

* Update grouper.py
 2025-05-17 15:33:56 -07:00
George Hotz
6ec88d94df
 		add tests for multi ram usage [pr] (#10376) 2025-05-17 15:33:40 -07:00
uuuvn
5a18eab908
 		Fix __del__ in remote program (#10372) 
Similar to #10341, broke after hypotesis unpin
 2025-05-17 21:29:44 +03:00
वेदांत
2453d99050
 		rms matching pytorch implementation (#10319) 
* rms matching pytorch implementation

* pre commit fix

---------

Co-authored-by: chenyu <chenyu@fastmail.com>
 2025-05-17 08:23:11 -07:00
nimlgen
da2b1834b4
 		hotfix: metal graph var vals (#10370) 2025-05-17 17:22:55 +03:00
qazal
e054b53a75
 		kernel count tests for pad [pr] (#10369) 
* kernel count tests for pads

* handcoded rand one kernel

* comment

* prerealize device rng counter

* test_rand_handcoded generates /0

* remove track_rewrites
 2025-05-17 17:20:46 +03:00
nimlgen
90c4bb10c0
 		fixedvars in all graphs (#10365) 
* cuda fixedvars

* metal: fixevars

* f

* ups

* count fixedvars
 2025-05-17 16:18:52 +03:00
chenyu
efa8dfe7fb
 		test cron job to run resnet (#10368) 2025-05-17 08:57:02 -04:00
uuuvn
2c706d363e
 		Remote higher timeout and overridable via REMOTE_TIMEOUT (#10367) 
sometimes a minute is not enough, 5 minutes should be but if it isn't
for some huge workload it can be overridden
 2025-05-17 15:30:49 +03:00
nimlgen
4fa1837916
 		metal: do not require icb fix on m3+ (#10366) 2025-05-17 15:30:40 +03:00
Xingyu
286b0f4051
 		Add equal function implementation and corresponding test (#10351) 
- Implemented a new function `equal` in the torch backend to compare two tensors for equality.
- Added unit tests for the `equal` function to verify its correctness with different tensor inputs.
 2025-05-16 23:39:49 -07:00
George Hotz
e13f2a3092
 		multi is O(1) (#10183) 
* multi is O(1)

* allreduce

* no new uops needed

* junk

* something

* simple

* that's really what i want

* closer

* inject _device_num

* pretty print

* cleanups

* this

* early dnum

* ops allreduce is good

* ish

* device is the tuple and this is fine

* simpler

* progress

* copy_multi

* work

* more tests

* more tests pass

* work

* no None axis

* tests

* no none multi

* type fixes

* pre commit passes

* lil

* remove this

* mlperf dataloader on mac

* that test was wrong

* unbind

* support DEBUG=2

* realize

* only unbind bound vars

* don't include fixedvars

* graph test

* one test

* fixedvars in hcq

* new ring reduce

* ring reduce

* simpler ring

* mselect

* mselect doesn't work

* Revert "mselect doesn't work"

This reverts commit c78b77bd7d.

* Revert "mselect"

This reverts commit bb2e430ac3.

* simpler

* fixups

* no optional

* fix jit

* move things around

* cleanup multi

* simpler multi

* simpler reshape
 2025-05-16 23:14:23 -07:00
George Hotz
e1a40e8040
 		add hcq fixedvars support [pr] (#10356) 
* add hcq fixedvars support [pr]

* different test

* fixedvars are only for comp_queues

* fix hcq varvals
 2025-05-16 22:05:53 -07:00
George Hotz
11b5895c85 hotfix: schedule timing in tensor.py 2025-05-16 20:10:32 -07:00
uuuvn
64409a8bda
 		Remote beam (#10357) 
* Use renderer properties instead of `.device`

* Remote beam
 2025-05-16 18:59:22 -07:00
George Hotz
7cc35a031b
 		don't use UOp.multi in Tensor.rand (#10362) 2025-05-16 16:09:36 -07:00
George Hotz
7703dbef99
 		view substitute [pr] (#10360) 2025-05-16 15:08:24 -07:00
Elnur Rakhmatullin
de2b323d97
 		Fixed a typo in "simplify" (#10358) 2025-05-16 14:45:14 -07:00
Harald Schäfer
ee5258328a
 		You never want multiple backends (#10354) 2025-05-16 13:10:39 -07:00
George Hotz
876d2275a1
 		changes from new multi (#10353) 
* changes from new multi

* revert hcq change
 2025-05-16 13:07:29 -07:00
wozeparrot
66e00c04dd
 		fix: skip kernel timing tests on ci cuda (#10348) 2025-05-16 11:48:06 -07:00
Ignacio Sica
a54fd745c3
 		simpler barrier match in remu (#10339) 
* s_barrier

* remove s_barrier from syncs
 2025-05-16 14:40:58 +03:00
qazal
e9e5b54e43
 		grouper cleanups and merge with insert_kernels [pr] (#10349) 
* grouper cleanups and merge with insert_kernels [pr]

* remove that
 2025-05-16 14:39:56 +03:00
b1tg
caded2f413
 		llvm diagnostic error (#10267) 
* llvm diagnostic info

* use decorator

* better error reporting

* fix mypy

* collect all diag msgs

* test diag error

---------

Co-authored-by: b1tg <b1tg@users.noreply.github.com>
Co-authored-by: chenyu <chenyu@fastmail.com>
 2025-05-16 02:03:20 -04:00
George Hotz
a4a25720b2
 		add test_multitensor_jit_input [pr] (#10347) 2025-05-15 20:47:57 -07:00
chenyu
c798f2f427
 		brew --quiet to suppress already installed warnings (#10346) 
example https://github.com/tinygrad/tinygrad/actions/runs/15057000247
 2025-05-15 23:31:18 -04:00
wozeparrot
12a1ccc680
 		clean: double import (#10345) 2025-05-15 20:15:09 -07:00
wozeparrot
1ed04f993b
 		move benchmark stat tracking to influxdb (#10185) 2025-05-15 16:14:56 -07:00
wozeparrot
f59ecf2116
 		fix: mockgpu cuda timing (#10343) 2025-05-15 14:14:14 -07:00
nimlgen
a825608dc2
 		hcq: fix progs' __del__ when shutdown (#10341) 
* debug ci

* better?

* and mute this?

* revrt that
 2025-05-15 23:26:48 +03:00
Ignacio Sica
47b3055fe2
 		set fail-fast behavior (#10336) 2025-05-15 11:24:45 -07:00
uuuvn
c2bf2c6bb0
 		Remote offset (#10311) 
For memory savings from memory planner. Also for some reason it makes hlb
cifar on mac noticeably faster.

master:
```
  3  210.12 ms run,    4.34 ms python,  205.78 ms REMOTE, 2075.90 loss, 0.002698 LR, 2.07 GB used,   1558.41 GFLOPS,    327.45 GOPS
  4  210.40 ms run,    4.33 ms python,  206.07 ms REMOTE, 2481.94 loss, 0.002262 LR, 2.07 GB used,   1556.34 GFLOPS,    327.45 GOPS
  5  188.08 ms run,    4.41 ms python,  183.67 ms REMOTE, 1967.49 loss, 0.001827 LR, 2.07 GB used,   1741.00 GFLOPS,    327.45 GOPS
  6  211.19 ms run,    4.26 ms python,  206.93 ms REMOTE, 1511.62 loss, 0.001392 LR, 2.07 GB used,   1550.51 GFLOPS,    327.45 GOPS
```

this:
```
  3  189.05 ms run,    4.50 ms python,  184.55 ms REMOTE, 2075.90 loss, 0.002698 LR, 1.60 GB used,   1732.08 GFLOPS,    327.45 GOPS
  4  187.81 ms run,    4.11 ms python,  183.71 ms REMOTE, 2481.94 loss, 0.002262 LR, 1.60 GB used,   1743.49 GFLOPS,    327.45 GOPS
  5  186.70 ms run,    4.09 ms python,  182.62 ms REMOTE, 1967.49 loss, 0.001827 LR, 1.60 GB used,   1753.89 GFLOPS,    327.45 GOPS
  6  187.18 ms run,    4.06 ms python,  183.12 ms REMOTE, 1511.62 loss, 0.001392 LR, 1.60 GB used,   1749.36 GFLOPS,    327.45 GOPS
```

(`PYTHONPATH=. REMOTE=1 REMOTEDEV=METAL BS=256 STEPS=10 python examples/hlb_cifar10.py`)

Clouldn't reliably reproduce the faster thing on tinybox though.
 2025-05-15 11:20:01 -07:00
Ignacio Sica
3c453e96a9
 		add ds_load_b96 and ds_store_b96 instructions (#10338) 2025-05-15 18:11:08 +03:00
qazal
be8202b293
 		add s_abs_i32 instruction to remu (#10334) 2025-05-15 16:47:58 +03:00
nimlgen
5efbe1c947
 		print offset only for subbuf (#10332) 2025-05-15 15:35:19 +03:00
qazal
7cfe367c07
 		failing test for slow embedding kernel with FUSE_ARANGE=1 [pr] (#10330) 2025-05-15 14:58:11 +03:00
nimlgen
5f03688280
 		usbgpu: remove max_read_len (#10328) 2025-05-15 14:49:58 +03:00
qazal
27b3dbe67e
 		remove FUSE_ARANGE_UINT [pr] (#10324) 2025-05-15 14:39:54 +03:00
qazal
0a45cd0cbe
 		grouper: merge views in fuse elementwise (#10325) 
* grouper: merge views in fuse elementwise

* with gradient api
 2025-05-15 13:17:09 +03:00
qazal
89d8d5b25e
 		add dims check in FUSE_ARANGE (#10323) 2025-05-15 11:33:21 +03:00
qazal
8fad0f0124
 		grouper: check for unsafe PAD in FUSE (#10322) 2025-05-15 10:53:44 +03:00
chenyu
f008e5f233
 		test_dtype_alu should cast bf16 input (#10320) 
when testing alu for bfloat16, it should cast inputs to bfloat16 first, otherwise numpy has both errors from input and errors from alu which is more inaccurate
 2025-05-15 01:11:39 -04:00
George Hotz
568d6d96e7
 		small changes from new multi [pr] (#10318) 2025-05-14 20:50:59 -07:00
chenyu
f6cf25fce4
 		cleanup test_conv2d_ceildiv_edge_case [pr] (#10317) 2025-05-14 23:35:28 -04:00
Kirill R.
50d7162acd
 		Add conv2d ceildiv edge case (#10303) 2025-05-14 22:50:23 -04:00
uuuvn
e5639b7788
 		Remote finalize (#10314) 
* Remote `.q(..., wait=True)`

Seems a bit cleaner than doing `.batch_request()` after `.q(...)` for
requests with return value.

* Remote finalize

---------

Co-authored-by: George Hotz <72895+geohot@users.noreply.github.com>
 2025-05-14 19:40:44 -07:00
George Hotz
bfc30fa6ea hotfix: typo in shm_name 2025-05-14 19:34:52 -07:00
George Hotz
2bc54b3e22 manually handle OSX 2025-05-14 19:17:51 -07:00
George Hotz
ab460486d7 Revert "resnet dataloader osx (#10316)" 
This reverts commit aef336930a.
 2025-05-14 19:15:07 -07:00