Commit graph

8,826 commits

Author SHA1 Message Date
nimlgen
a825608dc2
hcq: fix progs' __del__ when shutdown (#10341)
* debug ci

* better?

* and mute this?

* revrt that
2025-05-15 23:26:48 +03:00
Ignacio Sica
47b3055fe2
set fail-fast behavior (#10336) 2025-05-15 11:24:45 -07:00
uuuvn
c2bf2c6bb0
Remote offset (#10311)
For memory savings from memory planner. Also for some reason it makes hlb
cifar on mac noticeably faster.

master:
```
  3  210.12 ms run,    4.34 ms python,  205.78 ms REMOTE, 2075.90 loss, 0.002698 LR, 2.07 GB used,   1558.41 GFLOPS,    327.45 GOPS
  4  210.40 ms run,    4.33 ms python,  206.07 ms REMOTE, 2481.94 loss, 0.002262 LR, 2.07 GB used,   1556.34 GFLOPS,    327.45 GOPS
  5  188.08 ms run,    4.41 ms python,  183.67 ms REMOTE, 1967.49 loss, 0.001827 LR, 2.07 GB used,   1741.00 GFLOPS,    327.45 GOPS
  6  211.19 ms run,    4.26 ms python,  206.93 ms REMOTE, 1511.62 loss, 0.001392 LR, 2.07 GB used,   1550.51 GFLOPS,    327.45 GOPS
```

this:
```
  3  189.05 ms run,    4.50 ms python,  184.55 ms REMOTE, 2075.90 loss, 0.002698 LR, 1.60 GB used,   1732.08 GFLOPS,    327.45 GOPS
  4  187.81 ms run,    4.11 ms python,  183.71 ms REMOTE, 2481.94 loss, 0.002262 LR, 1.60 GB used,   1743.49 GFLOPS,    327.45 GOPS
  5  186.70 ms run,    4.09 ms python,  182.62 ms REMOTE, 1967.49 loss, 0.001827 LR, 1.60 GB used,   1753.89 GFLOPS,    327.45 GOPS
  6  187.18 ms run,    4.06 ms python,  183.12 ms REMOTE, 1511.62 loss, 0.001392 LR, 1.60 GB used,   1749.36 GFLOPS,    327.45 GOPS
```

(`PYTHONPATH=. REMOTE=1 REMOTEDEV=METAL BS=256 STEPS=10 python examples/hlb_cifar10.py`)

Clouldn't reliably reproduce the faster thing on tinybox though.
2025-05-15 11:20:01 -07:00
Ignacio Sica
3c453e96a9
add ds_load_b96 and ds_store_b96 instructions (#10338) 2025-05-15 18:11:08 +03:00
qazal
be8202b293
add s_abs_i32 instruction to remu (#10334) 2025-05-15 16:47:58 +03:00
nimlgen
5efbe1c947
print offset only for subbuf (#10332) 2025-05-15 15:35:19 +03:00
qazal
7cfe367c07
failing test for slow embedding kernel with FUSE_ARANGE=1 [pr] (#10330) 2025-05-15 14:58:11 +03:00
nimlgen
5f03688280
usbgpu: remove max_read_len (#10328) 2025-05-15 14:49:58 +03:00
qazal
27b3dbe67e
remove FUSE_ARANGE_UINT [pr] (#10324) 2025-05-15 14:39:54 +03:00
qazal
0a45cd0cbe
grouper: merge views in fuse elementwise (#10325)
* grouper: merge views in fuse elementwise

* with gradient api
2025-05-15 13:17:09 +03:00
qazal
89d8d5b25e
add dims check in FUSE_ARANGE (#10323) 2025-05-15 11:33:21 +03:00
qazal
8fad0f0124
grouper: check for unsafe PAD in FUSE (#10322) 2025-05-15 10:53:44 +03:00
chenyu
f008e5f233
test_dtype_alu should cast bf16 input (#10320)
when testing alu for bfloat16, it should cast inputs to bfloat16 first, otherwise numpy has both errors from input and errors from alu which is more inaccurate
2025-05-15 01:11:39 -04:00
George Hotz
568d6d96e7
small changes from new multi [pr] (#10318) 2025-05-14 20:50:59 -07:00
chenyu
f6cf25fce4
cleanup test_conv2d_ceildiv_edge_case [pr] (#10317) 2025-05-14 23:35:28 -04:00
Kirill R.
50d7162acd
Add conv2d ceildiv edge case (#10303) 2025-05-14 22:50:23 -04:00
uuuvn
e5639b7788
Remote finalize (#10314)
* Remote `.q(..., wait=True)`

Seems a bit cleaner than doing `.batch_request()` after `.q(...)` for
requests with return value.

* Remote finalize

---------

Co-authored-by: George Hotz <72895+geohot@users.noreply.github.com>
2025-05-14 19:40:44 -07:00
George Hotz
bfc30fa6ea hotfix: typo in shm_name 2025-05-14 19:34:52 -07:00
George Hotz
2bc54b3e22 manually handle OSX 2025-05-14 19:17:51 -07:00
George Hotz
ab460486d7 Revert "resnet dataloader osx (#10316)"
This reverts commit aef336930a.
2025-05-14 19:15:07 -07:00
uuuvn
7b4f27a219
Remote .q(..., wait=True) (#10313)
Seems a bit cleaner than doing `.batch_request()` after `.q(...)` for
requests with return value.
2025-05-14 19:07:20 -07:00
George Hotz
50181ab09f hotfix: bump to 13500 lines 2025-05-14 18:49:59 -07:00
George Hotz
aef336930a
resnet dataloader osx (#10316)
* mlperf dataloader on mac

* resnet dataloader [pr]

* simple should work
2025-05-14 18:31:26 -07:00
wozeparrot
9b14e8c3cd
feat: tag 0.10.3 (#10310) v0.10.3 2025-05-14 15:45:13 -07:00
George Hotz
18f532d110
small changes from O(1) multi [pr] (#10309) 2025-05-14 15:34:07 -07:00
wozeparrot
9bbc2bc2a7
hotfix: filter_too_much (#10308) 2025-05-14 15:31:51 -07:00
George Hotz
fc8ef63194
multi doesn't need tuple arg anymore [pr] (#10307) 2025-05-14 15:16:40 -07:00
George Hotz
7a3d4de59a hotfix: add GRAPH_ONE_KERNEL=1 to UsbGPU openpilot test 2025-05-14 14:50:37 -07:00
wozeparrot
2df2ec6640
feat: unpin hypothesis (#10306) 2025-05-14 14:26:28 -07:00
uuuvn
b52452d69f
Remote multi (graph) (#9902)
* Remote multi (graph)

* Remote multi (graph transfers)

---------

Co-authored-by: George Hotz <72895+geohot@users.noreply.github.com>
2025-05-14 10:54:23 -07:00
George Hotz
42e70193c9
multi: instead of real, just copy (#10289)
* multi: instead of real, just copy

* fix test

* remove real
2025-05-14 10:36:55 -07:00
qazal
043efc6ec4
do not require self for track_rewrites [pr] (#10302) 2025-05-14 18:23:32 +03:00
uuuvn
dd816d0237
All MultiGraphRunners can graph transfers (#10301) 2025-05-14 17:23:02 +03:00
nimlgen
e00679dc92
am_smi: fix layout with sleep mode (#10300) 2025-05-14 15:44:42 +03:00
chenyu
fbaa26247a
randn_like in minrf (#10298)
tested that it trains to similar loss
2025-05-14 07:59:50 -04:00
nimlgen
0788659d08
usbgpu: fast cold boot (#10260)
* usbgpu: fast cold boot

* cleaner

* assert

* xx

* compat

* fix

* fix
2025-05-14 14:58:55 +03:00
qazal
d342f7688d
remove some skips in test_schedule + use assertRaisesRegex [pr] (#10296) 2025-05-14 14:54:07 +03:00
qazal
40f4ce3390
enable AMD CI for TestRandomness.test_multinomial [pr] (#10295) 2025-05-14 14:32:22 +03:00
nimlgen
792853b9e2
usbgpu: enable cache for compute queue (#10294) 2025-05-14 13:05:36 +03:00
nimlgen
1218fc2230
usbgpu: enable cache for 64bit addresses (#10293) 2025-05-14 12:37:39 +03:00
qazal
1770e00c41
only CAPTURE_PROCESS_REPLAY=1 + add filterwarnings back [pr] (#10292) 2025-05-14 11:58:42 +03:00
qazal
1c97338be5
enable process replay assert for schedule [pr] (#10280)
* enable process replay assert for schedule

* start at unique+1
2025-05-14 11:10:47 +03:00
George Hotz
f1130ab3d3
openpilot benchmark test (#10290)
* openpilot benchmark test

* that
2025-05-13 22:49:28 -07:00
uuuvn
f726f79a9e
Remote multi (transfer) (#10285) 2025-05-13 18:26:32 -07:00
uuuvn
7bc4864bc4
Make dev a property of Allocator (#10286)
* Make `dev` a property of `Allocator`

(this is a prereq refactor for #10285)

At least `BufferXfer.copy` accesses it assuming it's always present,
currently most devices just add this property on their own repeating
the same code over and over again.

This is also a bit footguny, see `RemoteAllocator` that named this
property `device` instead of `dev`, i could obviously just change that
in one place but doing it globally seems like a better solution (and it
reduces code duplication too).

`MallocAllocator` is a bit special, but passing `None` works just fine.

* typing

* ignore type instead of cast
2025-05-13 17:01:01 -07:00
George Hotz
ec46f658d7
openpilot llvm test [pr] (#10288) 2025-05-13 16:51:41 -07:00
uuuvn
453b268342
Factor out remote connection and cache it (#10282)
Should be a small speed improvement but the main reason this is needed
is to have a defined ordering of RemoteRequests within one host so that
transfers won't required doing something like:
```python
src_dev.batch_submit()
dest_dev.q(Transfer(dest, src_dev.session, src))
dest_dev.batch_submit()
```
for correctness.
2025-05-13 15:02:06 -07:00
uuuvn
ddff9857b8
Remote properties is a dataclass (#10283)
Not strictly required for anything but soon there will be like 4 new
properties and having it be a huge json just seems like a bad taste.

It also seems right to not have a separate endpoint for this, just
`GetProperties` request that returns a repr of this similar to how
requests are sent in `BatchRequest`.

This will also make a switch to anything other than http much simpler
if it will be required for any reason, like just a tcp stream of
`BatchRequest`s
2025-05-13 11:56:58 -07:00
uuuvn
ba87eca0f1
Remote multi (basic) (#10269)
* Basic remote multi support

Simplest thing to be able to use remote with multiple gpus, very slow
because no transfers (copyin copyout for cross-device copies)

* tests
2025-05-13 09:52:47 -07:00
George Hotz
5f64bbc63d
improve multi tests + add support for fixedvars [pr] (#10281)
* improve multi tests + add support for fixedvars [pr]

* add support for fixedvars
2025-05-13 09:27:00 -07:00