tinygrad/examples/mlperf
George Hotz 55d3a5def9
preallocate all realized buffers (#14823)
* preallocate all realized buffers

* contiguous

* work

* comment that out

* move to schedule

* better

* correct fix

* just buffer

* disk bufs

* fixes disk tensor stuff

* fix symbolic stuff

* fix multi

* 162 failures

* bugfixes

* don't check that anymore

* fix schedule tests

* mnist should be contiguious

* type and buffer

* fix tests

* shrink axis correction

* mypy fixes

* tests skips

* same 37 failures

* dedup

* no shrink in the graph

* 29 failures

* skips

* fix custom kernel

* fix training

* those optimizations aren't supported currently

* simpler

* more correct

* tests

* 14 failures

* works

* fix that test

* broken

* 11 failures

* only kernel counts left

* fixes

* all tests pass

* remove tensor_map

* op test

* 200 -> 230

* test fixes

* fixes

* revert test_tiny thing

* guard

* revert that

* test tiny passes

* no contigs there

* base realize back

* Revert "no contigs there"

This reverts commit c45bb9fcfd.

* revert that

* chop many assigns

* 12 failures

* fix tests

* tests

* apply after

* pre-commit

* remove old code

* delete that

* fix types

* remove extra contig

* fix dataloader

* torch fix

* disk fix

* update kernel fusion numbres

* runs on amd

* restore kernel count

* add that rule back

* that

* disable that

* wrong

* add the correct rule for that folding

* more tests

* guard c1.arg

* no newlines

* realize those

* split into a different file

* remove detach/contig back

* skip 2

* update that
2026-02-20 20:05:54 +08:00
..
scripts download data and ckpts for sd train/eval (#12170) 2025-09-15 00:31:45 -04:00
training_submission_v4.0/tinycorp copy mlperf 4.0 to mlperf 4.1 (#5614) 2024-07-20 16:12:00 -04:00
training_submission_v4.1/tinycorp update mlperf systems and copy 4.1 to 5.0 (#7004) 2024-10-11 16:20:34 -04:00
training_submission_v5.0/tinycorp Stable Diffusion mlperf training (#11304) 2025-10-05 07:56:05 -04:00
training_submission_v5.1/tinycorp remove FUSE_ARANGE_UINT (#11567) 2025-08-07 16:49:06 -04:00
training_submission_v6.0/tinycorp feat: llama wqkv (#14841) 2026-02-17 23:01:33 -08:00
dataloader.py preallocate all realized buffers (#14823) 2026-02-20 20:05:54 +08:00
helpers.py train bert with fp8 (#13874) 2026-01-09 09:21:59 -05:00
initializers.py remove contiguous and use where in EmbeddingBert (#13632) 2025-12-09 15:49:21 -05:00
losses.py cleanups on losses and dataset tests (#9538) 2025-03-21 17:03:18 -04:00
lr_schedulers.py LR scheduler for Stable Diffusion mlperf training (#12201) 2025-09-30 21:21:08 -04:00
metrics.py log_perplexity metrics (#10912) 2025-06-21 10:44:47 -04:00
model_eval.py update llama dataloader (#13825) 2025-12-24 17:42:08 -05:00
model_spec.py remove Tensor.no_grad, it's meaningless now [pr] (#10556) 2025-05-28 22:20:02 -07:00
model_train.py seperate llama optim (#14810) 2026-02-17 13:02:35 -08:00
optim.py seperate llama optim (#14810) 2026-02-17 13:02:35 -08:00
README start on mlperf models 2023-05-10 16:30:49 -07:00

Each model should be a clean single file.
They are imported from the top level `models` directory

It should be capable of loading weights from the reference imp.

We will focus on these 5 models:

# Resnet50-v1.5 (classic) -- 8.2 GOPS/input
# Retinanet
# 3D UNET (upconvs)
# RNNT
# BERT-large (transformer)

They are used in both the training and inference benchmark:
https://mlcommons.org/en/training-normal-21/
https://mlcommons.org/en/inference-edge-30/
And we will submit to both.

NOTE: we are Edge since we don't have ECC RAM