mirrors/tinygrad
2
0
Fork 0
mirror of https://github.com/tinygrad/tinygrad.git synced 2026-06-24 02:14:17 +00:00
Code Issues Projects Releases Packages Wiki Activity
1,995 commits 280 branches 15 tags 325 MiB
Commit graph

199 commits

Author SHA1 Message Date
Diogo
d2b837c1d9
 		Adds floor/ceil (#989) 
* floor ceil impl

* control casting in numpy
 2023-06-17 10:56:21 -07:00
George Hotz
ba56ee6020
 		RDNA assembly backend ($1000 bounty) (#787) 
* Revert "Revert "ops rdna""

This reverts commit 0400315078.

* Revert "Revert "writing 2""

This reverts commit 325a3bf2cf.

* no dump

* 2x 2

* simple asm

* local size

* sub

* lil work

* support args != 3

* assembler work

* generate that

* ptx assembler

* begin index renderer

* max

* ptx loops

* gemms work

* valid works

* asm working a bit more

* close

* passing all ops tests

* ptx is a codegen only, not a backend

* ptx

* float16 support

* rdna goes here

* install types

* make amd disassemble

* ansilen for pretty print

* fix ptx log2/exp2

* assemblyinstruction

* new asm

* working gemm

* fix cmp

* more passing

* mod

* ptx works again

* rdan3 add works

* log exp

* sin is sin 2pi

* fix types

* progress

* loops work

* rdna xyz

* better addressing

* cleanups

* handle exception in early process

* div support

* rdna float4

* locals work

* fix neg index

* cast

* smaller diff

* yaml

* import only if selected

* fromimport

* types

* this all needs rewriting

* a few more
 2023-06-16 09:33:18 -07:00
George Hotz
80e665bddb a couple new tests 2023-06-13 12:36:05 -07:00
Diogo
2d4370b487
 		Adds tril & triu support (#936) 
* triu & tril support

* lint and kernel count error

* switched shape indicies

* larger shape tests

* reverted numpy removal until #942 is resolved
 2023-06-09 22:13:20 -07:00
George Hotz
48e9461197 broken tests for #862 and #942 2023-06-09 22:02:59 -07:00
cloud11665
43ea1614b0
 		fix inf/nan codegen (#935) 
* fix inf/nan codegen

* remove nasty oneliner, fix -inf

* inf/nan const mul/div tests
 2023-06-05 11:24:09 -07:00
Filip Dimitrovski
78460034ff
 		Initial ellipsis support when slicing Tensors (#843) 
* Initial ellipsis support when slicing Tensors

* Better comments in ellipsis slicing

* Formatting
 2023-06-05 07:52:49 -07:00
Tom Edwards
5bbcbd145c
 		Add cumsum with n-dim inputs (#922) 
* add cumsum with n-dim inputs, over arbitrary axis + relevant tests

* increased rtol for cumsum test

* move test_cumsum into test_ops

* skip arange test for images as relies on cumsum

* Fix typo

* rewrite cumsum to work with images
 2023-06-04 16:55:23 -07:00
Alexey Zaytsev
5feee9c94b
 		Fix .std() tests on torch=1.13 (#904) 2023-06-02 07:33:51 -07:00
SnakeOnex
67a7674787
 		added conv1d tests -> simple, padding, stride, asymmetric padding (#896) 
* added conv1d tests -> simple, padding, stride, asymmetric padding

* fixed linting

* skip conv1d tests for images
 2023-06-01 13:10:37 -07:00
Joqsan
ef129bcb85
 		Zero dim Tensor support (#777) 
* add and reorganize test_slice_* tests

* refactor Tensor.__getitem__()

* preliminary tests for 1) 0D tensors and 2) varargs for Tensor.zeros and Tensor.ones

* always compare shapes of the numpy arrays obtained from tinygrad and torch tensors

* add more tests for 0D support

* remove test_tensor.test_slicing(). All slicing tests at test/test_ops.py

* add zero-dim support

* make test_end2end.py consistent with 0dim support

* add test for tensor with zero in shape

* don't simplify ones if shape is ()

* skip tests that need zero-size tensor support.

- zero-size tensor support not related to 0dim tensors.

* add tests for __getitem__() supporting strides >= 1

* refactor __getitem__: support for strides >= 1

* minor refactors and add comments to __getitem__

* add tests for slices with negative steps

* add support for slices with negative strides
 2023-06-01 11:32:02 -07:00
kposborne2
ae83e9844c
 		add output_padding to transposed conv (#875) 2023-06-01 00:03:22 -07:00
Tom Edwards
115903a15c
 		Add unbiased std and corresponding tests (#881) 
* add unbiased std and corresponding tests

* replaced unbiased with correction + tests
 2023-05-31 16:32:36 -07:00
Ubaidullah Khan
502e33652f
 		add Tensor.full and Tensor.full_like and reuse them (#852) 
* add Tensor.ones_like()

* add full_like and full and reuse in zeros,ones

* add tests for full and full_like
 2023-05-29 17:48:09 -07:00
Marcello Fuschi
6ea5df19b2
 		Fix conv_transpose2d asymmetric padding (#840) 2023-05-29 07:57:06 -07:00
George Hotz
59f9bcd4a4
 		Disktensors! (#819) 
* make empty a real thing

* start ops_disk

* disk tensor works

* interpreted cleanup

* slice write to disk

* preprocess imagenet

* fix custom function
 2023-05-28 15:40:37 -07:00
Marcello Fuschi
6d49925a26
 		Add max_pool2d dilation (#833) 2023-05-28 15:16:48 -07:00
Kirill R
081b3ab639
 		Tensor.where method (#830) 2023-05-28 10:20:33 -07:00
Kirill R
0c0c7380af
 		Add Tensor.where (#826) 
* Add Tensor.where

* fix linter

* fix mypy
 2023-05-28 08:04:56 -07:00
kposborne2
2163a1b049
 		Add shrink step to fix strided conv_transpose2d, and add to nn (#823) 
* implement conv transpose 2d

* don't inherit, remove old assert

---------

Co-authored-by: Kyle <kposborne@gmail.com>
 2023-05-28 07:52:45 -07:00
George Hotz
26014a0fa1
 		add convtranspose (#809) 
* add convtranspose

* onnx convtranspose
 2023-05-26 12:35:03 -07:00
Aneesh Durg
6d4a728f62
 		Don't collapse dimensions during batched matmul (FIX #799) (#800) 
* Don't collapse dimensions during batched matmul (FIX #799)

* Avoid reshaping tensor to the same shape

* Skip batched matrix multiply when IMAGE is set
 2023-05-26 11:15:34 -07:00
Diogo
c19ef0fcce
 		Add sin/cos/tan (#794) 
* added sin/cos/tan

* fix lint

* added onnx ops support
 2023-05-25 09:04:56 -07:00
Rabia Eda Yılmaz
e5b4b36cba
 		add std to tensor.py (#767) 
* add std

* delete comment

* edit: one liner std, add: test

* adjust

* fix: shape mismatch

* set unbiased to False

* added unbiased option

* fix unbiased option in test and clean code

* better

* generalize axis

* holly coffee molly

* generalize axes without unbiased opt.

* hopefully done

* complete unbiased true for axes

* Update test_ops.py

* fixed

* std completed without bessels correction

* fix comment

* ups
 2023-05-13 12:20:44 -07:00
George Hotz
810f03dafa
 		conv3d + unet3d (#772) 
* conv3d, needs test

* test passes, padding wrong on unet

* unet3d

* no conv3d on images
 2023-05-12 13:54:07 -07:00
Joqsan
0b9d4126d0
 		Add Tensor.stack() and Tensor.repeat() (...trying to make einops work with tinygrad) (#758) 
* add stack() and repeat() methods

* make stack a static method
 2023-05-01 09:37:46 -07:00
George Hotz
3d15769a8f 50 TFLOPS cuda matmul 2023-04-19 14:38:24 -07:00
George Hotz
133521e730 relu UnaryOp is back 2023-04-14 07:12:53 -07:00
worldwalker2000
552a048a33
 		make maximum split the grad like torch when equal (#738) 
* make maximum split grad

* added test for maximum split grad when equal

* minor expr simplification

* (2-eq)/2 only once

* update test bc one more sum output child stays
 2023-04-14 00:17:46 -07:00
Andre Slavescu
39d6e1525f
 		Added activation ops + tests (#729) 
* activation ops

* type hints + more testing

* formatting correction + parameter testing

* fixes to shape testing

* hardtanh to use clip + removed type hints

* assign val fix
 2023-03-28 13:17:53 +04:00
George Hotz
b12b60af20
 		fix binop, other tests failure (#723) 
* fix binop, other tests failure

* that was a bad idea

* better layernorm

* inference kernel count tests

* new style reshape pushing

* fixup replacement

* 199 kernels is okay. fix flops

* push reshape through unaryops only

* GRAPH=2 draws the phantom ops

* found resnet issue

* non working test

* mul is cheaper than div

* OPT inflation

* SHUFFLE_PAD_OPS in OPT=2
 2023-03-22 18:15:07 -07:00
George Hotz
5495c7d64e
 		linearizer! (#714) 
* linearizer outputs something

* working ish

* cstyle codegen

* clang mostly works

* fix load valid

* fix numberless loop

* fancy gen

* working

* fix enet compiler

* cleanups

* float4 upcasting

* less lines

* supports_float4

* constant folding

* mulacc

* internet tests flaky in CI

* 90% image support

* fix image generic

* bugs exposed with shapetracker and single view

* new llvm

* use vload, remove OLD

* that's really poorly done

* ending up being more lines
 2023-03-19 23:43:49 -07:00
George Hotz
902906f909
 		Fix constant folding (#713) 
* fix

* codegen

* contiguous is real

* no bufs_to_delete

* don't assign rawconst

* remove neg and not

* need exec to fix custom function jit
 2023-03-18 17:52:46 -07:00
George Hotz
f5467cfedc
 		Devicebufferless (#708) 
* runs one metal kernel

* conv2d works

* ops tests are passing

* const folding

* all ops work

* pre commit always passes

* torch works

* working still

* fix graph test

* tests passing

* image almost works

* image conv works

* most images

* fix custom

* fix assignment

* fix compile enet

* clean up comments

* fix realize return value

* include shapetracker in LB repr

* copy should make a copy

* reenable method cache

* fix lna

* dtypes in graph

* forward only for IMAGE=2

* simple realize

* getting close

* fixup new api, it's good except the kernel count

* back to 197 kernels

* tests should pass

* go to a real float

* no type_on_cpu

* fix the docs

* put shapetracker back in it's proper place
 2023-03-18 14:40:23 -07:00
George Hotz
c594a0a835 fix flip bug, add new unit tests 2023-03-12 23:55:31 -07:00
George Hotz
61071f881a fix bug, and add unit test to catch failure 2023-03-11 16:57:25 -08:00
Diogo
784afc6c6f
 		Eq magic function support (#683) 
* add eq magic func

* changed from eq to __eq__

* ignore type for linter

* mypy doenst like descriptions :(
 2023-03-11 10:31:46 -08:00
George Hotz
0b03216cc3
 		losing lines (#678) 
* losing lines

* FLIP -> STRIDE

* shapetracker refactor
 2023-03-10 21:57:05 -08:00
George Hotz
1a039306d2
 		good changes from llama branch (#671) 
* good changes from llama

* transpose behavior changed
 2023-03-09 20:51:22 -08:00
George Hotz
e0244baf60 3 letters for graph op 2023-03-07 19:20:48 -08:00
George Hotz
382f346523
 		clean up opt (#649) 
* clean up opt

* don't let global kernels get too small

* 8192 -> 1024

* disable local shape for clang

* fix can_merge

* unroll the 5x5 depthwise convs in op

* load float4 check
 2023-03-05 20:49:36 -08:00
George Hotz
0335cb86b9 refactor comparison. there's a bug in the method cache 2023-03-02 10:10:16 -08:00
George Hotz
11f257ddf9 clean up cmp tests 2023-03-02 09:35:16 -08:00
Diogo
52204a7b88
 		adding comparison operators (#616) 
* Less, LessOrEqual, Greater, GreaterOrEqual, Equal

* lint fix

* using built in functions

* overriding __eq__ breaks things

* backwards pass for less - foward only tests

* one other spot

* removing backwards for comparison ops to match pytorch

* raise runtime error

* more tests for comparison ops

* fixed the lineup

* added number upcast tests
 2023-03-02 08:10:44 -08:00
George Hotz
e9e71fbfc4
 		remove mlop (#619) 
* remove mlop

* lil simpler
 2023-02-28 17:58:24 -08:00
George Hotz
4c4d88aad4 fix the last bug, and make HLOP the default 2023-02-28 17:04:28 -08:00
George Hotz
686a74de92 fast zeros and ones 2023-02-27 06:46:26 -08:00
George Hotz
8b96522e1d instant identity removal 2023-02-25 09:46:04 -08:00
George Hotz
2c5e13a513
 		Reluless (#600) 
* replace relu for maximum

* fix for other backend

* clean up RELU and GT0

* tests for maximum

* had to clean that up

* why reverse a maximum?
 2023-02-25 01:21:16 -08:00
George Hotz
f8f026e8bb oversized expand for HLOP convs 2023-02-24 21:48:47 -08:00