mirrors/tinygrad
2
0
Fork 0
mirror of https://github.com/tinygrad/tinygrad.git synced 2026-06-24 02:14:17 +00:00
Code Issues Projects Releases Packages Wiki Activity
2,878 commits 280 branches 15 tags 325 MiB
Commit graph

69 commits

Author SHA1 Message Date
Davi Silva
df41a57e09
 		Fix: missing n_kv_heads for smaller models from huggingface (#2438) 
* fix: missing n_kv_heads for smaller models from huggingface

* a lil golfing
 2023-11-25 10:29:04 -08:00
George Hotz
5bb720a777 Cocoa is no longer used 2023-11-23 14:31:21 -08:00
George Hotz
2dec86970a hotfix: default remains gen 1 llama 2023-11-21 14:43:02 -08:00
Oleg Rybalko
7220f5c9fc
 		fixed hf convert and now it's working with tinyllama (#2374) 
* fixed hf convert and now it's working with tinyllama

* added tinyllama config

* refactored code and made it work with all llama models

* prettier order

* prettier order

* fixed suffix for tinyllama and refactored convert_from_hf

* dynamically update help if MODEL_PARAMS changes and default size is the 1st
 2023-11-21 14:36:52 -08:00
Friedrich Carl Eichenroth
75676ab8e1
 		Profiling-helper (#2321) 
* change profiler

* remove unused imports

* remove unused imports

* change lazybuffer references

* remove unused line

* remove unused import

* remove unused stuff

* add types

* typing

* typing

* typing

* trigger actions

* -1 loc

* fixup

* trigger actions

* revert lazy typing changes

* WIP profiler helper

* replace old start & stop profiler

* fixup

* linting

* Update llama.py

---------

Co-authored-by: George Hotz <72895+geohot@users.noreply.github.com>
 2023-11-16 14:15:56 -08:00
George Hotz
3baaf298d6
 		two stage cumsum in tensor.py (#2331) 
* two stage cumsum in tensor.py

* 2 more kernels for llama cumsum

* gpt-2 and llama use fast multinomial
 2023-11-16 12:09:53 -08:00
George Hotz
70a65c201e
 		JIT support in Interpreted (#2314) 
* factor that out

* jit is supported everywhere

* fix some tests

* there's no jit supported device, the jit is everywhere

* fix test uops
 2023-11-15 11:13:38 -08:00
George Hotz
01f8781c26
 		fix CI (#2300) 
* might work

* might work 2

* might work 3

* sneak that in to llama too

* pin them all
 2023-11-14 11:02:59 -08:00
chenyu
a72b370066
 		llama take int and convert to Variable internally (#2284) 2023-11-12 17:11:37 -05:00
chenyu
453f48ce02
 		pad None means (0,0) (#2273) 2023-11-11 09:50:26 -08:00
chenyu
880e693207
 		fix llama n_kv_heads in kvcache (#2267) 
* fix llama n_kv_heads in kvcache

* trigger ci
 2023-11-10 21:44:39 -05:00
chenyu
a753c8e071
 		examples of new GPT2 and JIT change (#2261) 
* var_vals are global

* working with global ish

* better

* fix export model

* fix tests

* better kv cache

* does it run?

* use where for kvmask

* fix excessive var_vals

* fix import

* how does multigpu use this?

* llama kinda work

* faster and simpler

* cleanup

* fix conversation mode

* test cleanups

* fix one more test

* test cleanup

---------

Co-authored-by: George Hotz <geohot@gmail.com>
 2023-11-10 15:07:02 -05:00
George Hotz
2f7aab3d13
 		move optimize_local_size (#2221) 
* move optimize_local_size

* interpret_ast
 2023-11-05 21:00:52 -08:00
chenyu
8548b20b23
 		fix codellama params and repeat_kv (#2181) 2023-10-30 10:16:26 -07:00
will
bc0829b677
 		Fix llama json loading (#2160) 2023-10-27 10:21:56 -10:00
nimlgen
e21bf776c8
 		fix debug=1 llama/gpt2 timings (#2143) 2023-10-24 15:45:00 -04:00
chenyu
e2b83f1b42
 		Variable.bind newer (#2017) 
* Variable.bind attempt 2

* ShapeTracker.unbind

* fix llama

* fix types

* test case

* View.vars cleanup

* include mask in symbolic source

* mask can be sint

* st.unbind in bufferops

* assert ast contain free Variable only

* cleanup

* conservative unbinding reduce op arg

* move reduceop unbind

* fix llama JIT arg behavior
 2023-10-10 10:03:01 -07:00
chenyu
25555c836f
 		llama default to JIT only if device supports JIT (#2028) 2023-10-09 17:26:02 -07:00
chenyu
05be57f57f
 		Fix llama with empty prompt (#1997) 
* fix llama with one token prompt

* llama is all_jitted
 2023-10-06 06:48:07 -07:00
chenyu
da2b3e55f4
 		simpler llama - don't shrink twice (#1981) 2023-10-05 14:31:46 -07:00
chenyu
ebcda8a714
 		Move var_vals from ShapeTracker to LazyBuffer (#1819) 2023-09-08 09:25:10 -07:00
Yixiang Gao
22cf15e9d0
 		convert function into tinygrad (#1803) 2023-09-06 14:41:26 -07:00
badcc
fd25792c8b
 		Ensure freqs as type float32 in freqs_cis (#1798) 2023-09-06 10:24:15 -07:00
George Hotz
fb1cc6bf4b
 		llama jit is default, print tok/sec (#1774) 
* llama jit is default, print tok/sec

* jit not default in CI
 2023-09-05 10:12:16 -07:00
Yixiang Gao
66a6bbd029
 		codellama (#1702) 
* add codellama with pre-downloaded weights

* add rope_theta, fix param

* fix test

* add 7B-Python

* add 7B-Instruct

* replace single quotes with doulbe

---------

Co-authored-by: George Hotz <72895+geohot@users.noreply.github.com>
 2023-09-02 08:45:12 -07:00
nimlgen
b5cf274da3
 		remove memory peak for quantized llama (#1720) 2023-08-30 16:32:30 -04:00
chenyu
e4eb5d55c7
 		critical realize for unjitted llama (#1718) 2023-08-30 14:52:32 -04:00
Karan Handa
a8aa13dc91
 		[ready] Replacing os with pathlib (#1708) 
* replace os.path with pathlib

* safe convert dirnames to pathlib

* replace all os.path.join

* fix cuda error

* change main chunk

* Reviewer fixes

* fix vgg

* Fixed everything

* Final fixes

* ensure consistency

* Change all parent.parent... to parents
 2023-08-30 10:41:08 -07:00
chenyu
ac183568be
 		llama JIT python runtime speedup (#1633) 
* no JIT call in TransformerBlock

* idea

* move 2 reshapes to jitted function

shrink inside jitted too, 6.3ms

remove back reshapes, 5.5ms

isinstance -> __class__ 4.99ms

* think

revert ops_gpu.py

revert symbolic.py too

PYOPENCL_COMPILER_OUTPUT=1

* cleanup

* fix cache shape for conversational model

only reshape if start_pos > 0

* small cleanup

* include var_vals.keys() to st.key

* add comments

* llama small update

* everything jitted again, similar structure to gpt2

* fix typing

* add TODO for in place update cache
 2023-08-30 07:51:05 -07:00
Olivier Chafik
ee6d8de2dc
 		Llama: load models in HuggingFace format (incl. indexed, safetensors) (#1583) 2023-08-28 15:11:40 -04:00
George Hotz
a6d842af7a
 		move device to ops (#1646) 
* move device to ops

* mlops types

* 2 lines
 2023-08-23 08:30:17 -07:00
George Hotz
d3c401ba3c llama quantize: scale uses mul, not div 2023-08-22 11:48:56 -07:00
chenyu
89e13f2f04
 		support symbols in shrink (#1611) 2023-08-22 09:08:21 -07:00
George Hotz
718ced296c
 		move state to nn/state (#1619) 2023-08-22 07:36:24 -07:00
Umut Zengin
f720682beb
 		np.argmax to Tensor.argmax (#1608) 
* to tensor argmax

* removed keepdim

* training update
 2023-08-21 15:22:29 -07:00
George Hotz
4ea00bad38 track down llama bug 2023-08-21 15:14:21 -07:00
chenyu
ae39cf84ab
 		Symbolic Shape JIT main PR (#1353) 
* Symbolic Shape JIT

update tests

2 variables symbolic ops, adding more tests

test passing

cleanup

* more test cases

* single flag

* review update

* jit attention one piece

* realize

* symbolic_jit test for cuda

* old artifact

* works with cuda gpu but failed ci

* CUDACPU
 2023-08-18 14:39:55 -07:00
wozeparrot
55d95d1658
 		llama 70b (#1558) 
* feat: llama 70b

* feat: llama 70b but simpler
 2023-08-16 11:36:12 -07:00
Jacky Lee
ef5f648e2f
 		Tensor.scaled_dot_product_attention to match torch, used in LLaMA, and tested (#1502) 
* Implement scaled_dot_product_attention and test

* Support attn_mask

* Support is_causal too

* Use in llama

* Don't forget to reshape

* Set requires_grad=False for causal

* Remove staticmethod

* Remove extra spaces
 2023-08-08 23:27:13 -07:00
chenyu
827d13e64e
 		correct patch JIT llama chat (#1500) 2023-08-08 19:52:09 -04:00
chenyu
0415a48cfc
 		patch JIT llama chat mode (#1496) 2023-08-08 15:15:56 -07:00
George Hotz
67781fcf5d fix fail fast in CI 2023-08-05 10:24:24 -07:00
Francis Lam
9d142430cb
 		Add option in llama.py to quantize weights to int8 at runtime (#1289) 
* Add option in llama.py to quantize weights to int8 at runtime

Also added lm-eval to external

* Add support for llama-2 evaluation
 2023-07-24 17:22:38 -07:00
Pavol Rusnak
cd60b8561c
 		Add LLaMA-2 support (#1284) 
Co-authored-by: wozeparrot <wozeparrot@gmail.com>
 2023-07-24 17:12:02 -04:00
Francis Lam
3db57d3118
 		Fix llama.py to load and concatenate 13B, 30B, and 65B models (#1275) 2023-07-19 13:22:33 -04:00
Stan
9b6e57eccd
 		helpers.py: improved test coverage + exception handling (#1165) 
* Fixes + improved test coverage for helpers.py

- added exception handling in `proc`, if an exception was thrown, the thread would hang
- made `_early_exec_process` catch any Exception, before if an exception was thrown before the process was started, it would hand the thread

* Made `_early_exec_process` catch any Exception

 Otherwise, if an exception was thrown before the process was started, it would hang the thread. For example a type error for an argument passed to `subprocess.check_output`

* Fixed `from tinygrad.helpers import Timing` import

oops, for some reason my IDE cleaned that import from extra/helpers.

* Fixed import in llama.py

Another one that I skipped by accident, mybad

* Extracted a class for tests of early exec

* Normalize line endings, windows uses /r/n

* Made `cross_process` not a daemon
 2023-07-07 10:26:05 -07:00
Rayan Hatout
65cbaa3429
 		no need to slice A and B twice in LLaMa complex multiplication (#1054) 2023-06-26 14:42:58 -07:00
Diogo
2d4370b487
 		Adds tril & triu support (#936) 
* triu & tril support

* lint and kernel count error

* switched shape indicies

* larger shape tests

* reverted numpy removal until #942 is resolved
 2023-06-09 22:13:20 -07:00
George Hotz
ed1963b899
 		Fast DiskTensor to other Tensor (#916) 
* make disktensors fast

* loading

* loader for sd and llama
 2023-06-03 12:25:41 -07:00
George Hotz
d58586bb17
 		safetensors! (#903) 
* safetensors test

* safe_save

* load back with real safetensors

* bugfix in device name. add simple torch_load

* it works for llama, but it's slower...

* mmap

* no intermediate

* load mmaped

* readinto speed

* not ready yet

* revert that
 2023-06-02 13:41:09 -07:00