chenyu 22d5def113 download llama3 70B (#7868) ... use "nvidia/Llama-3.1-Nemotron-70B-Instruct-HF". ``` PYTHONPATH=. JITBEAM=2 python3 examples/llama3.py --download_model --size 70B --quantize int8 --benchmark ``` on M4 Max, 40 sec to load the model and ``` enqueue in 165.15 ms total 328.54 ms, 3.04 tok/s, 247.46 GB/s, param 221.20 GB/s enqueue in 5.31 ms total 168.48 ms, 5.94 tok/s, 482.54 GB/s, param 431.34 GB/s enqueue in 5.32 ms total 168.77 ms, 5.93 tok/s, 481.71 GB/s, param 430.60 GB/s enqueue in 5.69 ms total 169.51 ms, 5.90 tok/s, 479.61 GB/s, param 428.72 GB/s enqueue in 5.41 ms total 168.60 ms, 5.93 tok/s, 482.20 GB/s, param 431.04 GB/s enqueue in 5.18 ms total 168.98 ms, 5.92 tok/s, 481.12 GB/s, param 430.08 GB/s enqueue in 5.43 ms total 168.82 ms, 5.92 tok/s, 481.59 GB/s, param 430.49 GB/s enqueue in 5.27 ms total 168.94 ms, 5.92 tok/s, 481.23 GB/s, param 430.17 GB/s ```		2024-11-23 12:18:31 -05:00
..
conversation_data	Whisper + LLAMA + VITS (#2332)	2023-12-02 15:03:46 -08:00
llm.c	Remove UnaryOps, BinaryOps, TernaryOps, MetaOps [pr] (#7725)	2024-11-16 20:56:56 +08:00
mlperf	view doesn't have buffer, fix the tests [pr] (#7841)	2024-11-22 20:41:55 +08:00
openpilot	add test for compile3 [pr] (#7783)	2024-11-19 19:26:51 +08:00
other_mnist	beautiful_mnist in torch	2024-07-14 11:09:58 -07:00
rl	more beautiful_cartpole with exposed hparams	2024-01-07 17:41:09 -08:00
sovits_helpers	combine pad2d with pad (#7677)	2024-11-14 17:56:02 +08:00
tinychat	tinychat ui +/- 20 lines (#7471)	2024-11-06 14:23:55 +08:00
vgg7_helpers	move to new cached fetch (#2493)	2023-11-28 17:36:55 -08:00
webgl/yolov8	webgl backend in extra (#3041)	2024-01-08 09:29:13 -08:00
webgpu/stable_diffusion	s/lazydata.realized/lazydata.base.realized/g (#2914)	2023-12-22 14:45:13 -05:00
__init__.py	failing llama test	2023-03-11 16:28:10 -08:00
beautiful_cartpole.py	tinytqdm.set_description and tinytrange (#5101)	2024-06-22 14:45:06 -04:00
beautiful_cifar.py	Fix mypy examples/beautiful_*.py (#6978)	2024-10-10 11:34:29 -04:00
beautiful_mnist.py	added beautiful fashion mnist and example (#6961)	2024-10-10 12:01:07 +08:00
beautiful_mnist_multigpu.py	Fix mypy examples/beautiful_*.py (#6978)	2024-10-10 11:34:29 -04:00
coder.py	apply the same fix_bf16 in llama and coder (#3789)	2024-03-17 21:25:24 -04:00
compile_efficientnet.py	webgl backend in extra (#3041)	2024-01-08 09:29:13 -08:00
compile_tensorflow.py	fix various examples (#4691)	2024-05-22 20:43:21 -04:00
conversation.py	fix conversation.py quantize (#4663)	2024-05-20 17:36:37 -04:00
efficientnet.py	remove clang program header (#4422)	2024-05-04 08:38:01 -07:00
flux1.py	flux set model path in args (#7660)	2024-11-12 22:11:40 -05:00
flux1_seed0.png	Flux.1 (#6334)	2024-09-24 10:08:04 +08:00
gpt2.py	really not using numpy in gpt2 example (#7779)	2024-11-18 23:21:16 -05:00
handcode_opt.py	s/UOps/Ops (#7500)	2024-11-03 11:26:10 +08:00
hlb_cifar10.py	combine pad2d with pad (#7677)	2024-11-14 17:56:02 +08:00
index.html	Enable Multi-Output Export (#2179)	2023-10-30 18:42:26 -07:00
llama.py	remove numpy from gpt2 and llama examples (#7778)	2024-11-18 22:48:17 -05:00
llama3.py	download llama3 70B (#7868)	2024-11-23 12:18:31 -05:00
mamba.py	prev speed improvements (#5252)	2024-07-03 09:06:01 -07:00
mask_rcnn.py	change Tensor.stack to method (#4719)	2024-05-24 17:04:19 -04:00
mixtral.py	tinytqdm.set_description and tinytrange (#5101)	2024-06-22 14:45:06 -04:00
mnist_gan.py	tinytqdm.set_description and tinytrange (#5101)	2024-06-22 14:45:06 -04:00
openelm.py	nn.RMSNorm (#5272)	2024-07-02 21:39:01 -04:00
sdv2.py	Stable Diffusion v2 Inference (#5283)	2024-07-03 22:47:10 -04:00
sdxl.py	sdxl gen fix (#7459)	2024-11-01 13:57:01 -04:00
sdxl_seed0.png	default threefry (#6116)	2024-09-25 17:45:13 +08:00
serious_mnist.py	combine pad2d with pad (#7677)	2024-11-14 17:56:02 +08:00
simple_conv_bn.py	fix various examples (#4691)	2024-05-22 20:43:21 -04:00
so_vits_svc.py	combine pad2d with pad (#7677)	2024-11-14 17:56:02 +08:00
stable_diffusion.py	fix wino conv output dtype for half inputs (#7829)	2024-11-21 12:13:54 -05:00
stable_diffusion_seed0.png	default threefry (#6116)	2024-09-25 17:45:13 +08:00
stunning_mnist.py	stunning_mnist [run_process_replay] (#6828)	2024-10-01 15:00:48 +08:00
train_efficientnet.py	tinytqdm.set_description and tinytrange (#5101)	2024-06-22 14:45:06 -04:00
train_resnet.py	move things, clean up extra (#2292)	2023-11-13 20:18:40 -08:00
transformer.py	fix onehot and jit in examples/transformer (#3073)	2024-01-10 02:22:41 -05:00
vgg7.py	waifu2x vgg7: testcase, auto-RGBA->RGB, function to grab pretrained models, training "fix" (#2117)	2023-10-19 22:07:15 -07:00
vit.py	move to new cached fetch (#2493)	2023-11-28 17:36:55 -08:00
vits.py	docs: showcase remove mnist_gan and add conversation.py (#4757)	2024-05-28 11:09:26 -04:00
whisper.py	enable whisper batch for long sequences (#6458)	2024-09-17 00:42:10 -04:00
yolov3.py	Update yolov3.py (#2680)	2023-12-08 12:59:38 -08:00
yolov8-onnx.py	[ready] Replacing os with pathlib (#1708)	2023-08-30 10:41:08 -07:00
yolov8.py	combine pad2d with pad (#7677)	2024-11-14 17:56:02 +08:00