mirrors/tinygrad
2
0
Fork 0
mirror of https://github.com/tinygrad/tinygrad.git synced 2026-06-24 02:14:17 +00:00
Code Issues Projects Releases Packages Wiki Activity

Fix: missing n_kv_heads for smaller models from huggingface (#2438)

Browse source 
* fix: missing n_kv_heads for smaller models from huggingface

* a lil golfing
This commit is contained in:
Davi Silva 2023-11-26 01:29:04 +07:00 • committed by GitHub
commit df41a57e09
No known key found for this signature in database
GPG key ID: 4AEE18F83AFDEB23
1 changed files with 1 additions and 1 deletions
Download patch file Download diff file Expand all files Collapse all files

2
examples/llama.py
View file

@ -311,7 +311,7 @@ class LLaMa:
else:
weights = load(str(model_path))
if "model.embed_tokens.weight" in weights:
weights = convert_from_huggingface(weights, model, model_args["n_heads"], model_args["n_kv_heads"])
weights = convert_from_huggingface(weights, model, model_args["n_heads"], model_args.get("n_kv_heads", model_args["n_heads"]))
if quantize:
weights = AbsmaxQuantizedLinear.quantize(weights)