8 commits

Author	SHA1	Message	Date
qazal	390171d686	delete SAVE_SCHEDULE=1 [pr] (#7087)	2024-10-16 07:13:20 +03:00
Elias Wahl	4a114756f6	New BERT dataloader (#5881) ... * One file == One topic * update test * new dataloader * update train script * get index is faster	2024-08-02 15:12:23 -04:00
Elias Wahl	097268fab3	Add layerwise performance bench for bert (#5349) ... * add bert bench * dont disable by defauöt * remove lr * linter	2024-07-09 15:03:25 -04:00
Elias Wahl	04e237328b	Refactor to class style (#4804)	2024-06-04 14:08:31 -07:00
chenyu	b00b6b16f0	fix TRAIN_BEAM and Tensor.training for mlperf bert (#4525) ... also hard coded bert model config instead of looking up a file	2024-05-11 00:18:36 -04:00
Elias Wahl	babe87a8ae	BERT: Checkpoint loading tests (#4359) ... * Move checkpoint init to helpers. Add test * linters * Move the steps outside of the main train loop * Move data_get * data_get belongs to helpers	2024-04-30 14:43:41 -04:00
Elias Wahl	27613dd881	MLPerf BERT: Main training loop (#4288) ... * BERT language modeling head + trunc normal initializers * add train loop + helpers * shuffle in dataloaders + slight changes in main loop * beam change * Minor changes * random.shuffle * HParam update * Use deque for dataloader * wandb bert project name * half fixes * BENCHMARK + remove epoch * cast + print() --------- Co-authored-by: chenyu <chenyu@fastmail.com>	2024-04-29 14:35:27 -04:00
Elias Wahl	69341144ba	Wikipedia preprocessing script (#4229) ... * Preprocessing script * short seq prob * comments + env vars * Add preprocessing reference. Add test * lint fix + add eval test support * whitespaces * point to commit * comment * rename * better comments	2024-04-23 10:28:01 -04:00