Instruction-Tuning-Survey

This repository contains resources referenced in the paper Instruction Tuning for Large Language Models: A Survey.

If you find this repository helpful, please cite the following:

@article{zhang2023instruction,
  title={Instruction Tuning for Large Language Models: A Survey},
  author={Zhang, Shengyu and Dong, Linfeng and Li, Xiaoya and Zhang, Sen and Sun, Xiaofei and Wang, Shuhe and Li, Jiwei and Hu, Runyi and Zhang, Tianwei and Wu, Fei and others},
  journal={arXiv preprint arXiv:2308.10792},
  year={2023}
}

Table of Contents

• Overview
• Natural Language Instruction Tuning
  • Datasets
  • Models
• Multi-modality Instruction Tuning
  • Datasets
  • Models
• Domain-specific Instruction Tuning

Overview

Instruction tuning (IT) refers to the process of further training large language models (LLMs) on a dataset consisting of (instruction, output) pairs in a supervised fashion, which bridges the gap between the next-word prediction objective of LLMs and the users' objective of having LLMs adhere to human instructions. The general pipeline of instruction tuning is shown in the following:

In the paper, we make a systematic review of the literature, including the general methodology of IT, the construction of IT datasets, the training of IT models, and applications to different modalities, domains and application, along with analysis on aspects that influence the outcome of IT (e.g., generation of instruction outputs, size of the instruction dataset, etc). We also review the potential pitfalls of IT along with criticism against it, along with efforts pointing out current deficiencies of existing strategies and suggest some avenues for fruitful research. The typology of the paper is as follows:

Natural Language Instruction Tuning

Datasets

Type	Dataset Name	Paper	Project	# of Instructions	# of Tasks	# of Lang	Construction	Open Source
Generalize to unseen tasks	UnifiedQA	paper	project	750K	46	En	human-crafted	Yes
	OIG	-	project	43M	30	En	human-model-mixed	Yes
	UnifiedSKG	paper	project	0.8M	-	En	human-crafted	Yes
	Natural Instructions	paper	project	193K	61	En	human-crafted	Yes
	Super-Natural Instructions	paper	project	5M	76	55 Lang	human-crafted	Yes
	P3	paper	project	12M	62	En	human-crafted	Yes
	xP3	paper	project	81M	53	46 Lang	human-crafted	Yes
	Flan 2021	paper	project	4.4M	62	En	human-crafted	Yes
	COIG	paper	project	-	-	-	-	Yes
Follow users' instructions in a single turn	InstructGPT	-	-	13K	-	Multi	human-crafted	No
	Unnatural Instructions	paper	project	240K	-	En	InstructGPT-generated	Yes
	Self-Instruct	paper	project	52K	-	En	InstructGPT-generated	Yes
	InstructWild	-	project	104K	429	-	model-generated	Yes
	Evol-Instruct	paper	project	52K	-	En	ChatGPT-generated	Yes
	Alpaca	-	project	52K	-	En	InstructGPT-generated	Yes
	LogiCoT	paper	project	-	2	En	GPT-4-generated	Yes
	Dolly	-	project	15K	7	En	human-crafted	Yes
	GPT-4-LLM	paper	project	52K	-	En&Zh	GPT-4-generated	Yes
	LIMA	paper	project	1K	-	En	human-crafted	Yes
Offer assistance like humans across multiple turns	ChatGPT	-	-	-	-	Multi	human-crafted	No
	Vicuna	-	project	70K	-	En	user-shared	No
	Guanaco	-	project	534,530	-	Multi	model-generated	Yes
	OpenAssistant	paper	project	161,443	-	Multi	human-crafted	Yes
	Baize v1	paper	project	111.5K	-	En	ChatGPT-generated	Yes
	UltraChat	paper	project	675K	-	En&Zh	model-generated	Yes

Models

Model Name	# Params	Paper	Project	Base Model	Instruction Train Set
					Self-build	Name	Size
Instruct-GPT	176B	paper	-	GPT-3	Yes	-	-
BLOOMZ	176B	paper	project	BLOOM	No	xP3	-
		paper	project
FLAN-T5	11B	paper	project	T5	No	FLAN 2021	-
Alpaca	7B	-	project	LLaMA	Yes	-	52K
Vicuna	13B	-	project	LLaMA	Yes	-	70K
GPT-4-LLM	7B	paper	project	LLaMA	Yes	-	52K
Claude	-	-	-	-	Yes	-	-
WizardLM	7B	paper	project	LLaMA	Yes	Evol-Instruct	70K
ChatGLM2	6B	-	project	GLM	Yes	-	1.1 Tokens
LIMA	65B	paper	project	Yes	-	1K
OPT-IML	175B	paper	project	OPT	No	-	-
Dolly 2.0	12B	-	project	Pythia	No	-	15K
Falcon-Instruct	40B	-	project	Falcon	No	-	-
Guanaco	7B	-	project	LLaMA	Yes	-	586K
Minotaur	15B	-	project	Starcoder Plus	No	-	-
Nous-Hermes	13B	-	project	LLaMA	No	-	300K+
TÜLU	6.7B	paper	project	OPT	No	Mixed	-
YuLan-Chat	13B	-	project	LLaMA	Yes	-	250K
MOSS	16B	-	project	-	Yes	-	-
Airoboros	13B	-	project	LLaMA	Yes	-	-
UltraLM	13B	paper	project	LLaMA	Yes	-	-

Multi-modality Instruction Tuning

Datasets

Dataset Name	Paper	Project	Modalities		# Tasks
			Modality Pair	# Instance
MUL-TIINSTRUCT	paper	project	Image-Text	5K to 5M per task	62
PMC-VQA	paper	project	Image-Text	227K	9
LAMM	paper	project	Image-Text	186K	9
			Point Cloud-Text	10K	3

Models

Model Name	# Params	Paper	Project	Modality	Base Model		Train set
					Model Name	# Params	Self-build	Size
InstructPix2Pix	983M	paper	project	Image-Text	Stable Diffusion	983M	Yes	450K
LLaVA	13B	paper	project	Image-Text	CLIP	400M	Yes	158K
					LLaMA	7B
					LLaMA	7B
Video-LLaMA	-	paper	project	Image-Text-Video-Audio	BLIP-2	-	No	-
					ImageBind	-
					Vicuna	7B/13B
InstructBLIP	12B	paper	project	Image-Text-Video	BLIP-2	-	No	-
Otter	-	paper	project	Image-Text-Video	OpenFlamingo	9B	Yes	2.8M
MultiModal-GPT	-	paper	project	Image-Text-Video	OpenFlamingo	9B	No	-

Domain-specific Instruction Tuning

Domain	Model Name	# Params	Paper	Project	Base Model	Train Size
Medical	Radiology-GPT	7B	paper	project	Alpaca	122K
	ChatDoctor	7B	paper	project	LLaMA	122K
	ChatGLM-Med	6B	paper	project	ChatGLM	-
Writing	Writing-Alpaca	7B	paper	project	LLaMa	-
	CoEdIT	11B	paper	project	FlanT5	82K
	CoPoet	11B	paper	project	11B	-