// the find
teknium1/GPTeacher
A collection of modular datasets generated by GPT-4, General-Instruct - Roleplay-Instruct - Code-Instruct - and Toolformer
A collection of GPT-4-generated fine-tuning datasets in Alpaca format, covering general instruction-following, roleplay, code generation, and tool-use scenarios. It's for people who want to fine-tune open-source LLMs and don't want to generate their own synthetic data from scratch. Last updated September 2023, so it predates most modern fine-tuning workflows.
The dataset variety is useful — having separate splits at different similarity thresholds (0.6–0.9) lets you trade off diversity vs. quality without doing your own dedup work. The Alpaca-compatible format means it drops straight into most fine-tuning scripts without preprocessing. The Toolformer dataset covering search, Python, shell, and Wikipedia is the most distinctive part; tool-use data was genuinely scarce when this was released. Code-Instruct covering multiple languages in a single clean JSON file is convenient for quick experiments.
The repo is abandoned — no commits since 2023, and the toolformer documentation was never written ('coming soon' is still there). GPT-4-generated synthetic data from 2023 has obvious quality ceiling issues: factual errors are unchecked, reasoning steps aren't verified, and the outputs reflect GPT-4's biases and knowledge cutoff rather than ground truth. Alpaca format is also a dead end for modern fine-tuning — everyone has moved to chat/conversation format, so you'd need to reformat before using most current training scripts. There's no data card, no known license on the actual data (only code is MIT), and using GPT-4 outputs for training is a ToS grey area that OpenAI has since explicitly restricted.