Modern pipelines can label data at a cost as low as $.01 per polygon.
They can generate million images in 6 weeks.
I highlight that Web Rephrase Augmented Pre‑training (WRAP) uses an off‑the‑shelf instruction‑tuned model prompted to paraphrase web documents in styles like “like Wikipedia” or “question‑answer format”, creating a combined corpus of real web text plus synthetically generated paraphrases. Benefits include ✅ style diversity and ✅ higher “quality” than raw web‑scraped data.
Human raters scored WRAP‑augmented data 0.12 points higher on a 5‑point scale than raw web data.
Models pre‑trained on WRAP data achieved +1.8% absolute accuracy on the SQuAD‑style QA benchmark.
The same models showed +2.3% improvement on the Open‑Domain Retrieval task.
I note that the Tencent AI Lab team created 1 billion diverse personas from automatically curated web data.
Example persona categories include a moving company driver, a chemical kinetics researcher, and a musician interested in audio processing.
The framework was authored by Xin Chan, Xiaoyang Wang, Haitao Mi, and Dong Yu.
I suggest hiring thousands of people to create use‑case specific prompts, bootstrapping high‑quality LLM training data.
This billion‑person persona generation framework demonstrates that synthetic data can overcome training data scarcity by providing a massive, diverse pool of role‑specific profiles that can be turned into targeted prompts for LLM fine‑tuning.