Toptal Launches ExpertCrowd for AI Training and Evaluation

Toptal announced in a press release the launch of ExpertCrowd, a platform for training and evaluating AI models with specialized domain experts. It supports reinforcement learning from human feedback, reinforcement learning data generation, data labeling and model evaluation.

Toptal says ExpertCrowd draws from its network of more than 30,000 experts across over 140 countries. The network includes more than 2,000 AI and big data specialists, 12,000 professionals with advanced degrees and 1,000 AI specialists with doctorates. The platform provides multimodal annotation, customizable workflows and live project tracking.

Toptal also released the ExpertCrowd-SWE benchmark for evaluating AI coding agents on previously unseen software issues. It covers 87 codebases in 11 programming languages, with Toptal engineers authoring the tasks and grading rubrics. According to Toptal, ExpertCrowd generated more than $5 million in recurring revenue before its official launch, including work with Scale AI, Sterling Labs and Vetto.