Even some of the best AI can’t beat this new standard

The nonprofit Center for AI Safety (CAIS) and Scale AI, a company that provides a number of data classification and AI development services, released a report Challenge the new standard For frontier artificial intelligence systems.

The standard, called “The Last Test of Humanity,” includes thousands of group questions covering topics such as mathematics, humanities and natural sciences. To make the assessment more rigorous, questions are in multiple formats, including formats that include graphs and images.

In a Preliminary studyNo major AI system available to the public has been able to score better than 10% in humanity’s last test.

CAIS and Scale AI say they plan to open the standard to the research community so researchers can “dig deeper into the differences” and evaluate new AI models.