<\/div>\n
However, AI still isn’t able to answer difficult research questions and other intellectual questions.<\/p>\n
It also appears to score poorly on tests involving planning and visual pattern-recognition puzzles, according to Stanford University’s AI Index Report from April.<\/p>\n
Consequently, “humanity’s last exam” will require abstract reasoning to test how clever AI really is.<\/p>\n
The submissions shouldn’t be any ordinary quiz questions.<\/p>\n
“We found questions written by undergraduates tend to be too easy for the models,” the creators of the quiz said.<\/p>\n
Instead, they recommend that question writers have five or more years of experience in a technical industry job like SpaceX, or are a PhD student or above.<\/p>\n
The submissions should be difficult for non-experts to answer and “not easily answerable via a quick online search”, and trick questions should be avoided.<\/p>\n
“As a rule of thumb, if a randomly selected undergraduate can understand what is being asked, it is likely too easy for the frontier LLMs of today and tomorrow,” said the quiz creators.<\/p>\n
People who submit successful questions will be invited as co-authors on the paper and have a chance to win money from a $500,000 (\u00a3378,400) prize pool, with the writers of the best questions earning $5,000 (\u00a33,780) each.<\/p>\n
Questions should be submitted by 1 November.<\/p>\n<\/p>\n
This post appeared first on sky.com<\/div>\n
<\/p>\n","protected":false},"excerpt":{"rendered":"
Scientists are creating “humanity’s last exam” to test AI and see when it has reached…<\/p>\n","protected":false},"author":1,"featured_media":8062,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[24],"tags":[],"class_list":["post-8061","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-science"],"_links":{"self":[{"href":"https:\/\/dailywashingtoninsider.com\/index.php\/wp-json\/wp\/v2\/posts\/8061","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/dailywashingtoninsider.com\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/dailywashingtoninsider.com\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/dailywashingtoninsider.com\/index.php\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/dailywashingtoninsider.com\/index.php\/wp-json\/wp\/v2\/comments?post=8061"}],"version-history":[{"count":0,"href":"https:\/\/dailywashingtoninsider.com\/index.php\/wp-json\/wp\/v2\/posts\/8061\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/dailywashingtoninsider.com\/index.php\/wp-json\/wp\/v2\/media\/8062"}],"wp:attachment":[{"href":"https:\/\/dailywashingtoninsider.com\/index.php\/wp-json\/wp\/v2\/media?parent=8061"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/dailywashingtoninsider.com\/index.php\/wp-json\/wp\/v2\/categories?post=8061"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/dailywashingtoninsider.com\/index.php\/wp-json\/wp\/v2\/tags?post=8061"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}