François Chollet, an A.I. researcher, created a puzzle game called ARC in 2019 specifically designed to be easy for humans but challenging for machines. The colorful puzzles require players to identify visual patterns based on a few examples. A.I. systems, including OpenAI’s latest model o3, struggled with these puzzles initially but eventually surpassed human performance last December. The puzzles were created as a benchmark to measure progress towards artificial general intelligence (A.G.I.) but have since been criticized for their limitations in evaluating true intelligence. The ARC Prize, a $1 million contest financed by Chollet and Mike Knoop, sought to award anyone who built an A.I. system that exceeded human performance on the ARC-AGI benchmark. However, no winner emerged, with OpenAI’s o3 being disqualified due to high computing costs and lack of open sourcing. A new benchmark, ARC-AGI-2, has been introduced, featuring more challenging tasks that even advanced A.I. systems struggle to solve. The ARC Prize team is already working on ARC-AGI-3, set to debut in 2026, with a focus on dynamic, grid-based puzzles involving real-world interactions. Chollet believes that the ability to efficiently acquire new skills is crucial for A.I. to achieve artificial general intelligence. As technology advances, the goal posts for measuring A.G.I. will continue to shift. Chollet states that once it becomes impossible to create benchmarks that are easy for humans but impossible for A.I., then artificial general intelligence will have been achieved.
Note: The image is for illustrative purposes only and is not the original image associated with the presented article. Due to copyright reasons, we are unable to use the original images. However, you can still enjoy the accurate and up-to-date content and information provided.
