{"id":3485,"date":"2018-05-14T10:18:51","date_gmt":"2018-05-14T16:18:51","guid":{"rendered":"http:\/\/handyvandal.com\/?p=3485"},"modified":"2018-05-14T10:22:31","modified_gmt":"2018-05-14T16:22:31","slug":"artificial-intelligence-finds-surprising-solution-to-qbert","status":"publish","type":"post","link":"https:\/\/handyvandal.com\/wphv\/2018\/05\/artificial-intelligence-finds-surprising-solution-to-qbert\/","title":{"rendered":"Artificial intelligence finds surprising solution to Q*bert"},"content":{"rendered":"<p><img loading=\"lazy\" decoding=\"async\" src=\"https:\/\/handyvandal.com\/wphv\/wp-content\/uploads\/2018\/05\/Qbert-300x300.png\" alt=\"Q*bert\" width=\"300\" height=\"300\" class=\"alignright size-medium wp-image-3487\" srcset=\"https:\/\/handyvandal.com\/wphv\/wp-content\/uploads\/2018\/05\/Qbert-300x300.png 300w, https:\/\/handyvandal.com\/wphv\/wp-content\/uploads\/2018\/05\/Qbert-150x150.png 150w, https:\/\/handyvandal.com\/wphv\/wp-content\/uploads\/2018\/05\/Qbert.png 400w\" sizes=\"auto, (max-width: 300px) 100vw, 300px\" \/>Artificial intelligence sometimes solves problems using solutions which surprise humans. I find this quality charming &#8212; menacing*, but charming.<\/p>\n<p>The Verge reports: &#8220;<a href=\"https:\/\/www.theverge.com\/tldr\/2018\/2\/28\/17062338\/ai-agent-atari-q-bert-cracked-bug-cheat\">A video game-playing AI beat Q*bert in a way no one\u2019s ever seen before<\/a>.&#8221;<\/p>\n<blockquote><p>[A] trio of machine learning researchers from the University of Freiburg in Germany &#8230; were exploring a particular method of teaching AI agents to navigate video games (in this case, desktop ports of old Atari titles from the 1980s) when they discovered something odd. The software they were testing discovered a bug in the port of the retro video game Q*bert that allowed it to rack up near infinite points.<\/p>\n<p>As the trio describe in <a href=\"https:\/\/arxiv.org\/abs\/1802.08842\">[their] paper<\/a>, published on pre-print server arXiv, the agent was learning how to play Q*bert when it discovered an \u201cinteresting solution.\u201d Normally, in Q*bert, players jump from cube to cube, with this action changing the platforms\u2019 colors. Change all the colors (and dispatch some enemies), and you\u2019re rewarded with points and sent to the next level. The AI found a better way, though; the researchers report:<\/p>\n<p>&#8220;First, it completes the first level and then starts to jump from platform to platform in what seems to be a random manner. For a reason unknown to us, the game does not advance to the second round but the platforms start to blink and the agent quickly gains a huge amount of points (close to 1 million for our episode time limit).&#8221;<\/p><\/blockquote>\n<p>The research paper: <a href=\"https:\/\/arxiv.org\/abs\/1802.08842\">Back to Basics: Benchmarking Canonical Evolution Strategies for Playing Atari<\/a>.<\/p>\n<p>* By &#8220;menacing&#8221;, I mean real-world systems with life-and-death consequences &#8212; medical devices, weapons systems, etc. &#8212; not Q*bert.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Artificial intelligence sometimes solves problems using solutions which surprise humans. I find this quality charming &#8212; menacing*, but charming. The Verge reports: &#8220;A video game-playing AI beat Q*bert in a way no one\u2019s ever seen before.&#8221; [A] trio of machine learning researchers from the University of Freiburg in Germany &#8230; were exploring a particular method [&hellip;]<\/p>\n","protected":false},"author":7,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[235,62,8],"tags":[],"class_list":["post-3485","post","type-post","status-publish","format-standard","hentry","category-arcade-games","category-ai","category-video-games"],"_links":{"self":[{"href":"https:\/\/handyvandal.com\/wphv\/wp-json\/wp\/v2\/posts\/3485","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/handyvandal.com\/wphv\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/handyvandal.com\/wphv\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/handyvandal.com\/wphv\/wp-json\/wp\/v2\/users\/7"}],"replies":[{"embeddable":true,"href":"https:\/\/handyvandal.com\/wphv\/wp-json\/wp\/v2\/comments?post=3485"}],"version-history":[{"count":7,"href":"https:\/\/handyvandal.com\/wphv\/wp-json\/wp\/v2\/posts\/3485\/revisions"}],"predecessor-version":[{"id":3493,"href":"https:\/\/handyvandal.com\/wphv\/wp-json\/wp\/v2\/posts\/3485\/revisions\/3493"}],"wp:attachment":[{"href":"https:\/\/handyvandal.com\/wphv\/wp-json\/wp\/v2\/media?parent=3485"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/handyvandal.com\/wphv\/wp-json\/wp\/v2\/categories?post=3485"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/handyvandal.com\/wphv\/wp-json\/wp\/v2\/tags?post=3485"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}