“I used the controller inputs as a sort of binary code to rewrite the video game as my robot played it,” sounds like a line you would hear in some dumb hacker movie. But that is evidently what someone has done with Super Mario World, according to Ars Technica:
The most remarkable moment of the weeklong marathon, though, came when a robotic player took “total control” of an unmodified Super Mario World cartridge, reprogramming it on the fly to run simple versions of Pong and Snake simply by sending a precise set of inputs through the standard controller ports on the system.
It’s at 1:39 in the video where things really start going pear-shaped, as the fabric of the game’s reality comes apart at the seams for a few seconds before inexplicably transitioning to Mario-themed versions of Pong and Snake. Understanding what’s going on here requires some deep knowledge of the Super NES’ internal sprite and memory management, which is explained in detail here and here.
Suffice it to say that the first minute-and-a-half or so of this TAS is merely an effort to spawn a specific set of sprites into the game’s Object Attribute Memory (OAM) buffer in a specific order. The TAS runner then uses a stun glitch to spawn an unused sprite into the game, which in turn causes the system to treat the sprites in that OAM buffer as raw executable code. In this case, that code has been arranged to jump to the memory location for controller data, in essence letting the user insert whatever executable program he or she wants into memory by converting the binary data for precisely ordered button presses into assembly code (interestingly, this data is entered more quickly by simulating the inputs of eight controllers plugged in through simulated multitaps on each controller port).
This, in a way, is clear proof that movie scenarios like The Terminator or The Matrix will never come to pass; the people who are capable of developing computer programs that could pose an existential threat to humanity are clearly frittering away their talent on vintage gaming.