Special Recognition - Dr. Neil Singer, Jeremy Blum, Brandon Fischer, Zach Lynn, Alex Miller, Ben Shaffer
Guitar Hero is a video game franchise that has revolutionized the gaming industry. Players use a guitar-like peripheral to play many well-known rock songs by holding the fret buttons and strumming. Top ranked players achieve Internet celebrity status for their feats. In addition to being a great entertainment device, Guitar Hero robots catch the attention of millions of YouTube viewers and Guitar Hero fans. Most attempts at making machines to play Guitar Hero have been limited to two types: a) simple preprogrammed playback machines or b) expensive computer based machines that have performed poorly. Our goal was to create an inexpensive and accurate entertainment robot that could detect notes in real-time and play them for any song.
A group of High School Seniors working as interns at Convolve, Inc., an Armonk, NY based engineering firm, created a high performance Guitar Hero Robot. Our robot, DeepNote™, consists of a rack with five individual light sensors and standard Guitar Hero USB guitar modified to fit a Parallax propeller chip and other circuitry. The light sensors use photodiodes that are mounted on a custom printed circuit board (PCB) for signal conditioning. In addition, the PCBs are outfitted with red LEDs that illuminate when the corresponding sensor detects light. The rack that holds the light sensors is adjustable for various sized TV screens. The digital signals from the rack are sent directly into the propeller chip housed in the guitar. The propeller chip, mounted on a different PCB, has output connections to the guitar's functions via solid-state relays and is powered by either a wall supply or a 9 volt battery.
The propeller chip enabled us to maintain precise timing while running various simultaneous tasks in parallel. The first task involves running a 600 μs loop that processes the light sensors' signals. We discovered that the light sensors have an associated noise and “bounce” between on and off when a note is first detected. In order for the robot to play accurately, it has to sample the inputs at a fast rate and filter the notes. The propeller's speed allowed us to accomplish this task quickly by processing all five note in a single cog in parallel. The parallel processing algorithm takes advantage of the propeller's 32 bit architecture. Five notes are stored as individual bits in a register. The note are spaced five bits apart across the register. All math, storage, and retrieval operations are performed on an entire register containing all five notes. This is possible because none of the math operations cause overflow beyond the five bits allocated for each note. This configuration enables the the robot to run 600 μs loops in spin.
Since the light sensors detect notes before they must be played, the note detecting cog must store the notes in a circular buffer for another cog to later read and output. Again, the propeller's architecture lends itself to this application allowing two independent cogs to read and write from different locations of the same circular buffer (or delay line). The output cog has a user adjustable delay that is read from a thumb wheel switch mounted on the guitar. This cog also accommodates for misalignment of the light sensors and corrects for these errors before pressing the buttons and strumming the guitar.
The main cog monitors the thumb wheel switch for the delay calculations and controls the whammy bar in order to generate higher scores. Additionally, there are three utility cogs that serve to perform tasks that aren't critical to game play, though improve score and functionality. These three cogs extend notes, activate star power, and output internal data to a National Instruments box or oscilloscope for debugging. The note extension cog fixes shorter notes due to imperfections in the sensors. The star power cog monitors the circular buffer looking for many notes over a short period of time. When star power is deployed, the points earned are temporarily doubled. Without the propeller's eight cogs, many of these functions would be compromised and lost. Debugging was simplified because an independent cog could output data that we needed without disrupting our main code and slowing down its operation.
The performance of the robot is remarkable and it is able to play all songs, in real-time, in expert mode with 98-100% accuracy. Our robot rivals the best human players in the world without any previous knowledge of the songs. In multiple cases, it has scored significantly higher than any human.
DeepNote Guitar Hero Source Code (.zip)
DeepNote Web Site (off-site)
DeepNote Guitar Hero Video
Full Description and Data (.pdf)