Play the City
( 06 / 12 )
( Brief )
New York City is a place so alive with music that rhythm and melody resonate from literally everything and everyone there. The Recording Academy wanted to capture this musicality to celebrate the return of the Grammys to New York in 2018. So we created an AR experience that used an Uber as the playhead moving through the natural score unfolding in the city outside the backseat window. Every person, car, light and many other objects appeared in their rhythm and position to compose the real-time music and AR visuals of the streets and sidewalks of New York.
( Role )
I acted as tech director, creative director and support dev for the project, guiding the client, agency and internal team through important considerations in implementing computer vision driven AR in a moving vehicle. The tech team consisted of myself and one other dev who specializes in openFrameworks. After helping the client avoid suboptimal environments for installing public AR we decided on AR-in-a-car. From this point I led the team in prototyping and running many tests to establish guidelines on car speed, route, display technology, etc. Additionally I worked with the client and composer to plan a dynamic software musical instrument to generate structured music while ingesting computer vision data and outputting visual AR in real-time.