Constellation Prototype (Vive)

Constellation is a 3D music sequencer project i have been involved with for several years on and off in the Unity game engine. It is a unique music application/experience based upon the idea of being able to rotate a musical idea along a 3D while viewing it on 2D screen. The name of the app is based on the phenomenon that any given arrangement of stars in a constellation is not actually equidistant from the viewer. Instead stars are at different levels of distance from the viewer. Thus rotation of the constellation from a center point would result in vastly different patterns than what was originally perceived.

 
The basic idea revolves around a fairly common grid based sequencer approach that many application and hardware sequencers take, where pitch is controlled by the vertical (Y) axis and time the horizontal (X) axis. In this case, however i have a third axis, making the design a 8x8x8 cube. This last axis (Z) controls timbre with different instruments playing. Rotating the cube then not only changes what notes or pitches are playing at various times, it also changes the instrument used to play them. Because the sequencer rotations are somewhat constrained in terms of outcomes, the resulting transformations of the sequence can be heard as more or less distantly related to the original idea, but in a slightly different manner than typical motivic transformation via inversion, retrograde and so on.
This prototype uses Csound as the sound engine running within Unity as well as a rudimentary cross modulation between layers. Future expansions will revise the interaction to be more intuitive, and will have a 16 x 16 x 16 cube, as well as many more cross modulating and processing capabilities.

 

Read More >>

Death’s Feather
This project published on Itch.io is another collaboration with Grammy winning engineer and composer for the film ‘Super Size Me’ Steve Horowitz, and it’s an homage to a Randy Hostetler, a composer of promise who attended California Institute of the Arts but whose life was cut short only a few years after graduating with a Master’s degree. Steve had earlier written a piece dedicated to his memory called ‘Death’s Feather’ after a short poem by Dylan Thomas, but rather than a basic recording or recorded concert video he decided an interactive setting would be an interesting and engaging experience that would bring a different audience’s attention to Randy’s music and aesthetic.

We discussed various possibilities and settled on a FPS setting in a 3D maze environment. The piece would be the basic soundtrack, which would temporarily end musically by entering into special areas where players could experience a video of a performed piece by Randy or examine his influences. Once the player left the area the music would return. Eventually by visiting all four of these areas you were allowed to leave the maze by falling through it, which would trigger the ending and credits.

I handled all the level design and C# scripting for this in Unity which involved some usage of the Unity Timeline in the intro, as well as the ending and music management along with video clip playback playing on textures in special rooms. Also managed to find a text animation asset that created spooky floating text which was typed out for an introductory poem as well the basic directions. Overall a brief but fun project!

Read More >>

Essential Game Audio App
I designed all of the music and sound effects for this title to accompany our book The Essential Guide to Game Audio. Both the book and the app were published by Focal Press. I also programmed all of the interactive game examples in the app (The Holodeck section) in Unity which also featured videos and quizzes by the developer Zephyr Media and Teravision. Last, I ended up in charge of creating and testing the final build in Xcode and delivering it to Teravision.

Due to the application being 32 bit and there being no desire to update it by Focal, the app is unfortunately no longer available.

Read More >>

Ruby Isle Adventure
This game lesson using Unity and Wwise is a 3rd person point and click puzzle based adventure game somewhat in the tradition of Myst. For this project I was assisted by a talented student, Brent Heatherwick, who designed the 3D models as well as the C# logic for the puzzles, while I handled all of the camera management, animation controls, in-game object interaction, inventory management and UI in Unity, as well as the sound design, implementation in Wwise, and documentation.

Read More >>

Smuggler’s Dilemma TPS
This third person Unity and FMOD Studio sci-fi shooter game lesson for GAI started life as a brief collaboration and ended up largely a solo dev project, starting first from an existing free FPS framework and then changing over into various other TPS frameworks, along with supplemental scripting filling out the interaction. I handled all the level design, player/enemy and weapon setup, and the audio system, starting with Unity audio and then transitioning into FMOD Studio event and parameter triggering. The code was revised and classroom tested several times over the years, and its usefulness as an advanced game lesson covering all of the major audio categories as well as several features of FMOD Studio, has been well-proven. I also wrote the documentation, and produced tutorial videos on the implementation.

Read More >>

DangerousDungeon
Dangerous Dungeon is a third person 3D top down dungeon crawler adventure featuring procedurally generated architecture with randomly selected rooms with items, puzzles or traps. Guide your intrepid adventurer to success in this Unity based game lesson with music, sound effects and VO.

For this project I handled nearly all of the coding for the C# scripting for player animations, object interactions, notifications, UI, the room and item generation as well as all visual level design. Models were sourced from Blendswap, OpenGameArt and Bitgem.

Read More >>

DayAndNight-FMOD
This game lesson was adapted from an existing project into a simple but complete game lesson covering adaptive music and ambience in FMOD Studio and Unity. I adapted existing scripts to drive FMOD Studio event parameters, as well as some more level design, and all of the documentation, in both written and video tutorial form. I also composed the music and designed the ambience for the demos.

Read More >>

Music Maze
I designed this level in Unity in order to show students how you can use different types of interactive music to navigate your way through a maze that only has two directions. In one direction you’ll get progressively positive reinforcement and in the other direction the feedback will be increasingly negative. Utilizing a procedural maze generator I adapted the output to conform to a more simplified layout and designed all interactions and UI in Unity and C#. I also created the demo music. Graham Terry added a lovely bit of live violin on the top.

This game lesson is available in FMOD and Unity audio based flavors through the Game Audio Institute.
Click here to Play in WebGL (only on Mac or PC for now). This demo is a WebGL version based on Unity Audio (FMOD does not work in a browser currently).

Use arrow keys or WSAD and mouse to navigate your way through the demo.

 

 

 

 

Read More >>

Hyperpiano DSP 1.0

 

Probably one of my most ambitious projects, this tangible interface was the first step in trying to create a portable prepared piano of sorts. It utilized a custom interface featuring piezoresistive fabric ribbons on a conductive copper base, using objects that had Reactivision fiducial symbols that would be recognized by Reactivision via an overhead webcam and turned into data that would control instrument switching in MaxMSP using a fluidsynth object playing samples of prepared piano. Sensor data for the ribbons (onset and resistance value) were sent to Max via a SlipOSC code installed on a Teensy microcontroller. Additionally extra controls over the sound were possible by adding a Korg Nanokontrol. The included clip is a demo video, briefly explaining the setup. A short performance clip is available here:

Many thanks to Adrian Freed of CNMAT for invaluable assistance with materials and use of the Teensy!

Read More >>

Hypertap Instrument

This is the second stage of my obsession with a portable piano of sorts. After abandoning the fabric controllers which were not stable, I opted instead for real strings. Several designs were considered and discarded when I discovered the Harpejji, a variation of an earlier custom instrument called the Starr Board. This is a tapping instrument where both hands face the same direction rather than a Chapman Stick where the hands face in opposite directions. As a Harpejji was out of my price range, I enlisted the help of a tapping instrument maker (Kevin Siebold of Krappy Guitars) to make the body, bridge and strings to my specs, and a young engineer to design the active pickup system ( a dual octal buffer, for 16 strings). I then found  a set of piezo saddles which i adapted and wired up to the buffer circuit boards, and the Hypertap was born.

Future versions of this will exploit the individual string output available to do A/D conversion (onset,pitch detection) to potentially controll other kinds of processing.

Read More >>