INTER_

Creative Technologist for Chemistry Creative
Soho / Chinatown, NYC

2021 - 2023

Project Description

INTER_ is a 2-story interactive and immersive exhibition located within the Historic National City Bank of New York, from creator Pete Sax, with the incredible team at Chemistry Creative

My role as lead Creative Technologist included everything from pre-production, development, and installation of all interactive tech and video components throughout the project. I was also involved in creative development, and provided concept art and pre-visualization for key installations throughout the pre-production process.

This project would not have been possible without key assistance from many fellow creative technologists and team members including Dave Tandem (Interconnected), Jonathan Masterson, Georgios Cherouvim, Beau Burrows, Brian Snigur, Seth Scoville, Nicholas Motyka, (Intersensory), Charlie Gangemi, Tommy Martinez (Internal / Interplanetary), Taliesin Gilkes-Bower (Cymatic Well), Andrew Giugno, Julian Baeza Hochmuth (Lighting), Danny Taylor, MPC, Oliver Chapoy, Henrik von Sivers, Danush Dopierala (Audio), Lovejoy Paradise, Kasey Burgess, May Ghadanfar, Pablo Gnecco, Anthony Montalbano (Production + Direction), Mal Mal, Hozi, Patrick A. Moore, Marta Kaminska, Robert Scoville, Mark Musters, Hillary Ureña, Hannah K. Hollis, Simon Fernandez, Jin Pak, Anthony Azcona, many more scenic artists and fabrication specialists.

The exhibition contains dozens of projectors and 4k displays with hundreds of feet of video and data cabling, as well as a handful of large LED walls with unique pixel map configurations. Servers were configured by-hand during pre-production and installed on-site with a Unifi fiber network throughout the building.

As visitors move through the exhibit, there are a number of interactive moments that utilize live-camera feedback- unbeknownst to visitors at the time, these interactions are temporarily recorded and used to create generative art on display in the final room- the Interconnected gallery. This ephemeral, full-circle payoff is a nod to culminating experience or metaphorical 'life-review' that one may experience at the moment of death.

Integration with scenic, lighting, audio, production crew, and exhibition staff was crucial throughout this project. Cheers to everyone involved!

Project Highlights

https://jonathanmasterson.com/
Jonathan Masterson
https://ch3.gr/work/vfx/
Georgios Cherouvim
https://www.matthewpattersoncurry.com/
MPC
https://www.bryansnigur.com/
Brian Snigur
https://lowres.co/
Andrew Giugno
https://www.mayghadanfar.com/
May Ghadanfar
https://yopablo.studio/
Pablo Gnecco
https://www.linkedin.com/in/kasey-burgess/
Kasey Burgess
https://www.linkedin.com/in/anthony-montalbano-35a979162/
Anthony Montalbano
https://www.instagram.com/jules.b.h/?hl=en
Julian Baeza Hochmuth
https://www.instagram.com/picturesbyrobots/?hl=en
Dave Tandem
https://www.futurewife.tv/
Beau Burrows

Taliesin Gilkes-Bower
Tommy Martinez  
Oliver Chapoy 
Danny Taylor

https://www.instagram.com/malcolmnyc/?hl=en
Marta Kaminska
https://www.instagram.com/hhoozzii/?hl=en
Hozi
Marta Kaminska
https://m.imdb.com/name/nm14411966/
MARTA KAMINSKA
Marta Kaminska
https://marta-kaminska.squarespace.com/
http://www.hannahkhollis.com/
Hannah K. Hollis
Patrick A. Moore

INTER_ consists of nine thematic rooms:

1. Lobby / Entryway
2. Interorientation

3. Interpersonal
4. Intersensory
5. Interplanetary / Hanging Gardens
6. Interaction / Cymaic Well
7. Interstitial
8. Internal
9. Interconnected

I was involved with the pre-production, installation, and creative development of interactive technology and video throughout the exhibition. Each installation is running on TouchDesigner and uses a network of IR, NDI cameras, ultrasonic sensors, and LiDAR.

Device maps for pre-production and installation

'INTERorientation'

Once guests are checked in for their experience time, they are invited to wait in the orientation room.

This room acts as a holding chamber for timed groups, as well as an introductory film which guides visitors through a range of emotions and abstract thoughts.

There are 2x hidden NDI cameras in this room which capture preliminary interactions from guests.

Introductory film playing on a large LED wall at the start of the experience
LED mesh screen opposite main wall

'INTERpersonal'

After the conclusion of the introductory film, visitors head up a mirrored escalator passage known as Interpersonal.

On either side of the up-escalator, there is a built-in structure of angled mirrors and LED tubes. There are 3x ultrawide displays and an NDI camera hidden behind 2-way mirror panels. There is an additional NDI camera mounted on the ceiling at the far end of the 2nd floor, angled down the escalator.

The displays shine through the 2-way mirrors and appear within the mirror tunnel, while the camera feed is recorded and fed back into the system.

3x hidden displays and 2x NDI cameras were embedded into the structure
Video showing guests moving up the escalator through the hall of mirrors and hidden cameras

'Intersensory'

After storing their belongings and watching a brief introductory film, visitors are lead upstairs to the 2nd floor, where they enter a large ovular room with 360 degree audio and projections.

Visitors movements and presence are seen by 2 LiDAR sensors in the ceiling, and are encouraged to move around and interact with the projections.

My responsibilities included the pre-production and development of motion tracking, installation of the projectors and cables, and building a custom show control UI in TouchDesigner for playback and interactivity.

There was also a high level of coordination between construction, scenic work, sensor installation, lighting installation, DMX programming, and additional environmental features like haze and bubbles.

I also used TouchDesigner to create pre-visualization tools for creative development, as well as a unique visual sequence that was used as a part of the show.

360 Projection pre-visualization for creative development- made with TouchDesigner
Interactive 'Idle' state development- made with TouchDesigner
Custom visual sequence during a live show- made in TouchDesigner with Nicolas Motyka

'INTERplanetary'

Upon exiting the Intersensory dome, visitors move through a psychedelic garden full of lights and color.

There are 6x Arduino units and ultrasonic sensors which control lights and sounds- installed and programmed with Charlie Gangemi, Andrew Giugno, and MPC.

Wi-fi Arduino module and diagram of arch interaction

'The Cymatic Well'

The 'Cymatic Well' is a large steel pan filled with water and surrounded by dichroic crystal structures. Originally conceptualized by Taliesin Gilkes-Bower- I worked with a small team to prototype controllers and test user experience at Chemistry Creative in Brooklyn.

After some iteration, we ended with 3 crystal balls that use a laser sensor- similar to the track ball of a computer mouse- and allow guests to control sound and play the installation like a large instrument.

There is a large transducer attached to the bottom of the well, which is fed and controlled by an Ableton Live session. The vibration from this transducer creates a number of distinct resonant patterns in the water, which are visible on the surface.

Due to the nature of cymatics and the shape of the vessel, only specific frequencies would create patterns on the water. Throughout development, I worked with Tommy Martinez to find the frequencies that would produce a pattern. Sound supervision was coordinated by Oliver Chapoy  and MPC, with speaker installation by Danny Taylor.

There are also a number of unique visual interactions that are triggered based on specific user participation and are displayed on the large LED wall behind the well.

Guests can turn the crystal balls to control the lights and sounds-
Distinct patterns emerge in the water with certain frequencies.
Live demo with three users - 'ohm' state activated
Multi-color LED demo and visuals from pre-production

'INTERnal'

Once visitors are finished on the 2nd floor, they descend another escalator into a dark, rocky cavern known as Internal.

After turning the corner and reaching the end of a long corridor, guests are invited to explore the walls of the chamber and look for motion activated control points. These control points work like a large organ, and play various ground-shaking tones when activated.

In the center of the room there is a volcano structure known as 'The Crucible.'

Here, guests are invited to release something into the fire. Inside the crucible there is a hidden Leap motion controller, which allows users to affect the audio around the room, while their hand appears in the LED wall in front of them.

Atop the LED wall there is an Azure sensor which reveals a glowing magma character when activated.

After leaving Internal, guests are guided through a nearly pitch-black room full of mechanical gongs and haptic floors, know as the Void- before moving to the last room, Interconnected.

Central chamber of Internal- showing The Crucible (right) and The Transformation (center)
Process photos for leap motion placement- Crucible structure completed by Marta Kaminska
Internal interactive walkthrough- immersive rock corridor opens to main chamber with hidden hand sensors
Demo of Leap motion hand sensor and Crucible interaction

'Interconnected'

'Interconnected'

Covered in all-white tiles and containing 10x large 4k displays, the Interconnected gallery replays the interactions of each user's experience.

Each display is linked to a camera feed from a specific interaction, and sources recordings from the past 2 hours of the exhibition. Movie files are stored on 1 of 4 servers and are erased daily to make room for storage efficiency and privacy.

On either side of the room there is a large horizontal display containing all of the interactions that are playing in the room at that time.

When a user steps in front of the frame, the image morphs into a live-camera feed, creating an infinite tunnel that visitors can capture with a video recording device of their desire.

Overview of interaction recordings sourced throughout any given day
Live camera feedback - activated by motion on the 2x large, mirrored displays on either end of the gallery
Interactions recorded throughout the day are cycled through the generative system.

'The Wave'

The heart of the installation is a series of interactive "time portals," each representing a different historical era. These portals are equipped with motion sensors, touchscreens, and augmented reality interfaces, allowing visitors to step into different time periods and explore them in an engaging and informative way.

The Portholes

Our team has meticulously choreographed the installation process to ensure that every element seamlessly fits into the overarching narrative. We've worked closely with artists, historians, and technologists to guarantee that every aspect, from lighting and sound to digital displays and physical artifacts, complements one another perfectly.

Communal Action Wall

Our team has meticulously choreographed the installation process to ensure that every element seamlessly fits into the overarching narrative. We've worked closely with artists, historians, and technologists to guarantee that every aspect, from lighting and sound to digital displays and physical artifacts, complements one another perfectly.

Lobby + Misc

Upon entering and exiting the building, visitors can walk around a rectangular lobby area.

On the back wall there is a large TV mosaic with a unique arrangement of TVs. We worked to provide a cost-effective solution and provide preliminary content.

Guests can turn the crystal balls to control the lights and sounds.
Distinct patterns emerge in the water when activated with various pre-determined frequencies.
Sound waves can be visualized as they ripple through the water.