installation multimedia news Software Development

ASCII – ASCII Sound Continuum Interactive Installation

Experimental sound art joining with 3D projection

ASCII (Ascii Sound Continuum Interactive Installation) is an audio-video installation that uses the environmental sounds in order to create a ‘waterfall’ of ASCII characters in 3D retro-projection mapping, projected on a three-dimensional space appropriately created.

The work describes the continuous path between the simplification of the  input data, (obtained through spectral averages of environmental sounds in terms of frequency and time), and the complexity, that springs out with the use and the translation of such data in the diverse environments, both in video and audio.

The project is developed by Edoardo Pedio, specialized in the technological, artistic and creative aspects of sounds, and Mario Salvucci, award winning composer and software developer.

aligning the 3D grid on the cube

aligning the 3D grid on the cube

The Cube (actually 3 faces of a cube) is composed by transparent material with 80 cm of side on which are projected images using the technique of 3D projection mapping from the rear.

The sound (stereo) sources are at the bottom of the cube; the projector is at a distance of about 2 meters behind the cube. A stereo microphone is used to sample the ambience sound. The projector, microphone, and speakers are wired to a laptop running pure data and processing.

Through the microphone the audio is captured from the ambience where the cube is placed. The audio input is processed within an application (Pure Data) and through the use of FFT is quantized to a discrete number of frequency bands.

The amplitude of each band is treated in a manner similar to the brightness of the pixel blocks in the well known ascii algorithm applied in computer graphics.

The high-level concept of ascii art in computer graphics is very simple:

ascii art

It is possible to give each printable character an approximate weight in grayscale. For example, the @ character is visually darker than the + sign; the effect changes according to the font and spacing used. The input image is then divided into rectangular blocks of pixels of fixed width and height (e.g. 4 pixels wide and 5 pixels high rectangles). Each block will become an output character. The mean grayscale value is calculated in each block and is chosen the character which is the best rounding to the value. Therefore data amplitude values associated with individual bands are sent in sets to an host application which implements a simple particle system (Processing) via OSC.

The host application also takes care of mapping the values on an alphabet composed of characters from the ASCII standard set. For each band the weight is identified and consequently generated a particle with the appropriate values.

The resulting animation is posted in rear projection on the surface of the cube.

The sound generated by the cube is made following the same principle: each frequency band is associated with an oscillator tuned on the average frequency which plays with a proportional amplitude, also each slice is associated with the sound playback of a phoneme from the sound poem for phonemes “Ursonate” (1922-32) by Kurt Schwitters.


Mario Salvucci and Edoardo Pedio

Download the ASCII operation details (PDF, 268 Kb)