The Mixed Reality Lab (MXR) aims to push the boundaries of research into interactive new media technologies through the combination of technology, art, and creativity.
In Classical Music Meets Interactive Media, The main goal of this project is linking classical music and visual art through a real-time processing on a violin performance.
A meaningful 3D visualization will be correspondence within the original thinking of the composer after an exhaustive analysis of the score for the concert attending to the different parameters of the music language: harmony, counterpoint, instrumentation, formal analysis, style and background of the composer. The violin performer’s expression will be interacting with the music and the 3D visualization to create an interactive visual art in real-time.
The audience will be able to listen to classical music and at the same time appreciating the meaning of the musical scenes in visual art. The dynamic visual art projected behind the orchestra adds a new dimension to classical music appreciation and it helps the audience to apprehend the intention of the composer and the expression of the performer.
As one of the most common aims of fine Arts and specially in the traditional thinking of Music, one of the earliest uses was in the accompaniment of theatrical dance or story-telling, so it is very usual that composers should from time to time produce what we know as “program music” – music written to portray events, activities or moods such as pastoral scenes or storms.
Thinking on an specific score, what defines a composer to another are a lot of little details that, along the time have been elaborated and transforming in one or other way; so there are some factor in a Mozart score that could never been attributed to Haydn. Those several items are taken carefully by the analysis process in order to distinguish the real parameters for a composer.
Pre-Concert Artistic Process
As an application tool for designing in real time there are different types of artistic parameters we have to consider previously.
Before designing what kind of image we are going to obtain by processing the music as an input signal we have to consider the artistic parameters are embodied as an inherent part of the score, and witch is the most important part of the process.
Violin Bow Tracking:
In-Concert Live Visual Art Creation
We will put two color strips on the violin bow (Figure 1) to track the bowing actions. Expressions like staccato, accent., pizz., can be captured by tracking the bow actions as well as other violin issues: Sautille, alla punta, tallone, sul tasto, sul ponticello, col legno, etc…
Real-time Sound Processing
Dynamics of the music piece like cresc., decresc., p, f, sfp, etc can be tracked using the microphone output and be represented into visual effects.
We will also track and process the pitch and beat of the violin sound in real time. Pitchddle~ patch is designed by Carlos D. Perales on Max MSP, based on fiddle patch design by M. Puckette and R. Climent. This pitch estimator and sinusoidal peak finder the pitch and amplitude of an incoming sound, both continuously and as a stream of discrete “note” events. Fiddle~ optionally outputs a list of detected sinusoidal peaks used to make the pitch determination.
Stop breadboarding and soldering – start making immediately! Adafruit’s Circuit Playground is jam-packed with LEDs, sensors, buttons, alligator clip pads and more. Build projects with Circuit Playground in a few minutes with the drag-and-drop MakeCode programming site, learn computer science using the CS Discoveries class on code.org, jump into CircuitPython to learn Python and hardware together, TinyGO, or even use the Arduino IDE. Circuit Playground Express is the newest and best Circuit Playground board, with support for CircuitPython, MakeCode, and Arduino. It has a powerful processor, 10 NeoPixels, mini speaker, InfraRed receive and transmit, two buttons, a switch, 14 alligator clip pads, and lots of sensors: capacitive touch, IR proximity, temperature, light, motion and sound. A whole wide world of electronics and coding is waiting for you, and it fits in the palm of your hand.
Have an amazing project to share? The Electronics Show and Tell is every Wednesday at 7:30pm ET! To join, head over to YouTube and check out the show’s live chat – we’ll post the link there.
Join us every Wednesday night at 8pm ET for Ask an Engineer!
Maker Business — How 3M is able to ramp up production of N95 masks
Wearables — Flip UV on its head
Electronics — To Y5V or not to Y5V?
Biohacking — Vitamin-C + Gelatin for Accelerated Recovery
Python for Microcontrollers — Virtually Maker Faire, HackSpace Magazine, and more! #Python #Adafruit #CircuitPython @circuitpython @micropython @ThePSF
Adafruit IoT Monthly — Quarantine Clock, Smarter Than your Speaker, and More!
Microsoft MakeCode — Arcade Mini Game in a Text Adventure
No comments yet.
Sorry, the comment form is closed at this time.