Mixed Reality Lab- Classical Meets Media

Via Mixed Reality Lab

The Mixed Reality Lab (MXR) aims to push the boundaries of research into interactive new media technologies through the combination of technology, art, and creativity.

In Classical Music Meets Interactive Media, The main goal of this project is linking classical music and visual art through a real-time processing on a violin performance.

A meaningful 3D visualization will be correspondence within the original thinking of the composer after an exhaustive analysis of the score for the concert attending to the different parameters of the music language: harmony, counterpoint, instrumentation, formal analysis, style and background of the composer. The violin performer’s expression will be interacting with the music and the 3D visualization to create an interactive visual art in real-time.

The audience will be able to listen to classical music and at the same time appreciating the meaning of the musical scenes in visual art. The dynamic visual art projected behind the orchestra adds a new dimension to classical music appreciation and it helps the audience to apprehend the intention of the composer and the expression of the performer.

Description

As one of the most common aims of fine Arts and specially in the traditional thinking of Music, one of the earliest uses was in the accompaniment of theatrical dance or story-telling, so it is very usual that composers should from time to time produce what we know as “program music” – music written to portray events, activities or moods such as pastoral scenes or storms.

Thinking on an specific score, what defines a composer to another are a lot of little details that, along the time have been elaborated and transforming in one or other way; so there are some factor in a Mozart score that could never been attributed to Haydn. Those several items are taken carefully by the analysis process in order to distinguish the real parameters for a composer.

Pre-Concert Artistic Process

As an application tool for designing in real time there are different types of artistic parameters we have to consider previously.

Before designing what kind of image we are going to obtain by processing the music as an input signal we have to consider the artistic parameters are embodied as an inherent part of the score, and witch is the most important part of the process.

Violin Bow Tracking:

In-Concert Live Visual Art Creation
We will put two color strips on the violin bow (Figure 1) to track the bowing actions. Expressions like staccato, accent., pizz., can be captured by tracking the bow actions as well as other violin issues: Sautille, alla punta, tallone, sul tasto, sul ponticello, col legno, etc…

Real-time Sound Processing

Dynamics of the music piece like cresc., decresc., p, f, sfp, etc can be tracked using the microphone output and be represented into visual effects.

We will also track and process the pitch and beat of the violin sound in real time. Pitchddle~ patch is designed by Carlos D. Perales on Max MSP, based on fiddle patch design by M. Puckette and R. Climent. This pitch estimator and sinusoidal peak finder the pitch and amplitude of an incoming sound, both continuously and as a stream of discrete “note” events. Fiddle~ optionally outputs a list of detected sinusoidal peaks used to make the pitch determination.

See more!


Have an amazing project to share? The Electronics Show and Tell is every Wednesday at 7:30pm ET! To join, head over to YouTube and check out the show’s live chat and our Discord!

Join us every Wednesday night at 8pm ET for Ask an Engineer!

Join over 38,000+ makers on Adafruit’s Discord channels and be part of the community! http://adafru.it/discord

CircuitPython – The easiest way to program microcontrollers – CircuitPython.org


New Products – Adafruit Industries – Makers, hackers, artists, designers and engineers! — New Products 11/15/2024 Featuring Adafruit bq25185 USB / DC / Solar Charger with 3.3V Buck Board! (Video)

Python for Microcontrollers – Adafruit Daily — Python on Microcontrollers Newsletter: A New Arduino MicroPython Package Manager, How-Tos and Much More! #CircuitPython #Python #micropython @ThePSF @Raspberry_Pi

EYE on NPI – Adafruit Daily — EYE on NPI Maxim’s Himalaya uSLIC Step-Down Power Module #EyeOnNPI @maximintegrated @digikey

Adafruit IoT Monthly — The 2024 Recap Issue!

Maker Business – Adafruit Daily — Apple to build another chip at TSMC Arizona

Electronics – Adafruit Daily — SMT Tip – Stop moving around!

Get the only spam-free daily newsletter about wearables, running a "maker business", electronic tips and more! Subscribe at AdafruitDaily.com !


No Comments

No comments yet.

Sorry, the comment form is closed at this time.