COMPUTATIONAL SOUND DESIGN

Sound Design is a solipsistic process where creativity, errors and consciousness take to a new perceptual results. Can an Artificial Intelligence added in the loop give some benefit?

 

COMPUTATIONAL SOUND DESIGN

Definition:

The term stems from the application of Computational Intelligence to Sound Design. It is an umbrella term for all these algorithms that include machine learning, deep learning, optimization methods and artificial intelligence to help the sound designer in novel and challenging tasks.

This page reports recent papers and resources.

iwan_puni-synthetischerman.jpg

ESTIMATION OF PARAMETERS FOR AN ACCURATE CHARACTERIZATION OF PHYSICAL MODELS

Abstract:

In the process of estimating parameters for acoustic physical models we introduce Deep Learning and Stochastic Methods for the timbre matching problem. Our preferred use case is a flue pipe physical model from Viscount that has more than 60 free parameters. We also explore the high-dimensionality parameter spaces using PCA and T-SNE and conduct subjective listening tests to learn more about user preferences.


Related papers:

  • "A Multi-Stage Algorithm for Acoustic Physical Model Parameters Estimation" (L. Gabrielli, S. Tomassetti, S. Squartini, C. Zinato and S. Guaiana), in IEEE/ACM Transactions on Audio, Speech, and Language Processing, vol. 27, no. 8, Aug. 2019

  • "End-to-End Learning for Physics-Based Acoustic Modeling" (L. Gabrielli, S. Tomassetti, C. Zinato
    and F. Piazza), In IEEE Transactions on Emerging Topics in Computational Intelligence, vol. 2, 2018.

  • "Introducing Deep Machine  Learning for Parameter Estimation in Physical Modelling" (Leonardo Gabrielli, Stefano Tomassetti, Carlo Zinato and Stefano Squartini), In Digital Audio Effects (DAFX), 2017.


Audio Examples: http://a3lab.dii.univpm.it/research/10-projects/84-ml-phymod

nr3dplot-1.png

DEEP LEARNING FOR TIMBRE MODIFICATION AND TRANSFER

Abstract:

In the past years, several hybridization techniques have been proposed to synthesize novel audio content owing its properties from two audio sources. These algorithms, however, usually provide no feature learning, leaving the user, often intentionally, exploring parameters by trial-and-error. The introduction of machine learning algorithms in the music processing field calls for an investigation to seek for possible exploitation of their properties such as the ability to learn semantically meaningful features. In this first work, we adopt a Neural Network Autoencoder architecture and we enhance it to exploit temporal dependencies. In our experiments the architecture was able to modify the original timbre, resembling what it learned during the training phase, while preserving the pitch envelope from the input.

Reference:

  • "Deep Learning for Timbre Modification and Transfer: An Evaluation Study" (Leonardo Gabrielli, Carmine Emanuele Cella, Fabio Vesperini, Diego Droghini, Emanuele Principi and Stefano Squartini), In Audio Engineering Society Convention 144, 2018.

architecture.svg.png
 

©2018 by Leonardo Gabrielli. Proudly created with Wix.com