Sound – Process

I made research, about link between image and sound. Throught research I was very interested in the representation made using spectogram and the Viridis colormapping (that inspired the use of color in the screenprinting projects).

So I wanted to produce a system based on database, using graphical sound wave form. I began to use the COCO dataset, and to get for each category of image a sound wave representing the speach synthesis from google API text to speach. Once it made all the 80’s categories into image representation. I made a paring between random image of the dataset and their graphical form of representation.

Then, this images paring are used in a Pix2pix algorithms: an algorithm tries to learn to go from the sound wave to the image; It tries to decypher a way to translate from one state to another using convolution (It uses GAN algorithm).

Once trained (about 3-2 days on an RTX2070), I made a small program that listen to the microphone and tranfoms the input into a graphical representation and send it to neural network.