Does the soft sign have a sound. Drawn sound: from the past to the future Translating image into sound

A creative person is always interested in bold experiments that open up new horizons and opportunities. The fantastic idea - to draw music, to create unique pictures that embody graphics and sound, goes back to the beginning of the 20th century. In this article I will talk about the history of the issue, as well as two of my developments that allow you to do amazing - record and play sounds in a graphical form.

background

In 1904, French inventor Eugene Augustine Last presented a prototype system for optical recording of sound on film, and in 1911 staged what was probably the first ever film screening using the new technique. The era of the decline of silent cinema and revolutionary discoveries in the field of synthetic sound began - for the first time, it was possible to obtain a simple, convenient and very visual way to control audio information.

In the late 1920s, while working on one of the first Soviet sound films, the advantages of such a technique were noted by the composer Arseniy Avraamov, the designer Yevgeny Sholpo, and the director-animator Mikhail Tsekhanovsky. The logical chain was built as follows: if we clearly see a track with a recorded sound wave, then we can create the same wave artificially, simply by drawing it by hand. But what if you put an ornament there, a complex combination of patterns or primitives of Euclidean geometry? How fantastic will the result be? After all, in this way you can draw a completely unique sound that does not exist in nature, and you can write music without real instruments, microphones and performers.

Several laboratories were soon engaged in the study of these issues. And as a result, synthesizers of an optical soundtrack appeared: Evgeny Sholpo's "Variophone", Boris Yankovsky's "Vibroexponator", Nikolai Voinov's machine for marking "combs" from paper - the basic fragments of the synthesized sound. To the ear, all this was very reminiscent of modern 8-bit music, but with a greater degree of freedom: any form of oscillation, unlimited polyphony, the most unimaginable rhythmic patterns. Just think about it - an optical synthesizer, a music computer in the thirties of the last century! But these are just flowers. The thought of Soviet engineers went further.

Unlike his colleagues, acoustician Boris Yankovsky was one of the first to realize that in order to create complex, close to living sounds, it is not enough to describe only one form of oscillation. The most important part of acoustic information is the spectrum, which clearly defines the frequency composition of the sound, its color, according to which we give such subjective definitions as bright, warm, metallic, similar to a human voice, and so on.

Yankovsky began to structure the basic spectrum graphs into a kind of "Mendeleev's table" of sound elements, while simultaneously developing algorithms for their processing and hybridization to obtain new sounds based on "spectrostandards". Unfortunately, the changes in the country and the war did not allow Yankovsky to bring the work to its logical conclusion.

The topic was continued by his friend, a young inventor Evgeny Murzin, who was impressed by the developments in the field of “graphic sound” and conceived a grandiose project - a universal photoelectronic machine capable of synthesizing any sound, any musical system by drawing a spectrogram (spectrum versus time) on a special canvas without distracting operations. like developing and drying the film. This would simplify the painstaking work of the composer, providing unprecedented freedom for creativity.

Literally on his knees, working in the evenings in the room of a two-story barracks, Murzin completed a working model of the device in 1958. The device weighed more than a ton and outwardly had little in common with a musical instrument in the classical sense. The invention was named "ANS" in honor of the composer Alexander Nikolayevich Scriabin. Despite its appearance, ANS became a worldwide sensation, decades ahead of its time and very well fitting into the period of space euphoria with its unique atmospheric sound.

The ANS is somewhat reminiscent of a modern scanner, only it is not the scanning strip that moves in it, but the surface itself with the image - a large glass plate (score), covered with opaque paint. The paint in the right places is removed with a thin chisel, forming a spectrogram pattern of a musical work. The score moves smoothly, passing over a hole from which an intermittent "modulated" beam of light comes from an optical-mechanical generator of pure sound tones based on five special discs of an optical phonogram. Part of the light passes through the transparent areas of the score, after which it is focused on a set of photocells, from which sound is ready to be played in the form of electric current oscillations.

The heart of ANS is the aforementioned disk with a pattern of 144 tracks (like on a gramophone record), the transparency of which changes along a sinusoid with a certain frequency. The frequency difference between adjacent tracks is 1/72 of an octave. Thus, one disk contains two octaves, and an octave is divided into 72 pure tones - Murzin considered the classical 12-tone temperament to be a significant limitation. Essentially, each disc is an optical implementation of the Fourier Transform algorithm that underlies many modern software synthesizers and effects. This is in the order of things now, in the days of gigahertz and gigabytes, but 50 years ago it was simply incredible - a spectral synthesizer capable of playing 720 pure tones at the same time! No wonder the ANS is considered the world's first polyphonic musical synthesizer.

If you think you've never heard the sounds of the ANS before, you're probably wrong. Just remember Andrey Tarkovsky's films "Solaris", "Mirror", "Stalker", which captivate with the magic music of Eduard Artemiev. Or a nightmare scene from Leonid Gaidai's comedy The Diamond Arm. It is worth noting that Artemiev's career as an electronics composer began precisely with his acquaintance with the ANS and its creator in 1960. In addition to Artemiev, Alfred Schnittke, Edison Denisov, Sofia Gubaidulina, Stanislav Kreichi managed to work with the instrument, and the sounds of ANS at different times were used in their music by such groups as Coil and Bad Sector.

Unfortunately, only one copy of the ANS synthesizer, manufactured industrially at the end of 1963, has survived to this day. It is located in Moscow in the Glinka State Museum of Musical Culture. Despite the difficult fate, the device is still in working condition and from time to time plays for museum visitors under the strict supervision of Stanislav Kreichi. For those who are far from Moscow or just want to experiment with the sound of ANS at home, there is a software simulator called Virtual ANS.

Virtual ANS: graphics editor

The development of Virtual ANS has been carried out by the author of this article since 2007. The purpose of the program is to recreate as much as possible the key features, the atmosphere of the iron ANS, while expanding the original idea, taking into account the rich capabilities of modern computers. Of the main differences:

• the program is cross-platform (Windows, Linux, OS X, iOS, Android), which allows you to enjoy working with the instrument anywhere and on anything: from a cheap phone to a powerful studio computer;
• the number of basic pure tone generators is now limited only by the user's imagination and the speed of the central processor;
• it became possible to convert back from sound to spectrum.

Virtual ANS is a graphics editor with a classic set of tools: primitives, brushes, layers, effects, loading/saving PNG, GIF, JPEG. But the picture that you see on the screen is in fact the score of a musical work (aka sonogram or spectrogram), which at any moment can be listened to or listened to and drawn at the same time. The score decomposes the composition into "sound atoms" - indivisible pieces of pure tones (sinusoidal oscillations). Horizontally - time axis X (left to right). Vertical - pitch Y (from bottom to top from bass to treble). The brightness of a single pixel is the loudness of a pure tone with frequency Y at time X. The spectrum image is vertically divided into octaves, an octave into 12 semitones, a semitone into even smaller, barely audible microtones, for an accurate description of any musical scale, any the most unthinkable tone. If we draw a horizontal line one pixel thick on the ANS score, we will hear a single sinusoid with a constant frequency. The thicker the line - the more pure tones will be included in its composition, the more complex the sound will be, and the stronger the sound will approach white noise, saturated with overtones of all frequencies of the audible range. The combination of such lines with other figures of different brightness gives unexpected and interesting sound variations.

In the process of working on Virtual ANS, a curious thought came up. A fragment of an audio file or, say, a voice recording from a microphone can be converted into an ANS score, that is, into a spectrogram - a picture with sound encoded in it. And this sound can be easily reproduced back using the same program. There is a natural desire to print a picture of the spectrum on a printer and get a paper copy of your voice or music.

It is for these purposes that PhonoPaper was conceived - another project that inherits the ideas of the sound revolutionaries of the last century. What is PhonoPaper?

1. The image format in which the audio is encoded. This code differs from the ANS spectrogram only in that special markers appeared above and below, by which the reader accurately determines the boundaries of the block with the spectrum.
2. Scanner application for reading PhonoPaper codes in real time using the camera.
3. Recorder app to convert 10 seconds of audio into PhonoPaper code. Although for more precise control of the transformation, it is best to use the Virtual ANS described above.

PhonoPaper-code can be called analog, since it contains no digital information, and it can be recorded on any accessible surface (paper, plastic, wood). For this reason, various kinds of distortions are not critical for him: in poor lighting and crumpled paper, you will at least hear the “outlines” of the original message. To listen to the code, no access to the network is required - all the necessary information is stored directly on the picture, and playback starts immediately after entering the camera's field of view. At the same time, as in Murzin's ANS synthesizer, the user controls the speed and direction of the game by manually scanning the sound code (although there is also an automatic mode).

Does it make any practical sense? Imagine: sound prompts in children's books or textbooks; a piece of a new song on a disc or an advertising poster for the group; audio tags on goods; secret messages on the walls of buildings; sound cards and all sorts of art experiments. This would make sense if there was a very simple way to read such images. After all, it needs to be photographed, loaded into the program and accurately indicate the boundaries of the spectrum, the base frequency and the number of octaves.

Instructions for use

1. Install the PhonoPaper app on your iPhone or Android smartphone.

1. Launch the application.
2. Point to each track.

Instead of a conclusion

As you can see, the next turn of the spiral brings us back to the origins. And this is natural, because the world today is oversaturated with information processing processes hidden from humans and is increasingly immersed in the virtual space, digitized, encoded and packaged. Musical instruments hide their nature, you cannot touch them or look under the lid to touch the magic of the birth of a new sound, to feel its energy. Drawing music at the "atomic" level and transferring this process to the real world is undoubtedly a big step towards reducing the distance between the composer and the embodiment of his creative ideas. At the same time, the creation of music becomes accessible to lovers and representatives of related arts, we are no longer limited by strict limits and rules, and musical notation is now only an addition. We take a pen, paper and start creating a new masterpiece.

A person is faced with speech from birth. Initial acquaintance occurs with sounds. Speech sounds are what we make when we speak. We hear them when other people speak.

Acquaintance with letters begins later. We write letters and see when we read the written text.

Sound is impossible to write and see. And you can't pronounce the letter. But each letter has its own name: "A", "Be", "Er", "Sha". And they are needed in order to designate sounds in writing.

If we try to pronounce the sound that is indicated by the sign “b” in writing, then we will not succeed. At best, the name of the letter "Soft sign" will sound. But no sound is a soft sign. In Russian, he has a completely different role.

What is a soft sign for?

Despite the fact that this letter does not designate a sound, it has several functions in Russian.

An indicator of the softness of a consonant sound. If in a written word a soft sign is after the letter denoting a consonant, then this sound is pronounced softly when reading. An example showing the difference in the pronunciation of sounds denoted by the same letter, with and without a soft sign, can be the words "dal" and "dal".

separating function. In writing, a soft sign separates the letter denoting a consonant sound and the vowels I, E, E, Yu, I. At the same time, the consonant sound is read softly, and the indicated vowels denote two sounds: I - [Y, A]; E - [Y, E]; Yo - [Y, O]; Yu - [Y, Y]; And - [Y, I].

Designation of grammatical forms of words. At the end of feminine singular nouns (3 declensions), a soft sign is written.

It is also written in indefinite verbs, incl. before TSY. A soft sign is used in all verb forms after hissing and in verbs in the imperative mood, as well as in present and future tense verbs in the second person singular.

If the basis of the adverb ends in a hissing, they also write this letter.

And although the letter “Soft sign” itself does not mean any sound, it has a great influence on the pronunciation of consonant sounds.

In general, I recently decided to look at Wikipedia (a truly inexhaustible storehouse of knowledge), and came across the definition of a spectrogram there. As it turned out, there is something to borrow on the topic of drawing with sound. Firstly, this is a list of programs that allow you to synthesize sound from pictures. The list includes the program Coagula, which we know very well (by the way, it is in our subsection with software), but there are others, namely:

• MetaSynth for Macintosh;
• Coagula for Windows
• FL Studio's "BeepMap" additive synthesizer.

An open source multi-platform project showcases fun but highly educational experiences. The program allows you to turn sound into a spectral picture (with the specified resolution), and vice versa, synthesize sound from the picture (with the specified parameters).

Another pranksters are the band Plaid. In the song, "3recurring" contains the logo in its spectrogram.

And Nine Inch Nails also use the technique of hiding pictures in the spectrum of tracks from the album "Year Zero".

In general, this method obviously appealed to some musicians. In principle, the same method can be easily used as a steganography tool.

This topic is very interesting and, I think, there will be many more findings related to drawing in the spectrum and sounding pictures.