Markov chains based strategies for additive synthesis using Go
Introduction
TBD
Experiment #1
The first iteration should be considered, to borrow a software world term, a proof of concept. There was not a thoughtful process behind the aesthetics of the produced sounds but rather the focus was to come up with a working strategy of generating them. Further experiments will address the shortcomings of the first one.
Experiment #1 comes in the form of a small command line program that generates audio based on markov chains models. It has five available flags. -debug
to enable debug logging - useful while developing, -ngen
the number of successive generations to generate, -files
the directory to save the audio files of each generation, -models
the directory to save the generated markov chains models and -seed
the seed markov chain models to kick start the process.
Running it produces folders containing the generated audio and the Markov chain models used for the generation in json format.
βββ gen0
β βββ 0.wav
β βββ 1.wav
β βββ amp.json
β βββ dur.json
β βββ freq.json
βββ gen1
β βββ 0.wav
β βββ 1.wav
β βββ amp.json
β βββ dur.json
β βββ freq.json
βββ gen2
β βββ 0.wav
β βββ 1.wav
β βββ amp.json
β βββ dur.json
β βββ freq.json
...
Note that to run the program you need to either install go
or download the binary named markov1
from the moduleβs github under releases.
Under the hood the CLI uses the markov
package to generate a βtrainβ of sines waves. Experiment #1 uses additive synthesis to produce audio signal. The result is a monophonic synth. Each fundamental is treated for harmonics. Harmonics are read of a corresponding table and for iteration #1 are static and the same for each fundamental. Each generation uses the previous generationβs models to generate new values for the sine waves. Gen0 uses the seed models.
flowchart TD
a["Ngen"] -->
c["freq.json"] -->
d{"`**Markov Generator**
`"} -->
g["`**Create Train**
Combine the generated
frequencies, amplitudes and durations to conscutive sines.`"]
a -- Load the Markov models ---
e["amp.json"] -->
d
g
a -->
h["dur.json"] -->
d
g
g -->
k["`**Harmonics**
Create harmonics for each sine of the train.`"]
k -->
l["`**Generate Audio**
Generate audio signal for fundamentals and partials and add them together.`"]
l -->
p["Generate New Models"]
p -.-> a;
Strategy
Experiment #1 uses Markov Chains to generate variations of the initial seed. The starting point is the three seed models (frequencies, for amplitudes, durations.) Each model is read and fed into a Markov generator.
"0.000000": 137,
"1.000000": 206,
"10.000000": 94,
"10.148148": 123,
"10.232558": 93,
"10.301887": 122,
"10.461538": 121,
"10.476190": 92,
"10.627451": 120,
"10.731707": 91,
"10.800000": 119,
Sample values from the seed freq.json model.
flowchart TD
a["`**Freq Model**
0.000000
1.000000
10.000000
10.148148
10.232558
10.301887
10.461538
10.476190
10.627451
10.731707
10.800000
...`"]
b{Markov Generator}
a -- Load Model --- b
Each individual value of the model is fed into the generator until either a. the generator stops producing new values or b. the produced values stop being unique and start looping (e.g. 0.000000, 1.000000, 10.000000, 0.000000, 1.000000, 10.000000 ...
.)
flowchart TD
a{Markov Generator}
b["`**Freq Model**
0.000000
...`"]
b -- Feed First Value --- a
a --> c["`**Generated Values**
10.148148
2100.000000
19.130435
20.952381
2750.000000
...`"]
This produces an array of float values. Once the subprocess is over the generator is fed the next value of the model 1.000000
and generates a new array of float values. The process continues until we feed all values of the model to the generator.
The end result is three arrays of arrays [][]float
. Then the program reads through all generated values of frequencies, amplitudes and durations and creates a train of Sines.
type Sine struct {
Frequency float64
Amplitude float64
Duration time.Duration
}
flowchart TD
a["`Freqs
[0][0.000000, 1.000000 ...]
[1][10.76190, 19.13043 ...]
...`"]
b["`Amps
[0][0.510000, 0.350000 ...]
[1][0.450000, 0.510000 ...]
...`"]
c["`Durs
[0][129, 130 ...]
[1][143, 150 ...]
...`"]
d{Sine Constructor}
a & b & c --> d
d --> e
e["`Sine Train
Sine {
Frequency: Freqs[0][0],
Amplitude: Amps[0][0],
Duration: Durs[0][0]}
<br>Sine {
Frequency: Freqs[0][1],
Amplitude: Amps[0][1],
Duration: Durs[0][1]}
<br>Sine {
Frequency: Freqs[0][2],
Amplitude: Amps[0][2],
Duration: Durs[0][2]}
...`"]
At this point we have a representation of monophonic consecutive pure sine waves. The next step is to generate the audio signal based on them. Along the fundamentals we generate the partials for each and add them together.
2 0.02
3 0.03
4 0.04
5 0.05
6 0.06
7 0.07
8 0.08
9 0.09
10 0.1
11 0.11
12 0.12
...
Excerpt of the harmonics table. First column is the partial (the 2nd, the 3rd etc) and the second column is the factor to multiply against the fundamental amplitude to derive the amplitude of the partial.
The final step is to save the generated audio files and export the new models that will be used as seeds for the next n generation.
Seed
Generator
Result
Limitations
The code so far is very crude and suffers from many limitations. Limitations here are understood as impediments to reaching a richer and/or more controlled audio signal. The list bellow is not exhaustive but will be used as a guide for Experiment #2.
- Crude seed
The way the algorithm works the initial seed has the biggest influence on the end result regardless of how many generations past it we are. The two solutions are a. more complex seeds to begin with, b. a new strategy for combining multiple seeds in the generation process.
- Monophonic
The algorithm produces mono signal (one channel.) We managed to achieve stereo image by combining three generations and panning them. Modification are needed to auto generate stereo, quadraphonic audio etc.
- Monophony
The procedure of the audio signal generation is such that the end result can not be polyphonic. Modification are needed to achieve polyphony.
- Static harmonics
The same harmonics table is applied to all Sines of the train. A proper strategy is needed for dynamic harmonics.
- Generation sequence
Each modelβs values are sorted (see Sample values from the seed freq.json model. above.) For example value 0.000000
is fed to the Markov Generator. When the process finishes value 1.000000
is fed and so on. This has profound effect on the audio signal generated. A new strategy is needed to make this process more dynamic.
- Substitutions in Sine Constructor
As described previously, the Markov Generator is used to produce three arrays of arrays. Those are fed to the Sine Constructor. At the moment Constructor favours heavily the generated result of frequencies. This happens in two major ways. When a frequencies sub array (eg. Freqs[0]
) reaches the end, it ignores the remaining amplitudes and durations arrays (if any.) While reading the Freqs[0]
sub array if len(Freqs[0] > len(Amps[0])
, that is there is no corresponding amplitude value for the next frequency of the Sine Train, hardcoded default value 0 is used (no signal.) Same applies for duration, default hard coded value when there are no duration value left is 10 milliseconds.