August 2022

2022-08-10, Thursday

Back at work for almost two weeks already. Mostly preparing for teaching functional programming.

Generative art

I have discovered MidJourney, one of these machine learning image generation tools where you write a prompt and it generates an image. It is really fun! Of course it has its strengths and weaknesses. But, more than with other “AI things”, I feel like this has real, human value. Also, the community around MidJourney is exciting.

Generative art is nothing new. The core of generative art is that you write an algorithm which generates the art, rather than “performing it” yourself. Often you incorporate some source of randomness into the process, so that you can get a number of variations on the same theme. The algorithm can also take some user input, which affects the result in some way.

In the case where you wrote the algorithm, the result is clearly art you have made. But even if you are merely using someone else’s algorithm, you might claim that the result is art you have made, if one or a combination of the follow two cases hold:

1. Your input to the algorithm enabled your artistic expression.
2. Your selection of this output from many randomly generated outputs enabled your artistic expression.

Musical example

Here is a simple example from music of the second case: In modular synthesis you make music by connecting modules together to form a synthesiser.

Often the notes played is determined by a sequencer module, instead of using a keyboard. Some sequencers allow you to program in a sequence of notes played manually, but there is a sequencer called “Turing Machine”¹ which generates random sequences of notes.

The crucial thing about the Turing Machine is that it will play random notes until you lock it. Once the sequence is locked it will keep playing the last generated sequence until you unlock it. So, to produce a cool bassline for your track, you connect the Turing machine to the bass modules, and listen to its random playing until you find something you like. Then you lock it in, and that’s it! Clearly it is your bassline because you chose it among all the ones generated by the Turing Machine.

Here is a track where Mutable Instruments’ Marbles, another module like the Turing Machine², is used to decide the melody played:

The theme you hear repeat a few times throughout track was chosen by just letting Marbles “roll the dice” until a catchy melody came out. Then I was free to focus on other parts, such was how to make the most bad-ass distorted sound to play that melody with.

BTW, that distorted sound is a harmonic oscillator (sum of related sine waves) modulating a through-zero FM oscillator. Controlling the distribution of sines gives absolutely interesting shifts in timbre!

DALL-E, MidJourney and others

Currently, there are several competing prompted-image-generators. But the idea is that you give a short prompt such as “The invention of pizza by god, Italian renaissance painting”, and the program spits out:

In MidJourney the output is actually four images, and then you can iterate, making variations on each of them. This means that both the first and the second point above apply. The user provides both input to the algorithm, and selects from a supply of randomly generated output.

In both ways this enables the user to express themselves artistically. And this is the value I see in this: it enables more people to express themselves creatively using computer graphics. For instance, I will use MidJourney when creating concept art for my D&D sessions. The dessert mind flayer above is way better drawn that I would have been able to if I sat down in Krita or Blender.

Now, I don’t think MidJourney or DALL-E are any threat to actually skilled artists. A human artist actually understands what they are drawing, and can be relied upon in a completely different way. The pictures generated by MidJourney usually have very recognizable artefacts. And most importantly, the control you have when generating the picture is very tenuous. You might have a perfect idea in your head, but the algorithm is not a mind reader, so the picture coming out is usually completely different. Sometimes that’s OK, and a cool feature. But for most of the kind of work you pay an artist for, it would be completely useless.

Also, artists often produce 3D models, which can be rendered in different settings and reposed. This has very high utility compared to the lump of pixels from MidJourney.