Datamoshing with Ableton Live and modern plugins using sox and convert

My first introduction to datamoshing started with posts I read online about using Audacity to use their in-built audio effects and this idea that “all audio processing basically involves manipulating an audio stream in a repetitive and consistent manner” could also be used on non-audio streams. This was back in school and I tried doing something like it on my phone but aside from all the glitch apps that essentially have a preset method, there was nothing much I could do. So I left it there, until now.
Note: All the images shown here have been first processed through crunch
and then converted to jpeg
and aggressively compressed so that this blogpost can load within reasonable time, the high quality versions (and scripts) are available at https://github.com/kittywhiskers/ramen
The specific commands were:
find . -mindepth 1 -type f -name '*.png' -exec mogrify -format jpg {} \;
find . -mindepth 1 -type f -name '*.png' -exec rm {} \;
find . -mindepth 1 -type f -name '*.jpg' -exec jpegoptim --strip-all --size=100k {} \;
ramen.sh
is a bash script that uses sox
and convert
(a part of ImageMagick) to convert PNG images to a RAW RGB image (you can tell ImageMagick this by setting the extension, where I used .rgb
, .rgba
and .grey
are also interesting options to explore), then pretending it is RAW audio and then passing it through sox
to get a wav
file sampled at 176400 Hz (and additionally any format that sox
supports like mp3
, ogg
, au
, but not aac
unfortunately).
Note: The script on GitHub will sample at 44100 Hz because sox
does not accept obscenely high sample rates for lossy formats, there was no found quality difference between picking sample rate (and converting back and forth consistently, but changing it mid-conversion would be interesting and untested)
I found the usage of .rgba
was troubling when I wanted to pass a png
processed file with ffmpeg
instead of convert
(which used to happen because of unexpected end-of-file
errors by convert
), so I used .rgb
instead (interestingly, with rgba64le
instead of what seems obvious - rgb48le
, that is because parts of the image were getting cropped).
Playing with lossy formats⌗
Original Image | Lossy Compression using llf and 320kbps mp3 |
Lossy Compression using lff and 320kbps ogg |
Lossy compression using llf and the deprecated au format |
---|---|---|---|
![]() |
![]() |
![]() |
![]() |
![]() |
![]() |
![]() |
![]() |
![]() |
![]() |
![]() |
![]() |
![]() |
![]() |
![]() |
![]() |
Playing with lossless formats⌗
Original Image | Converting an image using wav (with bit depth 24) |
Converting an image using wav (with bit depth 32) |
---|---|---|
![]() |
![]() |
![]() |
![]() |
![]() |
![]() |
![]() |
![]() |
![]() |
![]() |
![]() |
![]() |
Based on the results, we decided to go with sdpth24
for it’s intact physical detail and aesthetic as the default preset to generate the asset audio that will be processed by Live. Even though the Q_DEPTH
is 32
and it seems natural to have S_DEPTH=$Q_DEPTH
, that resulted in images with worse visual fidelity. Oddly, S_DEPTH=32
and S_DEPTH=16
result in almost visually identical files, even though I was expecting some sort of degradation.
Playing with Ableton Live⌗
For using Ableton Live, we used the above export configuration and ran all the reference files through an export with an empty effects chain to check if it matches with sdpth24
, after confirming it does, we start experimenting
Playing with Stock Plugins⌗
Ableton Live’s built-in plugins were why I used Live as a host (for better or for worse, it doesn’t like header-less audio). For Live, we used a sample rate of 176400 Hz
, the export process had to involve making the track mono, otherwise ImageMagick would detect two images in the stream and print them out as [imagename]-0.png
and [imagename]-1.png
. Similarly, we tried to prevent any other sources of influence (except the plugin), so normalisation and dithering were disabled.
The most promising results came from Beat Repeat, Grain Delay and the VR Phat Rack, we intentionally kept signal mix at 10% (except for plugins which already forward a significant amount of the original signal or plugins that didn’t have a dry/wet knob, I could have rigged an Effect Rack to fix that but ¯\_(ツ)_/¯
)
Glue Compressor | Beat Repeat | Crow’s Phat Rack |
---|---|---|
![]() |
![]() |
![]() |
Aren’t they beautiful?
Original Image | Glue Compressor | Beat Repeat | VR Phat Rack |
---|---|---|---|
![]() |
![]() |
![]() |
![]() |
![]() |
![]() |
![]() |
![]() |
![]() |
![]() |
![]() |
![]() |
![]() |
![]() |
![]() |
![]() |
Reverb | Vocoder | Corpus |
---|---|---|
![]() |
![]() |
![]() |
Reverb didn’t result anything that seems unlike noise (but it does show interesting things when transformed back using llf
instead of wav
) so it has been omitted for now but I’ll be keeping the column for later.
Original Image | Reverb | Vocoder | Corpus |
---|---|---|---|
![]() |
Nothing notable rendered with wav |
![]() |
![]() |
![]() |
Nothing notable rendered with wav |
![]() |
![]() |
![]() |
Nothing notable rendered with wav |
![]() |
![]() |
![]() |
Nothing notable rendered with wav |
![]() |
![]() |
Grain Delay | Multiband Compression | Redux |
---|---|---|
![]() |
![]() |
![]() |
Unlike Reverb, I felt that I should do as little changes as possible with Grain Delay and the results are stunning.
Original Image | Grain Delay | Multiband Compression | Redux |
---|---|---|---|
![]() |
![]() |
![]() |
![]() |
![]() |
![]() |
![]() |
![]() |
![]() |
![]() |
![]() |
![]() |
![]() |
![]() |
![]() |
![]() |
Playing with External VSTs⌗
MeldaProduction and Tritik⌗
MeldaProduction MBitFun | MeldaProduction MComb | MeldaProduction MAutoPitch |
---|---|---|
![]() |
![]() |
![]() |
Tritik Krush | ||
![]() |
No matter what preset or wetness in MeldaProduction MAutoPitch, I couldn’t get it to give it an image at all and it is therefore omitted and I got rid of the files.
Original Image | MeldaProduction MBitFun | MeldaProduction MComb | Tritik Krush |
---|---|---|---|
![]() |
![]() |
![]() |
![]() |
![]() |
![]() |
![]() |
![]() |
![]() |
![]() |
![]() |
![]() |
![]() |
![]() |
![]() |
![]() |
Kilohearts⌗
kHs Flanger | kHS Bitcrush |
---|---|
![]() |
![]() |
Using plugins that come along with PhasePlant, using a very minimal amount of flanger brought with it very interesting results. The same thing cannot be said for bitcrush, which seems to be dominated with noise and rough outlines. I attempted to use kHS Ensemble and a few other plugins, which produced results similar to kHS Bitcrush so I omitted them and got rid of the files.
Original Image | kHS Flanger | kHS Bitcrush |
---|---|---|
![]() |
![]() |
![]() |
![]() |
![]() |
![]() |
![]() |
![]() |
![]() |
![]() |
![]() |
![]() |
XFer Records⌗
I admittedly placed this last because the results from LFOTool were not that impressive. We used 1/4 at 80bpm
and it doesn’t really seem that distinguishable from noise. This wasn’t the first result to show noise like results but it was the first one where convert
has errored out and could not complete the conversion back to image for the 8000x4500
image.
Converting file "./8000x4500.wav" to "8000x4500.raw" with Sox
Converting file "8000x4500.raw" to "8000x4500.png" with ImageMagick
convert: unexpected end-of-file '8000x4500.rgb': No such file or directory @ error/rgb.c/ReadRGBImage/247.
convert: no images defined `8000x4500.png' @ error/convert.c/ConvertImageCommand/3282.
Apparently LFOTool ducked away parts of the footer so we’re left with only three noisy files.
Original Image | LFOTool |
---|---|
![]() |
![]() |
![]() |
![]() |
![]() |
![]() |
![]() |
LFOTool vored the file footer :( |
Conclusion⌗
This is fun. 3.5/5