My first introduction to datamoshing started with posts I read online about using Audacity to use their in-built audio effects and this idea that “all audio processing basically involves manipulating an audio stream in a repetitive and consistent manner” could also be used on non-audio streams. This was back in school and I tried doing something like it on my phone but aside from all the glitch apps that essentially have a preset method, there was nothing much I could do. So I left it there, until now.

Note: All the images shown here have been first processed through crunch and then converted to jpeg and aggressively compressed so that this blogpost can load within reasonable time, the high quality versions (and scripts) are available at https://github.com/kittywhiskers/ramen

The specific commands were:

find . -mindepth 1 -type f -name '*.png' -exec mogrify -format jpg {} \;
find . -mindepth 1 -type f -name '*.png' -exec rm {} \;
find . -mindepth 1 -type f -name '*.jpg' -exec jpegoptim --strip-all --size=100k {} \;

ramen.sh is a bash script that uses sox and convert (a part of ImageMagick) to convert PNG images to a RAW RGB image (you can tell ImageMagick this by setting the extension, where I used .rgb, .rgba and .grey are also interesting options to explore), then pretending it is RAW audio and then passing it through sox to get a wav file sampled at 176400 Hz (and additionally any format that sox supports like mp3, ogg, au, but not aac unfortunately).

Note: The script on GitHub will sample at 44100 Hz because sox does not accept obscenely high sample rates for lossy formats, there was no found quality difference between picking sample rate (and converting back and forth consistently, but changing it mid-conversion would be interesting and untested)

I found the usage of .rgba was troubling when I wanted to pass a png processed file with ffmpeg instead of convert (which used to happen because of unexpected end-of-file errors by convert), so I used .rgb instead (interestingly, with rgba64le instead of what seems obvious - rgb48le, that is because parts of the image were getting cropped).

Playing with lossy formats⌗

Original Image Lossy Compression using llf and 320kbps mp3 Lossy Compression using lff and 320kbps ogg Lossy compression using llf and the deprecated au format

Playing with lossless formats⌗

Original Image Converting an image using wav (with bit depth 24) Converting an image using wav (with bit depth 32)

Based on the results, we decided to go with sdpth24 for it’s intact physical detail and aesthetic as the default preset to generate the asset audio that will be processed by Live. Even though the Q_DEPTH is 32 and it seems natural to have S_DEPTH=$Q_DEPTH, that resulted in images with worse visual fidelity. Oddly, S_DEPTH=32 and S_DEPTH=16 result in almost visually identical files, even though I was expecting some sort of degradation.

Playing with Ableton Live⌗

For using Ableton Live, we used the above export configuration and ran all the reference files through an export with an empty effects chain to check if it matches with sdpth24, after confirming it does, we start experimenting

Playing with Stock Plugins⌗

Ableton Live’s built-in plugins were why I used Live as a host (for better or for worse, it doesn’t like header-less audio). For Live, we used a sample rate of 176400 Hz, the export process had to involve making the track mono, otherwise ImageMagick would detect two images in the stream and print them out as [imagename]-0.png and [imagename]-1.png. Similarly, we tried to prevent any other sources of influence (except the plugin), so normalisation and dithering were disabled.

The most promising results came from Beat Repeat, Grain Delay and the VR Phat Rack, we intentionally kept signal mix at 10% (except for plugins which already forward a significant amount of the original signal or plugins that didn’t have a dry/wet knob, I could have rigged an Effect Rack to fix that but ¯\_(ツ)_/¯)

Glue Compressor Beat Repeat Crow’s Phat Rack

Aren’t they beautiful?

Original Image Glue Compressor Beat Repeat VR Phat Rack
Reverb Vocoder Corpus

Reverb didn’t result anything that seems unlike noise (but it does show interesting things when transformed back using llf instead of wav) so it has been omitted for now but I’ll be keeping the column for later.

Original Image Reverb Vocoder Corpus
Nothing notable rendered with wav
Nothing notable rendered with wav
Nothing notable rendered with wav
Nothing notable rendered with wav
Grain Delay Multiband Compression Redux

Unlike Reverb, I felt that I should do as little changes as possible with Grain Delay and the results are stunning.

Original Image Grain Delay Multiband Compression Redux

Playing with External VSTs⌗

MeldaProduction and Tritik⌗

MeldaProduction MBitFun MeldaProduction MComb MeldaProduction MAutoPitch
Tritik Krush

No matter what preset or wetness in MeldaProduction MAutoPitch, I couldn’t get it to give it an image at all and it is therefore omitted and I got rid of the files.

Original Image MeldaProduction MBitFun MeldaProduction MComb Tritik Krush

Kilohearts⌗

kHs Flanger kHS Bitcrush

Using plugins that come along with PhasePlant, using a very minimal amount of flanger brought with it very interesting results. The same thing cannot be said for bitcrush, which seems to be dominated with noise and rough outlines. I attempted to use kHS Ensemble and a few other plugins, which produced results similar to kHS Bitcrush so I omitted them and got rid of the files.

Original Image kHS Flanger kHS Bitcrush

XFer Records⌗

I admittedly placed this last because the results from LFOTool were not that impressive. We used 1/4 at 80bpm and it doesn’t really seem that distinguishable from noise. This wasn’t the first result to show noise like results but it was the first one where convert has errored out and could not complete the conversion back to image for the 8000x4500 image.

Converting file "./8000x4500.wav" to "8000x4500.raw" with Sox
Converting file "8000x4500.raw" to "8000x4500.png" with ImageMagick
convert: unexpected end-of-file '8000x4500.rgb': No such file or directory @ error/rgb.c/ReadRGBImage/247.
convert: no images defined `8000x4500.png' @ error/convert.c/ConvertImageCommand/3282.

Apparently LFOTool ducked away parts of the footer so we’re left with only three noisy files.

Original Image LFOTool
LFOTool vored the file footer :(

Conclusion⌗

This is fun. 3.5/5