How does a Soundwave Tattoo work?

When I was at HolyJS a few weeks ago some of the speakers were sharing this video of an app that scans waveform tattoos and plays back the audio:

It's creative technology at its best — indistinguishable from magic on some level. Immediately discussion turned to figuring out how this magical demo worked.

I'd just given a presentation on MIDI and JavaScript, which has an obvious sound/music component, and so someone asked me how I thought it worked. They pointed out that a waveform just showed time and amplitude, so how does it figure out pitch and sound?

I had no idea — my understanding was the same as hers. My assumption was that something else must be going on with that demo, but I didn't have any hunches and was more open to the idea that my understanding of the information present in a waveform must be wrong somehow. I remember making a comparison to map projections saying that a waveform isn't the complete picture of sound, but rather a representation of sound shown in a way that gives us useful information — duration and amplitudes.

This explanation was: 1) something I kind of pulled out of my ass, though I feel like it's still a decent analogy 2) didn't actually address the question as to how this tattoo scanning app worked. Ultimately I admitted I had no idea.

Humorous side note: this is the risky part of being presented as an “expert” on anything. Suddenly people will start asking you things you don't have answers to!

I'd been meaning to look into this since I got back and did a bit of Googling on yesterday. There's an explanation of the app on Incredible Things that has a quote from Skin Motion themselves. In that quote I found a few clues:

“A person uploads or records the audio they want into the app or website...”

Aha! That already tells me 90% of it. The audio isn't being read off the skin — it's probably being used like a fancy QR code to find a match in their database where they can load the audio into the app and play it.

Reading through the rest of the explanation it ends with:

”When the user opens our app and points the camera on their mobile device at the tattoo, it recognizes the shape of the Soundwave and plays back the audio.”

This makes a lot more sense, reaffirming my understanding of what a waveform actually represents (whew!) and my hunch that something else must be going on when I saw that demo.

The longs story short: Soundwave tattoos stores an image of the waveform and looks for a match when scanning your tattoo. Kind of like a fancy QR code. Obviously I can't 100% confirm this, but based on that quote and my understanding of waveforms I don't see else it could be.

Although there's a tiny part of me that's a little sad to discover it's not actually rendering audio from a tattoo representation of a waveform, I'm actually more impressed with what a clever, novel approach they've taken! There's a little bit of smoke and mirrors, but that doesn't mean it isn't a great magic trick :)

Right now you have to join a waiting list to get your Soundwave Tattoo. The app, I'm assuming, can be installed and removed from your phone with relative ease. The tattoo likely requires a bit more commitment.