This is Jessica. Speaking of data sonification (or sensification), Hyeok, Yea Seul Kim, and I write: 

Data sonification-mapping data variables to auditory variables, such as pitch or volume-is used for data accessibility, scientific exploration, and data-driven art (e.g., museum exhibitions) among others. While a substantial amount of research has been made on effective and intuitive sonification design, software support is not commensurate, limiting researchers from fully exploring its capabilities. We contribute Erie, a declarative grammar for data sonification, that enables abstractly expressing auditory mappings. Erie supports specifying extensible tone designs (e.g., periodic wave, sampling, frequency/amplitude modulation synthesizers), various encoding channels, auditory legends, and composition options like sequencing and overlaying. Using standard Web Audio and Web Speech APIs, we provide an Erie compiler for web environments. We demonstrate the expressiveness and feasibility of Erie by replicating research prototypes presented by prior work and provide a sonification design gallery. We discuss future steps to extend Erie toward other audio computing environments and support interactive data sonification.

Have you ever wanted to listen to your model fit? I haven’t, but I think it’s worth exploring how one would do so effectively, either for purposes of making data representations accessible to blind and visual impaired users, or for other purposes like data journalism or creating “immersive” experiences of data like you might find in museums.

But turns out it’s really hard to create data sonifications with existing tools! You have to learn low-level audio programming and use multiple tools to do things like combine several sonifications into a single design. Other tools only offer the ability to make sonifications corresponding to a narrow range of chart types, perhaps as a result of a bias toward thinking about sonifications only from the perspective of how they map to existing visualizations.

Hyeok noticed some of these issues and decided to do something about it. Erie provides a flexible specification format where you can define a sonification design in terms of tone (the overall quality of a sound) and encodings (mappings from data variables to auditory features). You can compose more complex sonifications by repeating, sequencing, and overlaying sonifications, and it interfaces with standard web audio APIs. 

Documentation on how to install and use Erie is here. There’s also an online editor you can use to try out the grammar. But first I recommend playing some of the examples, which include some simple charts and recreations of data journalism examples. My favorites are the residuals from a poorly fit model and a better fitting one. Especially if you play just the data series of these back to back, the better fit should sound more consistent and slightly more harmonious.

This was really Hyeok’s vision; I can’t claim to have contributed very much to this work. But it was interesting to watch it come together. During our meetings about the project, it was initially very unfamiliar to me, trying to interpret audio variables like pitch as carrying information about data values, and I can’t really say it’s gotten easier. I guess this gets at how hard it is to make data easily consumable in a serial format like audio, at least for users who are accustomed to all the benefits of parallel visual processing.