Hyperaudio - A Future of Audio
Hyperaudio
The A Future of Audio.
hy·per·audio
- 1. Audio integrated into the web experience.
- 2. Non-passive audio.
- 3. Anything you want it to be.
What makes audio so special?
- ☼ Requires only partial attention
- ☼ Conveys emotion well
(in evolutionary terms hearing developed before vision.)
Demo 1 - Martin Luther King
What do we need to make audio interactive?
- 1. Set the audio playhead position
- 2. Get the audio playhead position
(covers many cases)
Word time-aligned transcripts. Hypertranscripts?
<p class="transcript">
<span t="123">Buongiorno</span>
<span t="456">mondo!</span>
</p>
Word time-aligned transcripts. Hypertranscripts?
var words =
document.getElementsByClassName('transcript')[0]
.getElementsByTagName('span');
for (var i = 0, iLen = words.length; i < iLen; i++) {
words[i].addEventListener('click', function(){
audio.setCurrentTime(this.getAttribute('t'));
audio.play();
}, false);
}
Demo 2 - Hyperdisken
Make audio content searchable
Transcribing your audio makes it much more findable. It also makes excerpts findable and playable.
Social Integration
Now we have a good grip on our content's word timings, we can allow people to link to excerpts of audio as well as their associated text.
Language Switching
Precise timings mean we can switch languages on-the-fly.
Use Case : language learning applications.
Demo 3 - RadioLab
Going further with text
- 1. Highlight words as they are spoken.
- 2. Use colour and size of text.
- 3. Provide a mobile device experience.
"My partner is deaf, and I've really wanted her to experience Radiolab. Now she really can!
The use of colored text to indicate different speakers, the timing, the flow - it really captured the conversational experience that is Radiolab".
Technology (Lego)
- 1. Popcorn.js ↬
- 2. jPlayer ↬
- 3. vinegar, brown paper
Popcorn.js
- ✔ Light
- ✔ Modular
- ✔ Time based events built in
- ✔ Active community
- ✔ Library independent
- ✔ IE8+ compatible to an extent
jPlayer
- ✔ Light
- ✔ Modular
- ✔ Skinnable
- ✔ Active community
- ✔ Flash fallback built in
- ✔ IE6+ compatible
Popcorn.js + jPlayer
The key advantage of combining Popcorn and jPlayer is that you can support one video codec on all modern browsers.
jPlayer also provides a nice API wrapper that masks browser differences.
Going further with text
- 1. Highlight words as they are spoken.
- 2. Use colour and size of text.
- 3. Provide a mobile device experience.
Demo 4 - Radio24Syv
The elephant in the room
How do we create word-aligned timed transcripts?
- 1. Use third-party services
- 2. By hand
Third party transcription services (paid)
- $ 3PlayMedia ↬
- $ DotSub ↬
- $ Dragon Speech Engine ↬
- $ PlyMedia ↬
- $ Ramp ↬
- $ SpeakerText ↬
- $ VoiceBase (freemium) ↬
Third party transcription services (free)
- ☮ Amara (formerly Universal Subtitles) ↬
- ☮ CMU Sphinx ↬
- ☮ Shout Toolkit ↬
- ☮ Something that hasn't been made yet!
When we synchronise audio with text we can manipulate audio in the same way we can text.
Demo 5 - Hyperaudio Pad
Hyperaudio Pad
- 1. Create audio/video programmes easily.
- 2. Web based intuitive interface.
- 3. Each programme comes with a hypertranscript.
- 4. Remix the remixes.
- 5. Programmes come with source intact.
Nothing left on cutting room floor.
Hyperaudio Pad Applications
- ☼ Citizen journalism.
- ☼ Mainstream journalism when time is an issue.
- ☼ Prototype (first cut).
- ☼ Casual mash-ups. Art?
... and now for something completely different.
We've talked a lot about speech-to-text.
What about text-to-speech?
Demo 6 - Perceptive Media
Dynamically generated audio
We are starting to see the ability to dynamically generate audio content.
- ☼ Libs like Speak.js can generate audio on the fly.
- ☼ Advanced audio APIs allow 'real-time' effects.
- ☼ Standards for future audio are being forged!
Demo 7 - Internet Indians!
As most video has an audio track, we can apply the same techniques we use to control audio, to video.