Add new tag
Chris (a Technical Evangelist for Mozilla), took us on a useful whistle-stop tour of HTML5 and friends, technologies that I certainly wish to take advantage of in my final project. I feel that there’s so much potential yet to be tapped, it’s difficult to know where to start, but I’ll definitely be using the new audio and video APIs and looking into offline storage. Chris also touched on accessibility, something that I want to bake into any tool I come up with. To me accessibility is interesting, not just from the perspective of users with disabilities but also because I hope to do is make news (and the tools to create news) more accessible to all people.
- Providing forums and methods of communication and feedback with the initial release
- Building extensibility into the tool so people can add functionality via plugins
- Encouragement of collaboration – make it easy for people to get involved
- Making sure that the documentation is comprehensive and clear (and can be contributed to)
- Fanatical support and encouragement
I also attended the ‘extra’ webinar given by Jesse James. In some ways this was the most useful to me as it focussed on designing the user experience or UX, something I’m keen to learn more about and who better to tell me than the author of The Elements of User Experience. It was fascinating to take a peak at how UX experts like Jesse approached the subject of design. I sat there transfixed as he broke down the process and discussed ideas such as psychology, emotion and how the user feels and their relationship with a given product or tool. Jesse spoke about the personality of a product and how users relate to tools as they would another person, using similar parts of the brain as they do when relating to people.
I really want the tool I create to be as simple as possible to use, so my question to Jesse was :
“Is it possible to make a product so simple that it doesn’t need a personality, or needs less? Or is the simplicity its personality?”
The answer : “Simplicity is a personality.”
Great. Hopefully I’m on the right track then.
Following the fun we had making the Hyperdisken demo, I was happy to be asked by Mozilla, in collaboration with Radiolab and SoundCloud to help create another demo to show off the possibilities of hyper audio. This time we had an excellent Radiolab program as audio material and we wanted to get a little more ‘involved’. What was required was an application that would consist of many of the features of the Hyperdisken demo but also integrate deeply with SoundCloud API, and on top of this something extra, something to catch the eye.
I was fortunate again to work with the ideas-forge known as Henrik Moltke, who collaborated early on with Paul Rouget to produce something he dubbed the ‘Word River’ – a CSS3 manipulated flowing river of words that dynamically picked up content from an HTML transcript. We were also keen to make a pure HTML5 based solution and Paul helped figure out the hooks into the SoundCloud API that would allow us to achieve that. We were also very lucky to be given a great design by the multi-talented Lee Martin, SoundCloud’s experimenter extraordinaire.
So with proof of concept and some visual bling firmly in hand I was tasked with making this baby fly. Luckily I had help. SoundCloud engineers were at hand to answer any questions and crucially we had great support and code contributions from the popcorn.js group. I also managed to talk jPlayer author and all round JS media guru and of course, colleague Mark Panaghiston into giving me a hand. So despite the tight deadlines we were pretty much set.
Henrik has already blogged about the ideas and functionality that make up the demo. I want to write a little about the technology used.
Although I found out in retrospect, not strictly essential, we once again used jPlayer as our audio base, we’re familiar with it and we can move fast using it. It also meant that we could take much of the functionality developed in previous demos and plug it right in. Again, the excellent Popcorn.js was the engine that drove all the time based display of text and images and dealt with the parsing of data. Steven Weerdenberg, active Popcornista from Seneca College, very kindly wrote a plugin that grabbed, parsed and presented comments (amongst other things) from the track we used hosted on SoundCloud. This is where the Popcorn framework comes into its own as a plugin oriented architecture, something we took advantage of when we converted both the transcript and the word river functionality into plug-ins.
So where did the data come from? Well again the transcript HTML doubled as the source for richer interaction when used by the word river plugin. I like this approach I have to say. It means you don’t have to be a programmer to come in and immediately understand the content and change it accordingly. I also like the fact that the transcript is a separate HTML file, it pleases the separatist in me and means that it works as a standalone resource. We also used the standard speaker notations as a type of meta-data, the word river plugin hiding these parts for the purpose of display but using them to colour code each speaker’s text.
This part of the transcript:
is used to create this ‘word river’:
and this interactive transcript :
Data-wise, everything else came via the SoundCloud API, this included their trademark wave-form, both ogg and mp3 audio sources and all of the comments. We also hijacked the comments to make a crude content management system. The idea being that any comments posted by the Radiolab account with references to images in them, were picked up and displayed as images in the main content area, and did not show up as comments on the timeline. If two images were present it meant they were square, one and it was ‘widescreen’ a blank image was used to remove images when they were no longer needed.
The last pieces of the puzzle and one we’ve still some polish to apply to, (if polishing puzzles makes any sense to you) was getting it all working on the majority of tablets and mobile devices. Since this demo didn’t use Flash this was actually a possibility and we got our web designer Silvia Benvenuti to come out of maternity leave and sort this out for us at the 9th hour, leaving me quite literally holding the baby.
This was a tough gig but all in all I’m happy with what we achieved, everyone seemed to really enjoy taking part in the process, and I certainly enjoyed bringing it all together. Hopefully it will inspire both program makers, designers and developers to come together and explore the limits of what hyper-audio can do. As Inspiral Carpets would say, moo!
Source code for this project and other demos can be found on github
Follow me on Twitter if you want to hear more about this sort of thing.
- The Hyperaudio Pad – Next Steps and Media Literacy
- Breaking Out – The Making Of
- Breaking Out – Web Audio and Perceptive Media
- Playing web audio offline on mobile Safari. Mission impossible?
- Flash vs HTML5 Video and the Codec thing
- Altrepreneurial vs Entrepreneurial and Why I am going to Work with Al Jazeera
- HTML5 Audio APIs – How Low can we Go?
- Hyperaudio at the Mozilla Festival
- The Hyperaudio Pad – a Software Product Proposal
- Introducing the Hyperaudio Pad (working title)
- Accessibility, Community and Simplicity
- Build First, Ask Questions Later
- Further Experimentation with Hyper Audio
- Hyper Audio – A New Way to Interact
- P2P Web Apps – Brace yourselves, everything is about to change
Add new tag