jPlayer
Altrepreneurial vs Entrepreneurial and Why I’m going to Work with Al Jazeera
By Mark Boas
A few weeks ago I was asked to write a bio - explain a bit about myself in a few lines. I hate writing bios, all that faux third-personage - I never really know what to write.
The reason a bio was required was that I’d been offered a position as a Knight-Mozilla sponsored fellow with Al Jazeera - the happy end-result of a Knight-Mozilla news challenge that I have been actively enjoying over the last few months. It was agreed that I would work slightly less than full-time hours and on a largely remote basis and so I was glad to accept the fellowship knowing I could still dedicate time to my family and other interesting projects (not that I would classify my family as a project you understand). Working with Al Jazeera is of course an experience not to be missed. I couldn’t have imagined that my life would take such a turn a year ago or so, but I’m very glad that it did and I put it all down to my (broadly directionless) approach to working life.
I consider myself very lucky that because of my sedentary lifestyle I need very little money to get by. This has allowed me to follow my interests while working for our small ‘company’ Happyworm for the last 10 years. I consider this a very real success. We created a popular open source project and foster a fairly large community — from this many opportunities have arisen. Recently for example, I was asked by the W3C to run an online audio/video course and this has been a fantastic experience and now I have the opportunity to find out the world of journalism works and hopefully contribute. All this would probably not have occurred if I hadn’t simply thrown caution to the wind and followed by interests.
There are two of us now working for Happyworm, we used to be three but our web designer (also my partner) decided that at least one of us should have a steady income and generously offered to take up full-time work at a local council. So it’s just the two of us working with 3rd party designers when we need them. I’m based just outside Florence and Mark P’s in the heart of Edinburgh, although we are very different, we have similar requirements and a similar history - Mark P left a well paid job as a CMOS camera chip designer to come and work on web stuff.
This finally brings me to the title of this post, when writing my bio I foolishly used a little known, possibly non existent word, by describing Happyworm as tiny altrepreneurial web agency.
I stumbled on the word altrepreneur some years ago here : http://www.mylifecoach-online.com/MLC%20newsletter%20May%202005.htm#authentic
“The Altrepreneur, like their colleague the Entrepreneur, runs one of the 3 Million Micro Businesses in operation in the UK today. However unlike the Entrepreneur, with a financial and career focus, the Altrepreneur is doing it for entirely different reasons. It seems that 70% of those small businesses are being run because the owner/operator is focused on achieving a change in their life-style through running a small business, they are looking to increase their overall quality of life by putting in some up front hard graft. This goes hand in hand with the growing movement around Authenticity (…). The idea that the source of much tension in our lives is the conflict between our true selves and the roles that we play. Getting in touch with your true self and letting go of that tension will lead to a very different kind of life.”
This article chimed very strongly with me - my objectives and the decisions I made to leave a well paid job, set up Happyworm, move to another country, be my own boss and follow my own interests where possible. We now have two children, who I am fortunate enough to see a lot of. Recently I decided to look after the 9 month old Anna in the mornings for 4 days of the working week and then work from 14:30 until midnight with a 3 hour or so (I don’t time it) break for family dinner and games. I grow vegetables, cook at least once a day and am involved in the local community centre - finally I feel like I am achieving that mythical work-life balance.
So to me the word altrepreneurial seemed a perfect concise way of explaining what I did and how I see Happyworm. Incidentally Happyworm turned 10 years old in October. You might think 10 years is pretty good going for a small company but the truth is we would never have lasted so long if we were in it for the money, we’ve had good spells but also our fair share of dry spells where we worked on open source, brewed our own beer and patched our own clothes (or at least I did). Turns out following our interests and making jPlayer has been much more of a success than we could have imagined, I think we’re approaching half a million downloads and perhaps the best measure - a community of around two and a half thousand.
Despite the money, like most people I’m not really happy working long hours on projects I’m not interested in (although through necessity I’ve done my fair share), I don’t think actually I’m any good at something my heart isn’t in. Maybe I’ve been spoiled, but the most important thing for me is to enjoy my work and so life, the money is always a secondary consideration and that’s why, contrary to what you may see written in my bio, I work for a small altreprenurial web agency - not entrepreneurial. Damn you auto-correct!
This notebook has been lying on my desk for the best part of a decade.
Some perspectives on the Knight-Mozilla News Technology Fellowship :
My Life as a Startup - The Road not Taken by The Guardian fellow - Nicola Hughes
Journalism in the Open: the 2011/12 Knight-Mozilla Fellows by Project Header Upper - Dan Sinker
Knight-Mozilla names news technology fellowship winners Journalism.co.uk
Hyperaudio at the Mozilla Festival
This year has run contrary to previous years of my life in the sense that it seems to have lasted much longer than the year before. This may have something to do with a small addition to the family that arrived in March and the consequent ramping up of waking hours, but I prefer to think it’s because things are moving so quickly in that wonderful place we call the worldwide web, that it seems that too much has happened to compress into a year.
It was only a year ago for example, when I started tinkering with jPlayer to couple text and audio and was asked to show off a very small demo at the first Mozilla Festival in Barcelona last year.
A few months later I was introduced to and became involved with something known as Hyperaudio and was given the opportunity to create a couple of proof-of-concept demos : Denmark Radio’s Hyperdisken Demo and the Radiolab/Soundcloud collaboration ( a shout-out to Henrik Moltke for doing much of the groundwork for these demos and my colleagues at Happyworm without which they just wouldn’t have been possible).
Further on through the arc of this epic year, I took part in the Mozilla Knight Journalism challenge where I was encouraged to research and blog some ideas on a tool I called the Hyperaudio Pad. Happily I was flown over to Berlin with other like minded MoJo peeps, for a week of intense discussion, collaboration and hacking and managed to get the first semblance of an actual product out of the door.
Which brings me in a round-about way to this year’s Mozilla Festival, where I will again be demoing and chatting about the coupling of text and media and the other things that Hyperaudio is (or could be) about. This all at the Science Fair on Friday, 4th of November and a Hyperaudio Workshop / Design Challenge on Saturday, 5th.
The Science Fair will be all about demos, chatting to people interested in the general Hyperaudio concept and making sure I don’t spill my drink over my laptop. Meanwhile in the workshop we hope to put the Hyperaudio Pad through it’s paces and actually make something with it, while gathering ideas and maybe hacking on some of them into the bargain. It should be a fun mix of hacks, hackers, designers, audio buffs and basically anybody curious enough to be involved. The session will be fairly free flow but the plan so far is to:
- Briefly chat about and demo the Hyperaudio concept, what it can do and with whom we can collaborate.
- Breakout into groups to come up with new ideas, applications and designs.
- Reconvene and talk about the group’s ideas.
- Breakout into groups to work on these ideas, whether it is creating a programme, UI mockups, story-boarding functionality or even developing small demos.
- Creating something we can show other people - hopefully a program created with the Hyperaudio Pad but also some new ideas.
Hopefully we can cram this all into 3 hours. Folk from the BBC, Universal Subtitles, Sourcefabric and other cool people are planning to be in attendance, which should make for some interesting crossovers and fantastic opportunities to consult and maybe even influence organisations in a position to use the stuff we’re making. Additionally the BBC are providing us with some fantastic media to work with.
All will be revealed over the weekend, book your place and get your eyes and ears ready to be part of the Hyperaudio experience.
The Hyperaudio Pad - a Software Product Proposal
Imagine
Imagine that making your own video or audio news programs was really easy.
Imagine if you could pull together news stories from various sources, combine them and publish the results in minutes.
Imagine that the story you made came complete with a transcript, was fully accessible and could be picked up by search engines.
Imagine that you could easily embed your story into a web page and that others could incorporate your story into new stories.
Imagine that viewers of you story could share any part of it in seconds.
Imagine all this and you are imagining the functionality of the Hyperaudio Pad.
Listen to a use case.
Watch the screencast.
Hypertranscripts
Currently, the process of putting together a media program, the process will probably involve some fairly complex audio/video editing software. The result - a fairly ’static’ representation. The whole process is time consuming, wasteful and inflexible.
By changing the way we think about media, we can create a new way of manipulating it. The Hyperaudio Pad allows quick manipulation of media and takes advantage of a form of media representation referred to as a hypertranscript.
When we tell a story we invariably use the spoken word, the spoken word can be transcribed, and once transcribed can be used as an accurate reference to the media it is associated with.
Hypertranscripts are part of a broader concept dubbed Hyperaudio by Henrik Moltke, they are transcripts hyper-linked to the media they represent, a type of fine-grained subtitle, they are separate entities from the media they describe, existing as a word-level aligned references expressed in HTML.
Tools and services exist to create Hypertranscripts, word level timing can even be approximated from subtitles. In the past I have used libraries such as Popcorn.js, jPlayer and data from the Universal Subtitles project.
The good news is that Hypertranscripts already exist, here are a few examples:
Screencast [youtube] of the Danish Radio Demo
Screencast [youtube] of the Radiolab Demo.
Quick and simple demo I put together.
Demos by others include: Minnesota Public Radio Demo, Voxallead News and Liris Interactive Transcript
Using Hypertranscripts we can:
- Navigate through the media via the text.
- Select and play parts of the media by highlighting the relevant parts of the text.
- Share parts of the media creating URLs that contain start and end points.
- Make our media more discoverable (via search engines)
- Make our media more accessible.
- More easily visualize the media.
- Combine, embed and easily integrate media (using simple tools)
The Hyperaudio Pad
The Hyperaudio Pad is a tool to allow users to easily assemble spoken word based audio or video from Hypertranscripts.
It will allow users to create their media by simply copying and pasting text from existing transcripts into a new transcript. These Hypertranscripts link to the actual media that they transcribe, so when parts of the transcripts are copied the references to their associated media are also copied.
Note that the media itself is not copied, just references to parts of it, the result is mash-up of video or audio and an associated representative transcript.
The tool will be :
- Web based, cross-browser, cross-platform (work on mobile devices) and open source.
- Simply presented, taking cues from minimalist distraction-free tools such as iAWriter.
- Intuitive and extremely easy to use, taking advantage of the text editing paradigm.
- Extensible - written in such a way that it can easily be built upon.
The interface should be clear and minimal, the user can search for and choose from existing hypertranscripts, open them, play them and copy parts of them to their ‘document’ all from within the same page.
Edit Mode
Sketch of the Hyperaudio Pad in Edit Mode
Playback Mode
Sketch of the Hyperaudio Pad in Playback Mode
[video to come]
Who is this tool for?
- Newsroom journalists who need to very quickly assemble media from different existing sources and perhaps adapt the result as news unfurls.
- Podcasters, citizen journalists and mediabloggers who want a free and easy way of putting together media programs or newscasts.
- Anyone who wants their resulting media programs to include a transcript for increased accessibility and visibility.
- Anybody who is happy for their results to be taken apart, re-combined or referenced by others.
How does it work ?
Hypertranscripts are essentially defined in HTML. The simplest possible example for the use by a tool such as the Hyperaudio Pad could be:
WORD-FROM-THE-TRANSCRIPT
Example:
burgeoning
Note, we can add to this format should we wish to include meta and other data, for example :
WORD-FROM-THE-TRANSCRIPT
(from Julien Doran’s MetaFragment proposal document)
Transcripts will contain a series of these marked up words each individual word containing enough data to describe it’s associated media and at which time in the media it occurs.
Note that copying the Hypertranscript into any text editor will result in valid HTML and so can be used in other tools and applications external to the Hyperaudio Pad.
It is likely that the user’s resulting transcript will reference several different pieces of media and so it may beneficial for the application to ‘load’ and possibly cache the various media sources in advance for smoothest possible playback. We can achieve this by parsing the transcript after every save.
Future developments could include a simple scripting language that users can insert between transcripts to make transitions smoother and even the possibility to add background music or sound effects for example:
[fade out over 5 seconds]
or
[background fade in 'threatening music' over 3 seconds]
and then at the bottom
[reference 'threatening music' http://soundcloud.com/some.mp3]
You may say I’m a Dreamer
In order to start using the Hyperaudio Pad to its full potential we first need our media to be transcribed and to do this we need to make tools that make this easier and if possible free. However I believe that hypertranscripts deliver so many benefits that the incentive to transcribe media is high.
The Hyperaudio Pad is just one of many tools that could be built on the underlying Hypertranscript platform. To build this platform we should collaborate with others to :
- establish a standard markup for transcripts
- make the platform easy to build upon and enhance as new technologies become viable
- create a community with a pioneering spirit
- develop an ecosystem to facilitate the creation of a toolkit
During the course of this project it’s been a pleasure to find many other participants interested in the general theme of describing/transcribing media and utilizing the result. Over the last couple of weeks I have been collaborating with Julien Dorra, Samuel Huron, Nicholas Doiron and Shaminder Dulai. The excitement has been palpable and although only communicating virtually it felt like we were sparking off each other.
So I guess at least as far as this dream is concerned, I’m not the only one. 
Further Experimentation with Hyper Audio
Following the fun we had making the Hyperdisken demo, I was happy to be asked by Mozilla, in collaboration with Radiolab and SoundCloud to help create another demo to show off the possibilities of hyper audio. This time we had an excellent Radiolab program as audio material and we wanted to get a little more ‘involved’. What was required was an application that would consist of many of the features of the Hyperdisken demo but also integrate deeply with SoundCloud API, and on top of this something extra, something to catch the eye.
I was fortunate again to work with the ideas-forge known as Henrik Moltke, who collaborated early on with Paul Rouget to produce something he dubbed the ‘Word River’ - a CSS3 manipulated flowing river of words that dynamically picked up content from an HTML transcript. We were also keen to make a pure HTML5 based solution and Paul helped figure out the hooks into the SoundCloud API that would allow us to achieve that. We were also very lucky to be given a great design by the multi-talented Lee Martin, SoundCloud’s experimenter extraordinaire.
So with proof of concept and some visual bling firmly in hand I was tasked with making this baby fly. Luckily I had help. SoundCloud engineers were at hand to answer any questions and crucially we had great support and code contributions from the popcorn.js group. I also managed to talk jPlayer author and all round JS media guru and of course, colleague Mark Panaghiston into giving me a hand. So despite the tight deadlines we were pretty much set.
Henrik has already blogged about the ideas and functionality that make up the demo. I want to write a little about the technology used.
Although I found out in retrospect, not strictly essential, we once again used jPlayer as our audio base, we’re familiar with it and we can move fast using it. It also meant that we could take much of the functionality developed in previous demos and plug it right in. Again, the excellent Popcorn.js was the engine that drove all the time based display of text and images and dealt with the parsing of data. Steven Weerdenberg, active Popcornista from Seneca College, very kindly wrote a plugin that grabbed, parsed and presented comments (amongst other things) from the track we used hosted on SoundCloud. This is where the Popcorn framework comes into its own as a plugin oriented architecture, something we took advantage of when we converted both the transcript and the word river functionality into plug-ins.
So where did the data come from? Well again the transcript HTML doubled as the source for richer interaction when used by the word river plugin. I like this approach I have to say. It means you don’t have to be a programmer to come in and immediately understand the content and change it accordingly. I also like the fact that the transcript is a separate HTML file, it pleases the separatist in me and means that it works as a standalone resource. We also used the standard speaker notations as a type of meta-data, the word river plugin hiding these parts for the purpose of display but using them to colour code each speaker’s text.
This part of the transcript:
is used to create this ‘word river’:
and this interactive transcript :
Data-wise, everything else came via the SoundCloud API, this included their trademark wave-form, both ogg and mp3 audio sources and all of the comments. We also hijacked the comments to make a crude content management system. The idea being that any comments posted by the Radiolab account with references to images in them, were picked up and displayed as images in the main content area, and did not show up as comments on the timeline. If two images were present it meant they were square, one and it was ‘widescreen’ a blank image was used to remove images when they were no longer needed.
The last pieces of the puzzle and one we’ve still some polish to apply to, (if polishing puzzles makes any sense to you) was getting it all working on the majority of tablets and mobile devices. Since this demo didn’t use Flash this was actually a possibility and we got our web designer Silvia Benvenuti to come out of maternity leave and sort this out for us at the 9th hour, leaving me quite literally holding the baby.
This was a tough gig but all in all I’m happy with what we achieved, everyone seemed to really enjoy taking part in the process, and I certainly enjoyed bringing it all together. Hopefully it will inspire both program makers, designers and developers to come together and explore the limits of what hyper-audio can do. As Inspiral Carpets would say, moo!
Source code for this project and other demos can be found on github
Follow me on Twitter if you want to hear more about this sort of thing.
Hyper Audio - A New Way to Interact
Recently I had the privilege of working on a very interesting project with a few folk from Mozilla - it’s the type of project I love to work on, as it involves web audio and its deep integration into the general web experience.
Web audio is no longer consigned to being the passive play and pause experience of yesteryear, it has the potential to be much more, it can be a driver of much richer interactions, something Henrik Moltke explores with something he dubs Hyper Audio. The remit of the project was to take various media elements of a radio interview broadcast by Danish Radio station DR; audio, subtitles, transcripts, footnotes etc and link these in an intuitive and useful manner.
To say this project was right up my street would be an understatement - this project was in my flat, raiding my fridge and drinking my beerz. I was already fascinated by the concept. I’d been playing about, creating audio related demos for a couple of years and in November last year I decided to attend the Mozilla Drumbeat festival and created a demo for the event. The demo was accepted to be exhibited at the science fair on the opening evening and garnered some interesting feedback both on and offline, what it effectively demonstrated was the synchronization and bi-directional control of text and audio.
When Henrik asked me to work on this project, I naturally jumped at the opportunity. Due to time differences, pressing deadlines and the luxury of having a nice quiet office, I stayed up late most nights for a week, happily hacking away and helped out and supported by various Mozillians and the popcorn.js community.
So that’s the back-story, here’s the demo.
Some things to try :
- Switch the audio from English to Danish - it should continue from the same point in Danish, subtitles and the transcript should also change appropriately.
- Try clicking on words in the transcript - the audio should start playing from the corresponding point.
- Highlight a passage of transcript text - this should add a tweetable excerpt to the ’share’ box. The URL included should just play that part of the audio.
- Clicking the music note icons in the ‘media’ box should take you to the point of the audio where that resource was mentioned.
How did we achieve this? We used popcorn.js to display subtitles, footnotes and other time-related resources. In fact a lot of this was already in place when I picked up the project. I then integrated jPlayer for the audio playback and deeper interaction. Popcorn allows us to associate timings with actions and have these actions triggered by media when they hit said timings. So pretty much perfect for our needs. jPlayer provided a solid abstraction above the native audio API, it allowed me to easily synchronize and switch audio tracks and jump to specific points or sections in the audio, with very few lines of code. Importantly it also protected us from any cross-browser issues and allowed our designers to effortlessly create a custom skin for the player.
So this was the control, but what about the media? Well this part was a massive team effort. Henrik managed to provide a very accurately timed transcript. We had hoped to use the subtitles in SRT format but for convenience we parsed them or rather Scott Downe parsed them into JSON format.
One of the bigger issues we encountered was that we only had the transcript in English and the timings for the Danish transcript were naturally different. Luckily we had accurately timed Danish subtitles and legendary Bobby Richter on hand to convert the subtitles to individual words complete with their timings, which he did by cunningly interpolating the timing of words (based on word length) and based on their in-subtitle position. All knocked out in about 10 minutes and in 20 lines of code. It worked surprisingly well, of course you need to be able to understand Danish to truly tell. We could have probably parsed the subtitles into the transcript on the fly but due to time limitations we made them static.
Perhaps an aside not directly related to audio, I managed to hack together some code that allowed highlighted transcript text to be placed in the ’share’ box, and grab the timings of the first and last words, from there it was pretty much straightforward to make this excerpt tweetable.
This whole endeavor was very much a group effort, a huge thanks to the popcorn.js team, who made joining their IRC feel like walking into a pub full of friends.
Special credit and thanks then should go to Scott Downe, Bobby Richter, Barry Threw, David Humphrey, Brett Gaylor, Ben Moskowitz, Christian Valentiner, Silvia Benvenuti and of course Henrik ‘Tank’ Moltke whose baby all this was. It was great being part of such a talented team. Awesomesauce indeed.
Mark B
Previous Posts
- Altrepreneurial vs Entrepreneurial and Why I’m going to Work with Al Jazeera
- HTML5 Audio APIs - How Low can we Go?
- Hyperaudio at the Mozilla Festival
- The Hyperaudio Pad - a Software Product Proposal
- Introducing the Hyperaudio Pad (working title)
- Accessibility, Community and Simplicity
- Build First, Ask Questions Later
- Further Experimentation with Hyper Audio
- Hyper Audio - A New Way to Interact
- P2P Web Apps - Brace yourselves, everything is about to change
- A few HTML5 questions that need answering
- Drumbeat Demo - HTML5 Audio Text Sync
- HTML5 Media, Seeking and the Buffered Attribute
- Add a Stylish Audio Player to your Blog Without using Plugins
- The Future of Web Apps - Single Page Applications
Tag Cloud
-
Add new tag
AJAX
apache
Audio
band
buffered
Canvas
CDN
chrome
community
custom tags
firefox
gig
HTC
HTML5
Hyper Audio
internet explorer
java
javascript
journalism
jPlayer
jQuery
jscript
LABjs
leopard
media
Mozilla
MVP
opera
opera mini
osx
P2P
Popcorn.js
poster
prototyping
rewrite
safari
Scaling
simplicity
SoundCloud
timers
tomcat
video
Web Apps
web design