HTML5

A few HTML5 questions that need answering

Mark Boas

Challenge Accepted :)

Earlier this week Christian Heilmann posted a talk and some interesting questions on his blog. Perhaps foolishly I decided to accept the challenge of answering them and what follows are my answers to his questions. My hope is to stimulate a bit of civilised debate and so I hope that people challenge what I have written. I’ve tried not to be too opinionated and qualify things with ‘I think’, ‘probably’ etc where possible, but these are inevitably my opinions (and nobody else’s) and should be interpreted as just that.

I think Christian has done a great job putting these questions together – if they make other people think as much as they made me think they’ve certainly been worthwhile. Like many of life’s great questions the answers don’t always boil down to yes and no. In fact usually there is a grey area in which a judgment call is required, and so I apologise in advance if some of the answers are simply ‘it depends’.

Can innovation be based on “people never did this correctly anyways”?

Yes, I’d say that innovation can be based on almost anything. It’s probably better if innovation is based on solid foundations, however I believe the web is more of a bazaar than a cathedral and so solid foundations aren’t always feasible.

If we look at some older browsers where let’s say the spec was interpreted in an unexpected way, take specifically IE5.5 and the CSS box model – hacks are required to get around its ‘creative’ implementation and due to the huge win of being able to write cross-browser web-pages with the same markup, hacks are almost always found. Often hacks by their very nature will be ugly, but this can help serve to remind us of their existence.* Once the hack is found innovation continues again at its usual pace. I’d argue that innovation on the web is built on a series of (often ugly) hacks.

To answer the question in the context of HTML5′s tolerance for sloppy markup, it’s interesting to note that HTML5 does allow you to markup as XHTML if you so choose, personally and maybe because I’m over fastidious, this is what I try to do. Whether the browser picks up my mistakes or just silently corrects them is another issue. Modern editors’ text highlighting helps a lot but if you are taking your markup seriously you may want to validate it. At the end of the day innovation will take place regardless.

*http://tantek.com/log/2005/11.html

Is it HTML or BML? (HyperText Markup Language or Browser Markup Language)

I think HTML should be accessible to more than what we traditionally view as the browser, yes. From an accessibility standpoint it should be a responsibility to make sure that pages are reasonably accessible from a range of devices such as screenreaders. What ‘reasonably accessible’ means could be cause for considerable debate. Additionally it is in most people’s interest to make their pages readable by a search-engine bot. Some developers may find that they want their pages to be parseable in which case they should probably mark them up in XHTML.

I think though, that the most effective way to encourage people to mark up their pages for devices other than browsers is to make clear the benefits of doing so. Search bots and screenreaders will eventually catch up and I think at that point we will be in a better place.

On a side note hypertext is what the web is founded on – it’s the essence of the web if you like. If you stripped down the web to just hypertext you’d still have a web, albeit a limited version of what we have today. Hypertext is really just text or documents endowed with the ability to link to other documents. This ability to link is of course extremely powerful and enables anyone to create a web page that is instantly ‘part’ of the web. Some companies are starting to rely less on the humble hyperlink and invent markup for themselves that relies on JavaScript to actually work, Facebook’s FBML is an example of this and I would perhaps put this ‘extension’ of HTML in the BML camp.

But while so much has been added to HTML, the hyperlink still remains and developers are encouraged to use it, even if in some cases it is just a fallback for more complex interactions. Only the other day we experienced the fallout from Google’s hashbang method of ajaxifying a link, I feel that a large part of this stemmed from the fact that developers felt that they no longer had to populate their hrefs, but in reality it is still highly recommended they do. So no matter how complex a web application becomes I’d argue that it should always be based fundamentally on the good old hyperlink in which case, strictly speaking it would be HTML.


Should HTML be there only for browsers? What about conversion Services? Search bots? Content scrapers?

Some would argue that information published publicly on the web is there to be consumed by however or whatever chooses to consume it. However a semantic web not only makes content more accessible to humans but also to algorithm running computers that seek to diseminate and categorise the wealth of information out there, this new information could then be presented back to a human and so the wheel turns and the machine cranks on. Again I feel developers should be shown the benefits.

Touching again on accessibility, laws exist and I think rightly so, to mandate that certain websites should be accessible to all, just as they do for certain buildings especially public ones. Whether your house or say a merry-go-round should be accessible is less clear. During the development of jPlayer we decided to make elements of it accessible because we felt it was the right thing to do, but the very first versions weren’t. The ‘right-thing-to-do’ can often be more a powerful motivator than any rule, especially if it is difficult to enforce.


Should we shoe-horn new technology into legacy browsers?

I think we should aim to support as many legacy browsers as reasonably possible – yes. What is ‘reasonably possible’ has always been a contentious issue but I think most developers broadly agree on what this means.

A key decision is how consistent you want each browser’s interpretation of your web page or application to be. Note that the experience will never be identical across all browsers, even the modern ones, so a certain degree of comprimise is always required. I really just view the shoe-horning of new technology into legacy browsers as just another hack, although instead of correcting a misimplemented feature its aim is to augment the legacy browser’s feature set to more closely match newer browsers.


Do patches add complexity as we need to test their performance? (there is no point in giving an old browser functionality that simply looks bad or grinds it down to a halt)

If we take patches to mean any code styling or markup that can adequately add missing functionality to a browser, (are also referred to as shims, shivs or more recently polyfills), yes patches do add complexity, but well written patches that can easily be dropped into a project should mask the complexities from the web developer. You could argue that patches are essential if we are to start using new technologies like HTML5 in the wild. Again a decision needs to be made as to whether a patch slows down a browser unacceptedly, whether to fallback to a less complete solution for legacy browsers or whether not to use the new technology at all yet.


How about moving IE fixes to the server side? Padding with DIVs with classes in PHP/Ruby/Python after checking the browser and no JS for IE?

The trouble with moving fixes to the server-side is that, in my experience at least, front-end developers and especially designers (whose lives these fixes affect) like to be able to develop applications with as little contact with the server as possible. Some web pages don’t even require any real server-side interaction and so we would be creating an added burden to the designer or developer’s development cycle. That said if the web developer could continue to work as usual on a web application that relied on the server-side anyway, safe in the knowledge that the server would take care of any differences then I suppose that would be one possible solution.

Another issue is server-side solutions tend to be less transparent and this could cause unneccessary resistance to those who might seek to change and evolve these fixes. You would also have to deal with the situation where several server-side fixes existed (PHP/Ruby/Python etc), the exception perhaps being mods to the actual webserver (although webservers do vary).

It is true that developers or designers are unlikey to work without a webserver these days but on balance I think there are a lot of issues to be resolved before all but the simplest fixes get moved to the server-side. So I’d say some IE fixes on the server-side could well make sense, anything more complex probably isn’t warranted.


Can we expect content creators to create video in many formats to support an open technology?

I don’t think we can expect content creators to create a video in many formats to support a certain technology just because it is open. It’s more likely to depend on the content creators and the market that they are appealing to – whether they are creating a true native cross-platform solution or whether they are willing to fallback to say Flash to deliver content that a browser doesn’t natively support. I think this is an area where some sort of server-side video format conversion module could be useful in encouraging developers to support native video in every browser. Ironically any tool that converts to or from certain types of encoded video could be subject to royalties (I’d love to be proved wrong here). A content creator could decide to support only open video formats but for example if H.264 is assumed not to be open, the big question is how do you create a video fallback for iOS whose browser doesn’t support Flash or similar.


Can a service like vid.ly be trusted for content creation and storage?

The question that immediately springs to mind is what happens if vid.ly goes down? You are essentially introducing another point of failure and it’s probably a good idea to keep these to a minimum, depending of course on how crucial it is that that content is always available. You could also argue that depending on CDNs to host your favourite JavaScript library is a bad idea so I guess it’s a judgment call on how likely it is that a particular service is to go down. Leaving aside the issue of reliability, what happens if a service like vid.ly gets bought out or hacked or decide to change their policy? All these are important questions that we’d probably do well to ask ourselves before relying on such a service.

Content creation could be a winner though, in the sense that it would be nice to upload one format and several other formats be made available to you. It’s a bit unfashionable but I sometimes wonder whether the WordPress model of being able to download and drop in a useful module on to your LAMP stack makes more sense. I’d love to be able to host my own personal version of vid.ly for use on my website only – it also makes sense from a distributed computing point of view.


Is HTML5 not applicable for premium content?

If we take premium content to mean content available for a fee, it is certainly more difficult to protect that content from being downloaded with HTML5. Being able to view source is arguably one of the most important aspects of open technology and HTML5 is a group of open technologies. However there is almost always a way to grab premium content whether it is based upon open technology or not, so at the end of the day we’re really talking about how difficult you make this.


Mark B

Tags:

Thursday, February 17th, 2011 AJAX, configuration, CSS, development, HTML, HTML5, javascript, SEO 1 Comment

Drumbeat Demo – HTML5 Audio Text Sync

Last month I had the pleasure of travelling to Barcelona to participate in Mozilla’s Drumbeat festival (of which more details are to come).

I very much wanted to demo the capabilities of HTML5 audio and so set about creating a demo in keeping with the theme of the festival – ‘Learning, Freedom and the Web’. I ended up with a very rough prototype of a web app that synchronised audio to text, word for word, more accurately it provided an interface that allowed a person to synchronise the audio to text and then demonstrated a couple of things that were made possible once this synchronisation had taken place.

So what was possible?

The first thing I found was that once I had the timings I could easily create a mechanism to control the audio from the text. Now by clicking on individual words I could jump to the corresponding part of the audio, useful for navigating audio and also potentially as an aid to learning language. I took this a little further by allowing the user to highlight areas of the text and having just that part of the text played back, which was um, an interesting exercise*.

*This feature is very very experimental and needs some love.

Finally I added a bit of razzmatazz, tacking on ‘Image Overlay Mode’, which is really a text overlayed on image mode, but that was a bit too wordy. To achieve image overlay mode in my limited time, I used canvas, however I’m aware that CSS3 is probably a better fit for this type of ‘animation’.

The code is very crude – it really was just flung together in a desperate rush to get it working for the Drumbeat Science Fair, so I hesitate to say feel free to take it and do with it what you will. But please do consider all demos posted on our blog open source and dual licensed under the MIT and GPL licenses.

Once again the jPlayer library came in handy, providing a useful abstraction and ensuring that the solution works on various platforms.

The demo.

Instructions :

1. In Sync Mode – press play on the player and use the space bar or sync button to synchronise the words with the audio.

2. Switch to Playback Mode to see the words synched to the audio. Click on the words to play the audio from that point. Try selecting areas of text.

3. Hit Image Overlay Mode if you are so inclined and have a canvas enabled browser.

4. Try Hack mode if you want to adjust any part of the timings and/or words.

Alternatively watch a screen capture of the demo :

mp4 | ogv

(Flash version coming soon, honest.)

Feedback and ideas on this demo are particularly appreciated. I’m also interested in possible uses (apart from karaoke ;) ) and perhaps other projects that I could collaborate with to make something genuinely useful.

Thanks to @elmook, @f1lt3r, @sroux, @aulentina and others who gave feedback and encouragement. Special thanks to @bluetezza for the original idea.

Mark B

Sunday, December 5th, 2010 Audio, HTML5, javascript, jPlayer, jQuery 26 Comments

HTML5 Media, Seeking and the Buffered Attribute

Mark Boas

It can be very exciting playing with new technologies, HTML5 media being a case in point. The spec is still evolving and although native audio and video have only been about for little over a year in any useable form, we are already seeing browsers makers pushing the envelope and developers rushing to create new libraries.

We aim to incorporate features into the jPlayer library as they become available. Recently we have been looking at browser’s ability to jump to a point in a track that has not yet downloaded. A seeking of sorts. All the major HTML5 supporting browsers allow this type of seek (with the exception of Safari for Windows), but at the moment it seems only Chrome and Safari (both mobile and desktop versions) have taken this a step further by implementing the buffered attribute, although Firefox 4 also does, it is still in beta.

The buffered attribute allows us to determine what parts of a media track have been buffered so that we can seek or skip directly to that part without the need to pause.

More info on the buffered attribute can be found in this article HTML5 video ‘buffered’ property available in Firefox 4

Mark B

Tags: , , , , , ,

Monday, October 4th, 2010 Audio, HTML5, javascript, osx Comments Off

Add a Stylish Audio Player to your Blog Without using Plugins

Mark Boas

Admittedly, it’s a little crude, but it’s simple and it works.

If you’ve ever wanted to add an audio player to your blog post and didn’t want to go to the trouble of installing a specific plugin to do so, you can always embed jPlayer within an iFrame. The one-liner you need to achieve this would probably look a lot like this:


which could give you something like this :

Sure, it’s not that efficient, you may end up loading jQuery twice if it’s already included as part of your blog, but there’s the added advantage that the page will render progressively and you have a nice little sandbox to play in should you just want to alter the player.

I used to hate iFrames and shudder at their very mention, but I must admit I’m coming around to them.

Mark B

Safari Requires QuickTime for HTML5 Media Support

Over the weekend I upgraded to Safari 5.0.1, around about the same time my colleague Mark P informed me of a potential HTML5 audio issue with this latest incarnation of Apple’s browser.

It wasn’t until Sunday afternoon that I finally started to investigate the issue. The recently upgraded version of Safari on my Mac worked fine, in the sense that it passed all the expected tests we’d set up for jPlayer compatibility. So I decided to investigate further and installed on my little used Windows XP partition. I ran the test and sure enough it failed. Next I installed on a copy of XP running under Parallels Desktop. Same problem. Mark P on the other hand had it working under XP but not under Windows 7. Asking around on Twitter others had no such issues on Windows 7. Most strange.

I started to suspect it was an installation issue, but had no idea of the root cause until someone on the jPlayer group thread we’d started mentioned that they’d had similar issues with Safari 4 and that QuickTime had been the culprit. It turns out Safari relies on QuickTime to play HTML5 based media. Needless to say that this came as a bit of a surprise to us.

So Safari does not then support HTML5 audio or video without QuickTime being installed, just when we were getting used to browsers being ‘standalone’. Next to check if Internet Explorer 9 requires Silverlight to support canvas. Imagine the fuss if it did.

(Thanks to @trygve_lie and @getify for their help with this.)

Mark B

Tuesday, August 17th, 2010 Audio, HTML5 6 Comments