P2P Web Apps – Brace yourselves, everything is about to change
The old client-server web model is out of date, it’s on its last legs, it will eventually die. We need to move on, we need to start thinking differently if we are going to create web applications that are robust, independent and fast enough for tomorrow’s generation. We need to decentralise and move to a new distributed architecture. At least that’s the idea that I am exploring in this post.
It seems to me that a distributed architecture is desirable for many reasons. As things stand, many web apps rely on other web apps to do their thing, and this is not necessarily as good a thing as we might first imagine. The main issue being single points of failure (SPOF). Many developers create (and are encouraged by the current web model to create) applications with many points of failure, without fallbacks. Take the Google CDN for jQuery for example – no matter how improbable, you take this down and half the web’s apps stop functioning. Google.com is probably the biggest single point of failure the world of software has ever known. Not good! Many developers think that if they can’t rely on Google, then who can they rely on? The point is that they should not rely on any single service, and the more single points you rely on, the worse it gets! It’s almost like we have forgotten the principles behind (and the redundancy built into) the Internet – a network based on the nuclear-war-tolerant ARPANET.
I believe it’s time to start looking at a new more robust, decentralised and distributed web model, and I think peer-to-peer (P2P) is a big part of that. Many clients, doubling as servers should be able to deliver the robustness we require, specifically SPOFs would be eliminated. Imagine Wikileaks as a P2P app – if we eliminate the single central server URL mode and something happens to wikileaks.com, the web app continues to function regardless. It’s no coincidence that Wikileaks chose to distribute documents on bittorrent. Another benefit of a distributed architecture is that it scales better, we already know this of course – it’s one of the benefits of client-side code. Imagine Twitter as a P2P web app, imagine how much easier it could scale with the bulk of processing distributed amongst its clients. I think we could all benefit hugely from a comprehensive web based P2P standard, essentially this boils down to browser-built-in web servers using an established form of inter-browser communication.
I’m going to go out on a limb here for a moment, bear with me as I explore the possibilities. Let’s start with a what if. What if, in addition to being able to download client-side JavaScript into your cache manifest you could also download server-side JavaScript? After all what is the server-side now, if not little more than a data/comms broker? Considering the growing popularity of NoSQL solutions and server-side JavaScript (SSJS) it seems to me that it wouldn’t take much to be able to run SSJS inside the browser, granted you may want to deliver a slightly different version than runs on the server, and perhaps this is little more than an intermediary measure. We already have Indexed Database API on the client, if we standardise the data storage API part of the equation, we’re almost there, aren’t we?
So let’s go through a quick example – let’s take Twitter again, how would this all work? Say we visit a twitter.com that detects our P2P capabilities and redirects to p2p.twitter.com, after the page has loaded we download the SSJS and start running it on our local server. We may also grab some information about which other peers are running Twitter. Assuming the P2P is done right, almost immediately we are independent of twitter.com, we’ve got rid of our SPOF. \o/ At the same time if we choose to run offline we have enough data and logic to be able to do all the things we would usually do, bar transmit and receive. Significantly load on twitter.com could be greatly reduced by running both server-side and client-side code locally. To P2P browsers, twitter.com would become little more than a tracker or a source of updates. Also let’s not forget, things run faster locally 
Admittedly the difference between server-side and client-side JS does become a little blurred at this point. But as an intermediary measure at least, it could be useful to maintain this distinction as it would provide a migration path for web apps to take. For older browsers you can still rely on the single server method while providing for P2P browsers. Another interesting beneficial side-effect is that perhaps we can bring the power of ‘view-source’ to server-side code. If you can download it you can surely view it.
The thing that I am most unsure about is what role HTTP plays in all this. It might be best to take an evolutionary approach to such a revolutionary change. We could continue to use HTTP to communicate between clients and servers, allowing the many benefits of a RESTful architecture, restricting the use of a more efficient TCP based comms to required real-time ‘push’ aspects of applications.
It seems to me that with initiatives such as Opera Unite (where P2P and a web server are already being built into the browser) and other browser makers looking to do the same, this is the direction in which web apps are heading. There are too many advantages to ignore: robustness, speed, scalability – these are very desirable things.
I guess what is needed now is some deep and rigorous discussion and if considered feasible, standards to be adopted so all browsers can move forward with a common approach for implementation. Security will obviously feature highly as part of these discussions and so will common library re-use on the client.
If web apps are going to compete with native apps, it seems that they need to be able to do everything native apps do and that includes P2P communication and enhanced offline capability, it just so happens that these two important aspects seem to be linked. Once suitable mechanisms and standards are in place, the fact that you can write web applications that are equivalent or even superior to native apps, using one common approach for all platforms, is likely to be the killer differentiator.
Thanks to @bluetezza, @quinnirill, @getify, @felixge, @trygve_lie, @brucel and @chrisblizzard for their inspiration and feedback.
Further reading : Freedom In the Cloud: Software Freedom, Privacy, and Security for Web 2.0 and Cloud Computing, The Foundation for P2P Alternatives, Social Networking in the cloud – Diaspora to challenge facebook, The P2P Web (part one)
26 Comments to P2P Web Apps – Brace yourselves, everything is about to change
Previous Posts
- The Hyperaudio Pad – Next Steps and Media Literacy
- Breaking Out – The Making Of
- Breaking Out – Web Audio and Perceptive Media
- Altrepreneurial vs Entrepreneurial and Why I am going to Work with Al Jazeera
- HTML5 Audio APIs – How Low can we Go?
- Hyperaudio at the Mozilla Festival
- The Hyperaudio Pad – a Software Product Proposal
- Introducing the Hyperaudio Pad (working title)
- Accessibility, Community and Simplicity
- Build First, Ask Questions Later
- Further Experimentation with Hyper Audio
- Hyper Audio – A New Way to Interact
- P2P Web Apps – Brace yourselves, everything is about to change
- A few HTML5 questions that need answering
- Drumbeat Demo – HTML5 Audio Text Sync
Tag Cloud
-
Add new tag
AJAX
apache
Audio
band
buffered
Canvas
CDN
chrome
community
custom tags
firefox
gig
HTC
HTML5
Hyper Audio
internet explorer
java
javascript
journalism
jPlayer
jQuery
jscript
LABjs
leopard
media
Mozilla
MVP
opera
opera mini
osx
P2P
Popcorn.js
poster
prototyping
rewrite
safari
Scaling
simplicity
SoundCloud
timers
tomcat
video
Web Apps
web design
[...] P2P Web Apps – Brace yourselves, everything is about to change [...]
While I’m paying (dearly) for bandwidth, I think I’m going to have a fairly negative attitude about my phone serving content.
Interesting thoughts. My questions focus on data: Are you proposing to download the entire Twitter content data store if p2p capabilities are detected? On the ad data store side of the equation, how would you address concerns by the Twitter’s of the world who required ad rotation which has to be explicitly served up by them using variable information such has user interests, time of day, location, etc?
Perhaps share the massive data store’s across the p2p connections a la the music & file -sharing p2p services of the past whereby if you’re connection & server met strict guidelines for performance a certain amount of data could be stored and served up by your local machine?
I imagine I’m probably looking too far ahead toward the end game, whereas your intermediary steps would definitely provide for *less* points of failure and thus more stable, faster and robust.
Again, nifty concept…I’ll be interested to see where the feedback goes and if the catches on.
The biggest issue I can see quickly is TRUST. Imagine going to a link farm and you are downloading some of their code that starts serving some crap or convertigyour browser in a spam bot or similar. Unless that problem is solved with some strict protocol that browsers follow, I wouldn’t want my browser to get anybody’s code.
Otherwise, I agree with your idea.
So start creating apps for Opera Unite already. Standards follow usage.
I was really big on web3.0 for for a while–which is what this sounds like–until I got hooked on scalability–which I think has more potential revenue. With all the interest in nosql databases out there, I suspect that others have followed the same course.
On the flip side, p2p webapps is a much more comprehensive tool than nosql. But it is harder. I think it will come in time as the various technologies that it depends on mature.
@Brett – I think we need to separate public data and private data. Public data could be distributed over many peers, whereas private data could be stored locally and possibly also on the originating server.
@Myself – I agree, I think there needs to be significant discussion on security and standards before this model can go mainstream. Personally I don’t think I’ve suffered from any security related issues using native P2P applications. In theory there’s no reason why moving to a web based P2P model would cause any more issues than native.
@Bill – I have no idea how long this technology will take to mature, but there are too many possible ‘wins’ for it not to be explored thoroughly. You are right of course, I need to stop pontificating and start creating apps to really see what the issues are. Less talk – more action!
Hope to find some time for that pretty soon!
I like the idea of building P2P into the browser for distributing applications. At one point in time I think we’ll see the need for web applications talking directly to each other.
Many look upon the video codec war as something related to what we serve from a server, but it’s just not that aspect to the video codec war. Video conferencing have for many, many years been troublesome between different clients (MSN, Skype etc) due to video codecs. If all browses had a minimum multiple video codec we could start looking at video conferencing directly in the browser. Adding a server to the browser and this can be done directly between browsers without the need for routing trough a central server. I think this might be one of the first browser to browser communications we’ll see.
From there to building a distribution model for web applications based on P2P in browsers we are not fare away I think.
Tough; I would like to see the application code being distributed and not data. This does force us web developers to have a much stronger separation between application code and data then we perhaps have been used to until now.
What worries me in such a setup are security. It would be fairly easy to modify code passed trough such a setup if we leave it open. We need some sort of security and signing model on this.
Another question I’m asking myself these days are; what kind of building blocks do we need to build real life replacements of “native applications” (think full blown replacements of OpenOffice, Photoshop etc..) and are we there (soon)?
It seems the Chrome team are working on just this functionality, if you have a dev build and go to about:flags there is a P2P API you can enable. Although they say it’s not working at the moment.
There is also some proposal specifications for the web apps device api for peer-to-peer communication http://www.whatwg.org/specs/web-apps/current-work/multipage/commands.html#peer-to-peer-connections
One more thing, firefox also has a plugin called BrowserSocket that allows you to essentially do P2P communication using WebSockets, very cool. Interestingly the people behind BrowserSocket also have another experiment which pretty much puts into practice everything you’ve written here. http://www.peerscape.org/ P2P social network, you own your own data.
Opera Unite was a cool concept, but the fundamental tech to implement it is just a very simple to implement reverse proxy running on Opera servers (single point of failure), and some webapp code in the browser talking with the server. it’s powerful, but a very centralized form of power, and one that is definitely not the most bandwidth efficient.
Ryan’s links are great. The WHAT-WG’s Peer to Peer connections is a start of what’s required for webapps to be able to host content. Past the pure connectivity side of things, if you’re consuming content hosted by a relaying party, you probably want it signed. So a hosting page need something like Salmon protocol to have the contents all signed. This is a lot of client-side crypto work, so hopefully someone will spec out an interface for native JS encryption.
If you’re into ambient computing, there’s still a host of questions about how a page broadcasts it’s presence to other people. The client will need to implement service discovery. Solutions like mDNS/DNS-SD/Avahi rely on multi-cast, which gets you part way, but WiFi itself limits who you can talk to. Enter WiFi Direct.
Just one other historical footnote, Flash introduced a limited form of P2P connectivity called Cirrus to enable peer to peer video streaming.
I have my own reverse proxy to client implementation available in my asynchronous bidirectional http transport Pipe Layer:
https://github.com/rektide/pipe-layer/blob/master/src/main/nodejs/reverse.js
there’s also http://www.unhosted.org/ , which is an amalgamation of projects to create a decentralized web. at this time there doesnt seem to be a strong focus on webapp-as-host.
Like ‘Myself’ says, I also think trust is the largest issue, and it depends on how decentralized you want the system to be. Decentralization makes some things (single point of failure, massive scaling) easy, but others much more difficult.
For instance, Limewire was a mostly decentralized network and they spent most of their time fighting search spam because there was no way to verify which results were faked. Also, a significant portion of LimeWire’s network traffic was just search and response traffic.
Some slightly decentralized options are a bit easier. Quite a while back I remember DeviantArt using some mechanism to try to distribute site load to its users; it turns out it was called RedSwoosh. (http://en.wikipedia.org/wiki/RedSwoosh)
Another interesting idea is to have individual peers serve the content, and centralized bittorrent-style trackers keep track of who has which parts.
Lastly, Diaspora is a decentralized social network. Right now there are just a handful of public nodes, but the founders’ long-term vision is to have everybody run their own diaspora nodes from under their beds. While not truly p2p right now, it seems like a good way to slowly migrate that direction.
This needn’t be as difficult as it might first seem. The server side tools could be moduled up, and stored as chunks of data that could be distributed by something resembling a bittorrent client. You could even use public/private key encryption to ensure that only you’re going to be updating your modules. By doing this, the only thing that you’d need to do on the client side, is find out whether your server modules are the correct version, and if they aren’t download them via P2P via the closest fastest connection.
All of the web could be done this way, including DNS lookups. As long as you had a trusted source at the top of the tree for your DNS entries, all of the data could be a sub-set of that, controlled by whoever has the time to create a server object. This would be no more or less secure than the current setup we have. In fact, if done right, it would be far far more secure.
@Trygve You make some good points regarding video. It will be interesting to see what will be possible when P2P and the Device API starts to see adoption.
I think we are pretty close to being able to build the native apps you mention as web apps, all the building blocks are more or less there – I’d say local storage & the ability to run code and access assets while offline are the main two. Now it appears browsers are even being allowed access to the filesystem. I think currently as web apps are working at a higher ‘level of abstraction’ than native apps they can be a little less snappy, but I don’t see this as an issue going forward – especially as browsers start to take full advantage of GPUs and hook deeper into the OS. In fact I think web apps have a big advantage over native apps in that they have networking baked-in. If you look at all the real innovations happening with native apps just now, they all seem to be related in some shape or form to networks, collaboration etc.
@Ryan Very interesting links, they really add a lot to the post, thanks!
@rektide I haven’t gotten around to writing any P2P web apps yet (although I look forward to doing so) so I’m not at all an expert in this area, in fact I’ve only recently started looking into it, but it seems very hard to get away from the single point of failure, if you want efficiency and importantly security. It’s interesting that BitTorrent perhaps the most successful P2P protocol offloads some of its processing to central servers (trackers), which seems to have given it a competitive edge over competing protocols that were purely P2P (although other factors may also be responsible). Also thanks for the links, I look forward to exploring them in detail when I start getting my hands dirty – I am now following you on github
@Michael Yes it does look like some sort of centralisation can be advantageous, total decentralisation is a huge challenge I feel. I used to use DeviantArt a lot – I remember their scaling problems! RedSwoosh sounds cool but again it includes a centralised directory, so I guess it solves the scaling problem but not the SPOF issue. Creating a number of trusted public nodes does sound like a viable alternative, it will be interesting to see how the Diaspora project pans out. I like the idea of steadily migrating from one system to another, ironing out the issues along the way.
@Pi I admire your optimism, I have to say I’m optimistic too. Just as I’m sure there are very significant issues, I’m also sure that as there is so much to be gained, these issues will eventually be overcome. This really was the spirit of my somewhat naive but optimistic post
few relative links
http://www.readwriteweb.com/hack/2011/03/p2p-api-discovered-in-latest-b.php
P2P API Discovered in Latest Builds of Chromium
+
http://my.opera.com/community/forums/topic.dml?id=251416
http://my.opera.com/community/forums/topic.dml?id=716222
I love this, but I don’t entirely agree.
There is definitely a big sea change coming, I think a lot of developers feel it at the moment and there’s no doubt that SSJS is a big driving force behind this. The line between client and server is blurring for sure.
I just think this might be a step too far, and I know this was deliberately idealistic, but I don’t think it’s necessary to go to this extent to deliver more resilience. Simple portability between server/client code will give a lot of this, and we’re so close already!
Yesterday was about developing on the server and providing progressive enhancement through the client. Tomorrow will just flip that on it’s head.
Great post though, and really enjoyed reading it. Wherever we find ourselves in a years time, I think it’s going to be an exciting place with a far richer experience.
Posted my response here:
http://pwpwp.blogspot.com/2011/05/serverless-social-web-apps.html
A bit overblown but good to get blog responses.
There is a nice trend of cloud-based redundant architectures that are cropping up more and more that are only increasing the trend for client-server architecture. These CDNs are designed to be redundant across hundreds of servers so not sure the point about single point of failure. The cloud trend is only getting bigger, not smaller so “last leg” must be a huge leg that is like a red wood tree.
P2P has its place for specific apps that need it. It will be a great thing to have the ability to do P2P as a web standard but its not going to be a one-sized fits all thing for all apps. It has a specific niche which is very useful but not going over 10% of type of apps needed.
Thanks for making the player by the way. You have very nice design + programming skills which is uncommon. I’m a jquery plugin writer and have a local storage plugin along with slider and form validations. Your player is awesome though. Very useful.
@Mike McNally:
> While I’m paying (dearly) for bandwidth, I think I’m going to have a fairly negative attitude about my phone serving content.
I once had an idea how to deal with devices with limited resources (although I have never spent time in developing the idea further). In the presence of identities in a P2P network, you could perhaps introduce the notion of roaming identities. Reputation/share ratio/etc. that you have built up on your main machine could then be transferable to other devices like your phone, allowing e.g. your phone to be just a leeching peer without any penalties.
It will be interesting to see if P2P web apps like the ones on facebook will move over to gambling and real money exchanges.
hi!
I have been thinking about this a lot recently. the thing is, highly scalable services tend to be such at Amazon’s Dynamo, are often peer-too-peer behind the firewall. that is one argument for peer-to-peer. Another is that a single point of failure can mean more than just a server going down. It can mean an organisation that does not have my best interests at heart having control over my data and the interface through which I access it. which of course means that the interface gets tuned to maximize their benefit, rather than mine.
It’s interesting to hear that there are proposed peer-to-peer apis in browsers, I wasn’t aware of that.
trust is the biggest issue for a true peer-to-peer system, if you have replicated your profile data to your friends, then they could do anything with it. that is just reality. you can’t go telling secrets to entities that you do not trust. The way forward is developing ways to identify who you _can_ trust. the internet itself is a reliable system built from unreliable components. possibly, you could use some sort of peer validation to detect fraud. bitcoin works like this. although it’s possible to have a few nodes compromised, you can’t compromise the system unless you compromise at least half the nodes.
true peer-to-peer is down the road somewhat, what will come first, I think is peer-to-peer behind the firewall, or maybe peer-to-peer between multiple services that depend on a common data source is much closer.
I’m working towards some tools for distributed software in javascript. with https://github.com/dominictarr/crdt you can replicate real-time data between the server and client, another tool to replicate that data peer-to-peer on your servers is creeping up my to do list.
P2P only inside the firewall is reasonable. Security for P2P is a tough nut to crack.
Absolutely, but it’s a fascinating problem.
Behind the firewall you assume trust. we accept that those servers or communication may fail, but if we are running peer to peer ‘in the wild’ we have to consider that those servers may be fully functional, but are _lying_.
But we deal with this every day, humans are fundamentally peer to peer. when someone tells you something you think it through your self, is it plausible? also, you corroborate it with what other people have told you. Although humans lie, they are mostly pretty honest. We depend on each other, and if the network shunned you, you would be in a much worse position than if you had just told the truth in the first place.
The computational complexity of coming up with another story that fits, but is to your advantage, is prohibitively expensive — that’s how bitcoin works. of course, this means the story has to be checked.
It all gets wonderfully complex and interesting, what if you falsely accuse someone of lieing? what is the cost of accumulating sufficient trust so that you can pull off a few lies?
Although, this is all academic. We need more tools for behind the firewall peer-to-peer, when that gets easy we can approach the problem of trust.
[...] But what can we do with it? WebRTC technology is very powerful, representing as it does a paradigm shift in the traditional client-server model, as such it is potentially hugely disruptive. Developers can look forward to building video and audio chat right into their apps and of course there’s many possibilities for real-time multi-player games and anything that requires you to push data around. Although currently we may need to touch the server, the server’s involvement can be minimal and so we have the building blocks for decentralised networks as we move towards a P2P web. [...]
Have you looked at the principles of successful distributed systems, such as for example found in erlang?
I think your strategic fore thought is excellent, but your technological fore thought a little naive. It is like reading about someone who thought up the idea for bit torrent, but tried to implemented it with pieces of nfs or window networking. Look at how fundamentally different bit torrent and a file server are, then apply these differences more carefully to your idea. I would also recommend reading the dynamo paper for ideas of what p2p takes. Then think of the problems of trust. I think these ideas well be implemented a few times in different Web applications before the architecture is fully understood enough to create a platform.