This page expands on the basic ideas of this proposed friend-to-friend system Murmur. If you have comments, or suggestions why it's a good or bad idea, please send them in by email.
Think of the way BitTorrent works. You start up your BitTorrent program to get the latest Ubuntu distribution, and you connect to a "tracker" - a server somewhere on the internet which keeps a list of all the people connected to it. So your program can find who else is online at the moment, and can use the tracker to contact the other people directly. When you want to get a piece of the Ubuntu file, you get it directly from the other client, not through the tracker. That's why it's called "peer-to-peer", because you as a peer get the goodness directly from another peer, and not from a server.
In the case of this messaging server, it wouldn't even have to know who of those people currently connected are your friends. It wouldn't have to know who you want to communicate to, what you've sent them, or whether you've sent anything at all. Just about all it would know is where your computer is connecting from, what name you want to use, and when you connect and disconnect.
So what happens if I want to send a message to someone who isn't online right now? That's ok, I can send the message to any of my friends who are online now and ask them to pass it on. Then when my intended recipient comes online, and I've gone, they can get the message from whoever is still online now. If I specify a maximum forwarding limit of 2 hops, then the message needn't traverse the whole network, just those with an indirect connection to me.
And what's to stop my other friends reading this message if they act unwittingly as a relay? Encryption. Each user of the program has a public key (which is public, given to everybody) and a private key, which they keep secret. So if I want to send a message to George, I take my message and encrypt it with George's public key. I send the message to Paul, but he can't read it because he can't decrypt it. Only when Paul forwards the message to George can George then use his private key to decrypt the message and read it. Paul can then delete his copy of the message of course.
In fact, Paul doesn't even need to know that I'm writing to George. Paul can probably figure out that the message is from me, because I sent it to Paul and must be asking him to be a relay. But that's not much information to leak. Paul can try and decode it, but it doesn't work, so the message can't be for Paul. Paul doesn't know who it's for, but can send it to everybody in his friends list (with a flag saying not to forward it any further), and just hope that somebody can decode it. Obviously Paul doesn't bother sending it to anyone that I've already sent it to - this can be checked with a hashed message id.
Some time later, George comes online, contacts the tracker and finds out who else is currently online. He then asks directly those online (who George is friends with) if there are any messages, and if I or Paul or one of Paul's friends are online and have a copy of my message, they can send it directly to George who can read it.
If enough of this is transparent, it should be as easy to use as email, and all I see is that I've sent a message to George and all he sees is that he receives a message from me. Nobody else needs to be informed that messages are being exchanged because they're not readable by them. Even the encryption can be completely transparent to make it as easy as possible.
The above description sounds much like email, albeit with the extra security of encryption. And the lack of a server backup of course. Much of the "publishing" aspect comes then from sending such a message to multiple recipients, perhaps all your friends. This is done in exactly the same way, except that instead of one public key being used for encryption, several keys are used, one for each of the intended recipients.
This idea quickly runs into scalability problems. Let's say I want to send a 1 MB message to ten of my friends. If I have to encrypt the message individually for each recipient, that means I have 10 different copies of my 1 MB message to send. But I need to send all the messages to all of my friends so that they can act as relays, so suddenly I'm trying to upload 100 MB instead of 1 MB. If each of my friends then try to upload all these copies to each of their friends too, there is an awful lot of wasted bandwidth and storage.
A much better idea is to generate a random symmetric key with which to encrypt my 1 MB message. I send this encrypted message to all my friends, and in addition I encrypt this random key with each of the public keys of my recipients. This means that there's only one large file being passed around, and 10 different tiny files for the key.
So how is this proposed system any better than what we already have? Firstly it provides secure email. Of course it's possible to send email securely already, but people don't, and even if they do, the sender, recipient, timestamp and subject are all passed in plain text around the internet. So it's easy to log who is communicating with whom and how much and how often. With a decentralized system, it's much more difficult to track.
You also get the possibility of an individual "wall" containing personal updates, thumbnail pictures, status, news and everything that's going on in your life. You add items to your profile, specify who can see them, and your updates appear for them to view. You can have a screen showing all your friends and what they're up to, casually graze if anything new has happened, without the obligation to regularly write a long email to keep in touch. And then when you do want to send a quick message, you can respond either publicly or privately, with a short or a long reply as you wish.
So it has the advantage over email of a more broadcasting nature with casual browsing of published messages possible, together with higher security and privacy than regular email. It has the advantage over Facebook (and similar platforms) of total control over your personal data and confidence that the company won't stamp all over your agreed privacy terms. It has the advantage over Twitter of much more flexibility in terms of message type and maximum message length.
You also get no advertising, and no leakage of personal information to advertisers, developers of those silly games that your friends play (which leak your information back to the game's developers), and of course Facebook itself.
The big killer is popularity. It's the fax effect again, no communication platform is useful if noone's in it already. One of the big draws of Facebook is the fact that so many people are already there, which any new platform obviously doesn't have. This brings the important added function of finding people you'd lost contact with - with a new platform you have to find them first, then persuade them to join the new platform. Which sounds like a hurdle.
Second - who wants to install a new application? All the web-based social networks just need a browser, which everybody already has. Downloading and installing more software? Unlikely to be popular. Perhaps Murmur could be developed as a Firefox Add-on, making it immediately cross-platform and easily installable to any system. I don't know if there are already building blocks available, like Firefox Add-ons for torrent clients, encryption etc. I don't even know if Firefox Add-ons are sandboxed or are able to interact with the local file system, accept network connections etc... more research is needed here.
Another disadvantage is resource usage. Sending email is efficient because you only need to send one copy to your email server, which forwards it to the recipient's email server, and then at some time later, a copy is sent to the recipient. In contrast, Murmur's proposed scheme involves sending multiple copies of the message to all one's friends, asking them to send copies to all their friends too, negotiating between peers which messages need to be exchanged and so on, just on the hope that sometime soon the recipient will be online at the same time as someone who has the message. Then it needs a mechanism for the peers to mark the message as expired after a certain time and recover the storage. It's not very efficient - but the longer clients spend online, the greater the chance that messages can be passed directly rather than via inefficient relaying.
Of course, real-time chat is perfectly possible within Murmur as long as both parties are online, if not then messages suffer from potentially high latency depending on the size of your Murmur network.
There's also the issue of backups - there isn't the safety net of emails also being stored on a central server, because there isn't one. So some backup routine might be a good idea. But that brings up another problem - a central server is accessible from any web browser anywhere. If you have a central server, you can check in from home, from work (if they allow it), from your friend's house or an internet cafe on holiday or wherever you want, even from a mobile device like a smartphone. A decentral system such as Murmur is only available where you have installed it, and even then only when it's been synchronized with your friends. So it's much more oriented towards a single, home-based access point. Which may also be a big problem for some people. If you wanted access from multiple devices, then you're back to needing an always-on server under your control, like the "Personal server model" discussed earlier. And you'd need encrypted communications from this server of yours to each of your clients, of course.
It's certainly possible to build such a system, and it would bring big advantages, but it needs to be fairly complex to work well. Clients should obviously be cross-platform, with a GUI, and easy to use. Encryption and decryption need to be made as transparent as possible. Perhaps an initial version would use text-only messages of a limited size (maybe 2 kB) to keep traffic down before allowing larger messages. Perhaps larger messages should only be transmitted directly rather than via relays. Or perhaps efforts should concentrate on documenting the protocol first and letting open source clients be developed by anyone who wants to?
One stumbling block could be the complexities of peer-to-peer networking, where many computers now are stuck behind routers, firewalls, proxies and other network hardware. These all make establishing direct peer-to-peer connections difficult and unreliable. Perhaps some of the techniques from other open source programs (BitTorrent, Transmission, Ekiga, ...) could be taken and adapted? Hole punching perhaps? It certainly seems that there's a lot of work involved in getting it right.
Another suggestion is rather than develop a new application with a new communications medium, why not add support to an existing application? Would it maybe be possible to extend an already existing application like Ekiga or Pidgin, and add Murmur as a supported protocol? Well that might certainly help in persuading people to install it, but the specialized and complex GUI required by Murmur wouldn't really fit into an existing voice or IM application.
So will it happen? Will Murmur exist at some point? Maybe, maybe not. The first step will be to set up some kind of dynamic tracker server which can respond to requests, with a permanent internet presence. We'll see how easy that might be to set up...
After some more digging, it seems like this is far from a new idea, with the basic principle being that of a so-called "darknet" (see wikipedia), and more specifically a "friend-to-friend network" (F2F rather than P2P). Although Murmur isn't intended for file sharing, the same basic principle of a closed, private communications network still holds. However, there are important differences between this proposed Murmur network and existing platforms like Freenet or Gnutella. In most of the existing platforms, the focus is on finding and sharing files, without caring too much where or who they come from. Often, complete and reliable anonymity is key. Files are split into chunks to allow distribution of bandwidth, and file fragments are requested and sent in real-time (apart from deliberate network lag).
Murmur has a different focus - instead of searching for specific files offered by complete strangers, the idea is to request all new information from known friends. Connections are direct, rather than hopped, but it is complicated by the fact that sender and recipient might not (or indeed probably won't) be online at the same time. And anonymity isn't a goal at all, quite the opposite. Messages should be demonstrably from a known friend (proved by digital signing), otherwise they'll be rejected. Another big difference is the routing complexity - anonymous P2P networks need to route requests across many hops around a huge network, and finding where a given file fragment is, is difficult. With Murmur, you already know the host and can contact it directly.
Many existing peer-to-peer networks are criticised for encouraging or enabling illegal file sharing. Murmur is less suitable for such uses, as it is not anonymous and the files are not chunked. Plus you need to know the sender first. Network performance is not going to be optimized for large files, but for its intended uses of small message transfer it doesn't need to be extremely performant. If you want to illegally transfer a file to your friend, there are much better ways to do it than with a tool like Murmur.
Looking now at how much work has already gone into a huge variety of peer-to-peer software, perhaps a more fruitful way to progress would be to fork and modify one of the existing GPL'd cross-platform clients like for example Limewire or Oneswarm. If a cut-down version of one of these could be stripped of its unneeded functions and clothed in a new GUI, extending the cryptographic requirements and adding a more "social network" interpretation, it could use a lot of the existing networking technology which has already been proven. However, delving into the innards of such huge projects is also not likely to be easy.
One more comment regarding the existing social networking platforms (or "social notworking" as it's apparently jokingly called due to its use in the office), there will obviously be users who wish to continue using their current network (eg Facebook) while also keeping in touch with other friends using Murmur. It obviously makes no sense to ask them to copy and paste their postings into both applications, so ideally a module in Murmur running on their machine would note that they have made a posting, extract the post from their Facebook wall, and automatically forward it onto the specified Murmur recipients. So the non-Facebook users would get relayed copies of the posted content too. That's a great way for them to keep in touch with what's being said, and they can reply too, within Murmur. The only problem is that other people's Facebook replies are only visible within Facebook, and the Murmur replies are only visible within Murmur. So there's still a danger of conversation fragmentation.
Spam probably wouldn't be too much of a problem, as the sender needs to be in the recipient's friends list already, together with a public key for identification. If a particular user does send unwanted messages, they can simply be removed from the friends list. Similarly, impersonation of users is also tricky as the attacker would need to have access to the user's private key, which is hopefully rather difficult. Other media such as email or phone could quickly resolve any ambiguity about new certificates etc, as all the users are known to each other anyway.
The next question is, how do you become a friend of someone? The tracker could provide a basic list of users to search through but that's not particularly scalable. As soon as the network grew to any reasonable size, another mechanism would be necessary, presumably multiple trackers or distributed user lists. However it's quite a nice feature if even the relationships between users is unknown. So it could be that another friend of yours is using Murmur but you don't know about each other. So there needs to be a balance between keeping as many details as possible private, and allowing people to find each other and become friends. For instance, should the friends of your friends be visible to you? I guess that should be the choice of each user.
Once someone has found someone who they think they know, they need some way to contact them, and request a friendship, much like Skype does. So that would presumably be a different type of packet with an encrypted introduction message, containing enough detail to convince the user that it is who they think it is. Then they can exchange public keys and the friendship is established. Or the request could be denied or simply ignored.
General messages can safely be relayed to allow asynchronous delivery. It might be that other types of messages could stipulate only direct delivery, especially if they contain large attachments. This would avoid overloading the network with large amounts of data. Or individual clients could perhaps stipulate they're willing to act as a relay but only if the message is smaller than a given size, only for a given time, only up to a certain total cache space, and so on.
Also I've no idea in what kind of format the data should be stored locally - probably either in a flat file system or perhaps in a lightweight database. The data for relaying is obviously stored encrypted, and the user shouldn't need to know it's there. The personal data could also be left encrypted, possibly thereby hitting read performance. Probably an export into plain-text HTML would be useful too for backups.
These basic ideas are continued in the following pages about existing projects and more ideas.