Freenet

Freenet is a decentralized, peer-to-peer communication network designed to be resistant to censorship, which pools the contributed bandwidth and storage space of member computers to allow users to anonymously publish or retrieve information of all kinds. The term also has an older meaning, referring to text-based community networks offering limited Internet services at little or no cost.

Table of contents

1 Raison d'être
2 Technical design
3 History
4 Useful Tools

4.1 Freesite Insertion Wizard (FIW)
4.2 Fishtools
4.3 Freenet Utility for Queued Inserts and Downloads (FUQID)
4.4 FCPTools
4.5 Freenet Tools
4.6 Frost
4.7 Freenet Message Board (FMB)
4.8 FreeNews
4.9 Flinks

5 See also
6 External link

Raison d'être

Although many nations censor communications to different extents they all share one commonality in that a body must decide what to censor versus what to let flow freely. What may be acceptable to one group of people may be considered offensive or even dangerous to another. Freenet is a network which removes the possibility of any group imposing their beliefs or values on any other. In essence nobody is allowed to decide what is acceptable for anybody else. Tolerance for each others' values is encouraged and failing that the user is asked to turn a blind eye to content which opposes his or her views.

The same technology that lets the oppressed communicate to a large group without either the publisher or the readers' identities being revealed also allows child pornographers to post their pictures for anybody who actively seeks them to see. For this reason the Freenet manifesto has come under fire and accordingly the network has been designed with resiliency in mind.

Technical design

The Freenet file sharing network is designed to be highly survivable, with all internal processes completely anonymized and decentralized across the network. The system has no central servers and is not subject to the control of any one individual or organization. Even the designers of Freenet do not have any control over the overall system. The system is designed so that information stored in the system is encrypted and replicated across a large number of continuously-changing anonymized computers around the world. It is extremely difficult for an attacker to find out which participants are hosting a given file, since the contents of each file are encrypted, and can also be broken into sections that are distributed over many different computers. Even the participants themselves don't know what they are storing.

The end goal of the Freenet network is to store documents and allow them to be retrieved later by an associated key, as is now possible with protocols such as HTTP. The network is implemented as a number of nodes that pass messages among themselves peer-to-peer. Typically, a host computer on the network will run the software that acts as a node, and it will connect to other hosts running that same software to form a large distributed network of peer nodes. Certain nodes will be end user nodes, from which documents will be requested and presented to the human user. But these nodes communicate with each other and with intermediate routing nodes identically—there are no dedicated "clients" or "servers" on the network.

The Freenet protocol is intended to be implemented on a network with a complex network topology, much like the (internet protocol). Each node knows only about some number of other nodes that it can reach directly (its conceptual "neighbors"), but any node can be a neighbor to any other; there is no hierarchy or other structure. Each document (or other message such as a document request) in Freenet is routed through the network by passing from neighbor to neighbor until reaching its destination. As each node passes a document to its neighbor, it does not know or care whether its neighbor is just another routing node forwarding information on behalf of another, whether it is the source of the document being passed, or whether it is a user node that will present the document to an end user. This is intentional, so that anonymity of both users and publishers can be protected.

Each node maintains a data store containing documents associated with keys, and a routing table associating nodes with records of their performance in retrieving different keys.

To find a document in the network given a key, a user sends a message to a node (probably one running on the same machine as the client program) requesting the document, providing it with the key. If the document is not found in the local datastore, the node then finds the node in its routing table that it thinks will be able to locate the key most quickly, and forwards the request to that node, remembering that it has done so. Note that this is a change from the previous behavior: it represents the "Next Generation Routing" protocol. The old behavior was to remember which keys were retrieved from what nodes, and to route based on which node gave us the key closest to the one we were looking for. The effect is largely the same, but NGR, as it is called, should result in better overall performance.

The node to which the request was forwarded repeats the process until either the key is found or the request passes through a set maximum of nodes, known as the "Hops To Live" value. Along the route, if a node is visited more than once (and it will know this because it remembered forwarding the request the first time) then that node cuts off the loop by sending a message to the node that sent it the second request telling it to try the next-best choice, then the next-next-best, and so on.

Eventually either the document is found or the hop limit is exceeded, at which point the node sends back a reply that works its way back to the originator along the route specified by the intermediate nodes' records of pending requests. The intermediate nodes may choose to cache the document along the way. Besides saving bandwidth, this also makes documents harder to censor as there is no one "source node."

Essentially the same path-finding process is used to insert a document into the network: a request for the nonexistent document is made, and once it fails, the document is sent along the same path as the request. This insures that documents are inserted into the network in the same place as requests will look for it. If the initial request doesn't fail, then the data already existed, and the insert "collides."

Initially, each node has no information about the performance of the other nodes it knows about. This means that routing of requests will be essentially random. But since different nodes have different randomness, they will disagree about where to send a request, given a key. So the data in a newly-started Freenet will therefore be distributed somewhat randomly.

As more documents are inserted by the same node, they will begin to cluster with data items whose keys are similar, because the same routing rules are used for all of them. More importantly, as data items and requests from different nodes "cross paths", they will begin to share clustering information as well.

The result is that the network will self-organize into a distributed, clustered structure where nodes tend to hold data items that are close together in key space. There will probably be multiple such clusters throughout the network, any given document being replicated numerous times, depending on how much it is used. This is a kind of "spontaneous symmetry breaking", in which an initially symmetric state (all nodes being the same, with random initial keys for each other) leads to a highly asymmetric situation, with nodes coming to specialize in data that has closely related keys.

There are forces which tend to cause clustering (shared closeness data spreads throughout the network), and forces that tend to break up clusters (local caching of commonly used data). These forces will be different depending on how often data is used, so that seldom-used data will tend to be on just a few nodes which specialize in providing that data, and frequently used items will be spread widely throughout the network. This automatic mirroring counteracts the Slashdot effect, and due to a mature network's intelligent routing a network of size n should only require log(n) time to retrieve any given document. Freenet does not employ broadcast searches as used by Gnutella and other similar file sharing protocols.

One thing to keep in mind is that keys are hasheses, hence there is no notion of semantic closeness when speaking of key closeness. Therefore there will be no correlation between key closeness and similar popularity of data as there might be if keys did exhibit some semantic meaning, thus avoiding bottlenecks caused by popular subjects.

There are two main varieties of keys in use on Freenet, the Content Hash Key (CHK) and the Signed Subspace Key (SSK).

A CHK is an SHA-1 hash of a document and thus a node can check that the document returned is correct by hashing it and checking the digest against the key. This key contains the meat of the data on freenet. It carries all the binary data building blocks for the content to be delivered to the client for reassembly and decryption. The CHK is unique by nature and provides tamperproof content. A cancer node messing with the data under a CHK will immediately be detected by the next node or the client. CHKs also reduce the redundancy of data since the same data will have the same CHK.

SSKs are based on public-key cryptography. Currently Freenet uses the DSA system as its public key infrastructure. Documents inserted under SSKs are signed by the inserter, and this signature can be verified by every node to ensure that the data is not tampered with. SSKs can be used to establish a verifiable pseudonymous identity on Freenet, and allow for documents to be updated securely by the person who inserted them. A subtype of the SSK is the Keyword Signed Key, or KSK, in which the key pair is generated in a standard way from a simple human-readable string. Inserting a document using a KSK allows the document to be retrieved and decrypted if and only if the requester knows the human-readable string; this allows for more convenient (but less secure) URIs for users to refer to.

History

Freenet is an enhanced Open Source implementation of the system described by Ian Clarke's July 1999 paper "A distributed decentralized information storage and retrieval system." Shortly after the publication of this paper work started on Freenet by Clarke and a small number of volunteers. By March 2000 version 0.1 of Freenet was ready for release. Since March 2000 Freenet has been extensively reported on in the press, albeit primarily due to its implications for copyright rather than for its wider aim of freedom of communication.

Reports of Freenet's use in authoritarian nations is difficult to track due to the very nature of Freenet's goals. One group, Freenet-China, has translated the Freenet software to Chinese and is distributing it within China on CD and floppy disk.

According to Citeseer, Ian Clarke's "Freenet: A Distributed Anonymous Information Storage and Retrieval System" was the most cited computer science paper of 2000.

Useful Tools