BitTorrent

From Wikipedia, the free encyclopedia

The BitTorrent logo

BitTorrent is a peer-to-peer (P2P) communications protocol for file sharing. The protocol was designed in April 2001 and implemented in Summer 2002 by programmer Bram Cohen, and is now maintained by BitTorrent, Inc.

BitTorrent is a method of distributing large amounts of data widely without the original distributor incurring the whole of the corresponding costs of hardware, hosting and bandwidth resources. Instead of the distributor alone servicing each recipient, under BitTorrent the recipients each also supply data to newer recipients, thus significantly reducing the cost and burden on any given individual source as well as providing redundancy against system problems, and reducing dependence upon the original distributor.

CableLabs, the research organization of the North American cable industry, believes that BitTorrent could represent 55% of the upstream traffic on the cable company's access network.^[1] CacheLogic puts that number at roughly 35% of all traffic on the Internet.^[2] Another paper states that some 18% of all broadband traffic carries torrent files needed to initiate BitTorrent downloads.^[3] The large discrepancies in these numbers could be caused by dissenting opinions on the methodology to measure P2P traffic on the Internet.^[4]

There are numerous compatible BitTorrent clients, written in a variety of programming languages, and running on a variety of computing platforms.

Operation

BitTorrent 4.0.4 running under Windows XP

A BitTorrent client is any program which implements the BitTorrent protocol. Each client is capable of preparing, requesting, and transmitting any type of computer file over a network, using the protocol. A peer is any computer running an instance of a client.

To share a file or group of files, a peer first creates a "torrent." This is a small file which contains metadata about the files to be shared, and about the tracker, the computer that coordinates the file distribution. Peers that want to download the file first obtain a torrent file for it, and connect to the specified tracker which tells them from which other peers to download the pieces of the file.

Though both ultimately transfer files over a network, a BitTorrent download differs from a classic full-file HTTP request in several fundamental ways:

• BitTorrent makes many small P2P requests over different TCP sockets, while web-browsers typically make a single HTTP GET request over a single TCP socket.
• BitTorrent downloads in a random or "rarest-first" approach that ensures high availability, while HTTP downloads in a contiguous manner.

Taken together, BitTorrent achieves much lower cost, much higher redundancy, and much greater resistance to abuse or "flash crowds" than a regular HTTP server. However, this protection comes at a cost: downloads take time to ramp up to full speed because these many peer connections take time to establish, and it takes time for a node to get sufficient data to become an effective uploader. As such, a typical BitTorrent download will gradually ramp up to very high speeds, and then slowly ramp back down toward the end of the download. This contrasts with an HTTP server that, while more vulnerable to overload and abuse, ramps up to full speed very quickly and maintains this speed throughout. Furthermore, BitTorrent's non-contiguous download methods prevent it from supporting "progressive downloads" or "streaming playback," as is possible with HTTP.

Creating and publishing torrents

In this animation, the coloured bars beneath all of the clients represent individual pieces of the file. After the initial pieces transfer from the seed, the pieces are individually transferred from client to client. The original seeder only needs to send out one copy of the file for all the clients to receive a copy.

The peer distributing the file breaks it down into a number of identically-sized pieces, typically between 64 kB and 1 MB each. Pieces over 512 kB are used to reduce the size of torrent files for very large payloads, but also reduce the efficiency of the protocol[1]. The peer creates a checksum for each piece, using a hashing algorithm, and records it in the torrent file. When a peer receives the piece, it compares the recorded checksum to the actual checksum of the received piece to test that it is error-free. ^[5] Peers that provide the complete file are called seeders, and the peer providing the initial copy is called the initial seeder.

The exact information contained in the torrent file depends on the version of the BitTorrent protocol. By convention, the name of a torrent file has the suffix .torrent. Torrent files contain an "announce" section, which specifies the URL of the tracker, and an "info" section which contains (suggested) names for the files, their lengths, the piece length used, and a SHA-1 hash code for each piece, which clients should use to verify the integrity of the data they receive.

The peer places a link to the file on a website or elsewhere, and registers it with a tracker. The tracker maintains lists of the clients currently participating in the torrent.^[5] Alternatively, in a trackerless system (decentralized tracking) every peer acts as a tracker. It is implemented by the BitTorrent, µTorrent, BitComet and KTorrent clients through the distributed hash table (DHT) method. Azureus also supports a trackerless method but it is incompatible with the DHT offered by all other supporting clients.

In November 2006, BitTorrent Inc released a service that automatically creates a torrent file and tracks a download based on a website object. The service is located at https://www.bittorrent.com/publish and requires a client that supports web-seeding (currently only the official client, Azureus, and μTorrent).

Downloading torrents and sharing files

Users browse the web to find a torrent file of interest, download it and open it with a BitTorrent client program. The client connects to the tracker specified in the torrent file, which sends it a list of other peers currently downloading pieces of the file to be downloaded. The client connects to those peers to obtain the various pieces. Such a group of peers connected to each other to share a torrent is called a swarm. If the swarm contains only the initial seeder, the client connects directly to it and begins to request pieces. As peers enter the swarm, they begin to trade pieces with one another, instead of downloading directly from the seeder.

Clients incorporate mechanisms to optimize their download and upload rates; for example they download pieces in a random order, to increase the opportunity to exchange data, which is only possible if two peers have different pieces of the file.

The effectiveness of this data exchange depends largely on the policies that clients use to determine to whom to send data. Clients may prefer to send data to peers that send data back to them (a tit for tat scheme), which encourages fair trading. But strict policies often result in suboptimal situations, where newly joined peers are unable to receive any data (because they don't have any pieces yet to trade themselves) and two peers with a good connection between them do not exchange data simply because neither of them wants to take the initiative. To counter these effects, the official BitTorrent client program uses a mechanism called “optimistic unchoking,” where the client reserves a portion of its available bandwidth for sending pieces to random peers (not necessarily known-good partners, so called preferred peers), in hopes of discovering even better partners and to ensure that newcomers get a chance to join the swarm.^[6]

Adoption of BitTorrent

A growing number of individuals and organizations are using BitTorrent to distribute their own or licensed material. The most notable of these is BitTorrent Inc. which has amassed a number of licenses from Hollywood studios for distributing popular content at the company's website. Independent adopters report that without using BitTorrent technology, with its dramatically reduced demands on networking hardware and bandwidth, they could not afford to distribute their files.

Many sites imitating big BitTorrent trackers have turned to distribution of only legal material in general. With several different categories they often provide a working distribution method for parties who want to have big material packets available for large audience without investing large amounts of money.

Amazon S3 (Simple Storage Service), a scalable Internet-based storage service with a simple web service interface, comes with built-in BitTorrent support.

BitTorrent is often used to distribute free and open source software, especially ISO images of Linux distributions. Most Linux distributions offer BitTorrent as one of the download methods for installation CDs.

Sub Pop Records, the record label credited with popularizing grunge rock and current publisher of The Shins, The Postal Service and other popular bands, is releasing numerous tracks and videos on BitTorrent^[7] to distribute its 1000+ albums. The band Ween uses the website Browntracker.net^[8] to distribute free audio and video recordings of live shows.

Blizzard's World of Warcraft video game utilizes the BitTorrent protocol to send game updates to clients.

Anime fansub communities often use BitTorrent for their releases, and the most popular announce sites like Animesuki have RSS feeds.

Blog Torrent offers a simplified BitTorrent tracker to enable bloggers and non-technical users to run a tracker off their site. Blog Torrent also allows visitors to download a "stub" file which acts as a BitTorrent client to download the desired file, allowing users without BitTorrent software to benefit from it.^[9] This is similar to the concept of a self-extracting archive.

The game GunZ The Duel has a built-in BitTorrent client.

Podcasting software is starting to integrate BitTorrent to help podcasters deal with the download demands of their MP3 "radio" programs. Specifically, Juice supports automatically processing .torrent files from RSS feeds. Similarly, some BitTorrent clients, such as µTorrent, are able to process web feeds and automatically download content found within them.

The creator of the BitTorrent protocol, Bram Cohen, at one time worked for Valve Software. Valve uses the BitTorrent protocol in their Steam media streaming frontend.

Legal issues

Copyright enforcement

BitTorrent trackers have been frequent targets of raids and shutdowns due to claims of copyright infringement. BitTorrent metafiles do not actually store copyrighted data, and thus it is often claimed^[10] that BitTorrent trackers, which only store and track the metafiles and usually do not share any potentially copyrighted data, must therefore be legal. Despite this claim, there has been tremendous legal pressure, usually on behalf of the MPAA and RIAA, to shut down numerous BitTorrent trackers.

In December 2004, the Finnish police raided a major BitTorrent site, Finreactor.^[11][12] The case is before the courts, and 32 people, in September 2006, mostly administrators and moderators, are facing charges. Software and media companies are seeking damages worth 3.5 million euros (about 4.6 million USD) in total. Two defendants were acquitted by reason of being underage at the time, but they are being held liable for legal fees and compensation for illegal distribution ranging up to 60,000 euros. The court set their fine at 10% of the retail price of products distributed.^[13]

Suprnova.org, one of the most popular early BitTorrent sites, closed in December 2004, supposedly due to the pressure felt by Sloncek, the founder and administrator of the site. In December 2004, Sloncek revealed that the Suprnova computer servers had in fact been confiscated by Slovenian authorities.^{[citation needed]}

LokiTorrent, arguably the biggest torrent source after the demise of Suprnova, closed down soon after Suprnova. Allegedly, after threats from the MPAA, Edward Webber (known as 'lowkee'), webmaster of the site, was ordered by the court to pay a fine and supply the MPAA with logs (the IP addresses of visitors).^[14] Webber, in the weeks following his receipt of the subpoena, had begun a fundraising campaign to pay legal fees in a legal battle against the MPAA. Webber raised approximately US$45,000 through a PayPal-based donation system. It is unclear how much of that money went to the MPAA. Following the agreement, the MPAA changed the LokiTorrent website to display a message intended to intimidate filesharers.^[15] Webber did not comment on this change.

On May 25, 2005, the popular BitTorrent website EliteTorrents.org was shut down by the United States Federal Bureau of Investigation and Immigration and Customs Enforcement. At first it was thought that a malicious hacker had gained control of the website, but it was soon discovered that the website had been taken over by the US government. Ten search warrants relating to members of the website were executed. - Newspaper, Butler Eagle, PA

On October 24, 2005, a 38-year-old Hong Kong BitTorrent user Chan Nai-ming (陳乃明, using the handle 古惑天皇 Lit. The master of cunning, while the magistrate referred to him as Big Crook) allegedly distributed the three movies Daredevil, Red Planet and Miss Congeniality in violation of copyright, subsequently uploading the torrent file to a newsgroup. He was convicted of breaching the copyright ordinance, Chapter 528 of Hong Kong law.^{[citation needed]} The magistrate remarked that Chan's act caused significant damage to the interest of copyright holders. He was released on bail for HK$5,000, awaiting a sentencing hearing, though the magistrate himself admitted the difficulty of determining how he should be sentenced due to the lack of precedent for such a case. On November 7, 2005, he was sentenced to jail for three months but was immediately granted bail pending an appeal to the High Court. The appeal was dismissed by the Court of First Instance on 12 December 2006 and Chan was jailed immediately. On 3 January 2007, he was bailed pending appeal to the Court of Final Appeal on 9 May 2007.

On November 23, 2005, the movie industry and BitTorrent Inc., whose CEO is the creator of BitTorrent, Bram Cohen, signed a deal they hoped would reduce the number of unlicensed copies available through bittorrent.com's search engine, run by BitTorrent, Inc. It meant BitTorrent.com had to remove any links to unlicensed copies of films made by seven Hollywood movie studios. As it covered only the BitTorrent.com website, it is unclear what overall effect this has had on copyright infringement.^[16]

In June 2006, the popular website Newnova.org, an exact replicant of Suprnova, was also subject to closure.

The Pirate Bay is another popular BitTorrent website. It was formed out of a Swedish anti-copyright group. The site also contains torrents which point to copies of copyright-protected material. The Pirate Bay is notorious for its "legal" section^[17] in which letters and replies on the subject of alleged copyright infringements are publicly displayed. The replies are written in a humorous manner and a hard copy of one was even sold on eBay for USD $255. On May 31, 2006, however, The Pirate Bay's servers, which are based in Sweden, were raided by Swedish police; the site owners might be facing charges for copyright infringement or facilitating it according to the accusations on the search warrant. No charges have been made so far.^{[citation needed]} However, after securing new servers in The Netherlands and using a recent backup, The Pirate Bay was back online in less than 72 hours. Recently, The Pirate Bay has returned to Sweden. The return has been facilitated by the public and media backlash against the Swedish Government's actions. The Pirate Bay is now, supposedly, going to counter-sue the Swedish government for millions of Swedish kronor (SEK) lost from having their website shut down.^{[citation needed]} A film relating to these incidents can be found here.

HBO, in an effort to combat the distribution of its programming on BitTorrent networks, has been sending out cease and desist letters to the Internet Service Providers of BitTorrent users. Many users have reported receiving letters from their ISP's that threatened to cut off their internet service if the alleged infringement continues. HBO, unlike the RIAA, has so far declined to sue anyone for sharing the files.^{[citation needed][18]}

Legal defenses

There are two major differences between BitTorrent and many other peer-to-peer file-trading systems, which advocates suggest make it less useful to those sharing copyrighted material without authorization. First, BitTorrent itself does not offer a search facility to find files by name. A user must find the initial torrent file by other means, such as a web search. Second, BitTorrent makes no attempt to conceal the host ultimately responsible for facilitating the sharing: a person who wishes to make a file available must run a tracker on a specific host or hosts and distribute the tracker address(es) in the .torrent file. While it is possible to simply operate a tracker on a server that is located where the copyright holder cannot take legal action, this feature of the protocol does imply some degree of vulnerability that other protocols lack. It is far easier to request that the server's ISP shut the site down than it is to find and identify every user sharing a file on a traditional peer-to-peer network. However, with the use of a distributed hash table (DHT), a tracker is no longer required, although they are often still used so that clients that do not support DHT can still connect to the swarm.

Limitations and security vulnerabilities

BitTorrent does not offer its users anonymity. It is possible to obtain the IP addresses of all current, and possibly previous, participants in a swarm from the tracker. This may expose users with insecure systems to attacks.^[6]

Another drawback is that BitTorrent file sharers, compared to users of client/server technology, often have little incentive to become seeders after they finish downloading. The result of this is that torrent swarms gradually die out, meaning a lower possibility of obtaining older torrents. Some BitTorrent websites have attempted to address this by recording each user's download and upload ratio for all or just the user to see, as well as the provision of access to older torrent files to people with better ratios. Also, users who have low upload ratios may see slower download speeds until they upload more. This prevents users from leeching, since after a while they become unable to download much faster than 1-10 kB/s on a high-speed connection. Some trackers exempt dial-up users from this policy, because they cannot upload faster than 1-5 kB/s.

BitTorrent is best suited to continuously connected broadband environments, since dial-up users find it less efficient due to frequent disconnects and slow download rates.

Technologies built on BitTorrent

The BitTorrent protocol is still under development and therefore may still acquire new features and other enhancements such as improved efficiency.

In May 2005, Bram Cohen released a new beta version of BitTorrent that eliminated the need for web site hosting of centralized servers known as "trackers." It is now possible to have a torrent up in minutes, with a file, a website, and no understanding of how it works. In addition, Cohen launched a new search service on BitTorrent's website, similar to those found on other popular sites such as The Pirate Bay.

Cohen explained that the tracker removal feature is part of his ongoing effort to make publishing files online "painless and disruptively cheap". The move is only one of several designed to remove BitTorrent's dependence on centralized trackers.

This change is said to cause some trouble in the legal efforts to shut down illegal file sharing. However, Tarun Sawney, BSA Asia antipiracy director, said BitTorrent files could still be identified, since with or without the tracker sites, actual users still host the infringing files.^[19][20]

Opera Software ASA now incorporates BitTorrent downloads through its popular browser software.^[21]

Indexing

The factual accuracy of this article or section may be compromised due to out of date information. Please see the relevant discussion on the talk page

The BitTorrent protocol provides no way to index torrent files. As a result, a comparatively small number of websites have hosted the large majority of torrents linking to copyright material, rendering those sites especially vulnerable to lawsuits. In response, some developers have sought ways to make publishing of files more anonymous while still retaining BitTorrent's speed advantage. The Shareaza client, for example, provides three alternatives to BitTorrent: eDonkey2000, Gnutella, and Shareaza's native network, Gnutella2(G2). If the tracker is down, it can finish the file over the other protocols, and/or find new (Shareaza) peers over G2. The use of distributed trackers is also one of the goals for Azureus 2.3.0.2 and BitTorrent 4.1.2. Another interesting idea that has surfaced recently in Azureus is virtual torrent. This idea is based on the distributed tracker approach and is used to describe some web resource. Right now, it is used for instant messaging. It is implemented using a special messaging protocol and requires an appropriate plugin. Anatomic P2P is another approach, which uses a decentralized network of nodes that route traffic to dynamic trackers.

BitTorrent search / Trackerless torrents

In June 2005, BitTorrent Inc. released a BitTorrent search engine [2], which searches the web for .torrent files, including those on popular BitTorrent trackers. BitTorrent has also licensed content which it co-mingles with search results on its website at bittorrent.com.^[22] From software version 4.2.0, BitTorrent also supports "trackerless" torrents, featuring a DHT implementation that allows the client to download torrents that have been created without using a BitTorrent tracker.

• BitTorrent Mainline DHT: BitTorrent client (4.1.0+), µTorrent (1.2+), BitComet (0.59+), and BitSpirit (3.0+): They all share a DHT based on an implementation[3] of the Kademlia DHT , for trackerless torrents.

Web seeding

One recently implemented feature of BitTorrent is web seeding. The advantage of this feature is that a site may distribute a torrent for a particular file or batch of files and make those files available for download from that same web server application; this can simplify seeding and load balancing greatly once support for this feature is implemented in the various BitTorrent clients. In theory, this would make using BitTorrent almost as easy for a web publisher as simply creating a direct download while allowing some of the upload bandwidth demands to be placed upon the downloaders (who normally use only a very small portion of their upload bandwidth capacity). This feature was created by TheSHAD0W, who created BitTornado.^[23]. From version 5.0 onward the Mainline BitTorrent client also supports web seeds and the BitTorrent web site has a simple publishing tool that creates web seeded torrents. The latest version of the popular download manager GetRight supports downloading a file from both HTTP/FTP protocols and using BitTorrent.

Broadcatching

Another proposed feature combines RSS and BitTorrent to create a content delivery system dubbed broadcatching. Since a Steve Gillmor column for Ziff-Davis in December 2003, the discussion has spread quickly among many bloggers (Techdirt, Ernest Miller, and former TechTV host Chris Pirillo, for example). As Scott Raymond explained:

I want RSS feeds of BitTorrent files. A script would periodically check the feed for new items, and use them to start the download. Then, I could find a trusted publisher of an Alias RSS feed, and 'subscribe' to all new episodes of the show, which would then start downloading automatically — like the 'season pass' feature of the TiVo.

– ^[24]

While potential illegal uses abound as is the case with any new distribution method, this idea lends itself to a great number of ideas that could turn traditional distribution models on their heads, giving smaller operations a new opportunity for content distribution. The system leans on the cost-saving benefit of BitTorrent, where expenses are virtually non-existent; each downloader of a file participates in a portion of the distribution. One early adoption of this concept is IPTV show mariposaHD, which uses BitTorrent to distribute large (2-4 GB) WMVHD files of high-definition video.

RSS feeds layered on top keep track of the content, and because BitTorrent does cryptographic hashing of all data, subscribers to the feed can be sure they're getting what they think they're getting, whether that winds up being the latest Sopranos episode, or the latest Sveasoft firmware upgrade. (Naturally, however, ensuring that the same data reaches all nodes neglects the possibility that the original, source file may be corrupted or incorrectly labeled.)

One of the first open source attempts to create a client specifically for this was Democracy Player. The idea is already gaining momentum however, other Free Software clients such as PenguinTV and KatchTV are also now supporting broadcatching.

APIs

The BitTorrent web-service MoveDigital has the ability to make torrents available to any web application capable of parsing XML through its standard Representational State Transfer (REST) based interface.^[25] Additionally, Torrenthut is developing a similar torrent API which will provide the same features, as well as further intuition to help bring the torrent community to Web 2.0 standards. Alongside this release is a first PHP application built using the API called PEP which will parse any Really Simple Syndication (RSS 2.0) feed and automatically create and seed a torrent for each enclosure found in that feed.^[26][27]

Encryption

Main article: BitTorrent protocol encryption

Protocol header encrypt (PHE) and Message stream encryption/Protocol encryption (MSE/PE) are features of some BitTorrent clients that attempt to make BitTorrent hard to throttle. At the moment Azureus, Bitcomet and µTorrent, the three biggest BitTorrent clients, support MSE/PE encryption.

Some ISPs throttle BitTorrent traffic because it makes up a large proportion of total traffic and the ISPs don't want to spend money purchasing extra capacity.^[28] Encryption makes BitTorrent traffic harder to detect and therefore harder to throttle. Recently, ISPs have announced possible future hardware upgrades in order to minimize BitTorrent traffic.^{[citation needed]}

Several universities have already taken these steps, including the University of Maryland at College Park, Emory University, Brigham Young University, ASU, UTC, University of Washington, Washington University in St. Louis^[29], and WPI.^{[citation needed]}

Tor workaround

It is possible to route BitTorrent traffic into the .onion domain via Tor.

• However, client and server support for this is nearly non-existent, and there is generally a loss in network efficiency when doing so^{[citation needed]}.
• Future versions of BitTorrent may support Tor, but this may take two or three years to happen^{[citation needed]}.

Peer exchange

Main article: Peer exchange

Peer exchange (PEX) is another method to gather peers for BitTorrent in addition to trackers and DHT. Peer exchange checks with known peers to see if they know of any other peers.

Multitracker

Another unofficial feature is an extension to the BitTorrent metadata format proposed by John Hoffman.^[30] It allows the use of multiple trackers per file, so if one tracker fails, others can continue supporting file transfer. It is implemented in several clients, such as BitTornado, KTorrent and µTorrent. Trackers are placed in groups, or tiers, with a tracker randomly chosen from the top tier and tried, moving to the next tier if all the trackers in the top tier fail.

BitTorrent-related software

Main article: Comparison of BitTorrent software

Because of the open nature of the protocol, many clients have been developed that support numerous platforms and written using various programming languages.

BitTorrent-related hardware

An increasing number of hardware devices are being made to support BitTorrent. These include routers and NAS devices.

• Asus WL-700gE router
• Thecus YES Box N2100 BYOD NAS
• Coolmax CN-570 BYOD NAS

References

See also

Free software Portal

• Terminology of BitTorrent
• List of BitTorrent RSS feeds
• Comparison of file sharing applications
• Super-seeding
• Magnet link

External links

Find more information on BitTorrent by searching Wikipedia's sister projects
	Dictionary definitions from Wiktionary
	Textbooks from Wikibooks
	Quotations from Wikiquote
	Source texts from Wikisource
	Images and media from Commons
	News stories from Wikinews
	Learning resources from Wikiversity

• Official BitTorrent Inc Website
• BitTorrent Specification
• BitTorrent at the Open Directory Project (suggest site)
• Interview with chief executive Ashwin Navin
• BitTorrent Guide and FAQ
• BitTorrent Introduction