Bittorrent specification mess

I am the author of a (rather unpopular ;-)) BitTorrent client called Bitflu and started to notice that the protocol specification went from ‘very good’ to ‘bad’ in the last few months. Sometimes there are undocumented features/extensions, sometimes it’s just hard to find the location of the documentation.

This small page attempts to collect and extend the BitTorrent protocol documentation.

Bencoding

All clients need to implement some encoding called Bencoding (it’s like a binary form of XML). This is documented quite well at http://wiki.theory.org/BitTorrentSpecification#Bencoding.

TCP Trackers

Well documented at http://wiki.theory.org/BitTorrentSpecification#Tracker_HTTP.2FHTTPS_Protocol and http://bittorrent.org/beps/bep_0023.html (BEP 023 is about ‘compact’ responses). Note that there are many broken and facistic trackers out in the wild.

UDP Trackers

Documentation about UDP trackers can be found at http://bittorrent.org/beps/bep_0015.html. The IPv6 support is documented at http://opentracker.blog.h3q.com/2007/12/28/the-ipv6-situation/. Note that there are currently no known UDP-Trackers supporting the IPv6 extension.

On Wire protocol

There are three on wire protocols: The original, the Azureus version and the uTorrent version. I’m only focusing on the original version that is documented at http://wiki.theory.org/BitTorrentSpecification#Peer_wire_protocol_.28TCP.29 and http://bittorrent.org/beps/bep_0003.html

Kademlia / DHT

The original Kademlia documentation can be found at http://bittorrent.org/beps/bep_0005.html

You should also read http://www.cs.rice.edu/Conferences/IPTPS02/109.pdf and http://pdos.csail.mit.edu/~petar/papers/maymounkov-kademlia-lncs.pdf.

Information about the IPv6 extension can be found at http://bittorrent.org/beps/bep_0032.html.

Clients should also start to implement this extension: http://www.rasterbar.com/products/libtorrent/dht_sec.html.

Eproto

Information about the extension protocol handshake can be found at http://www.rasterbar.com/products/libtorrent/extension_protocol.html.

ut_pex

This extension can be used to exchange peers between two connectd peers.

Note that clients do not have to request PEX messages, they simply receive it from other peers. A typical payload looks like this:

dropped => 'compact_list_of_dropped_peers',
added   => 'compact_list_of_new_peers',
added.f => 'flag_bitmap',

Also have a look at the implementation used in Bitflu.

ut_metadata

Used to exchange bencoded metadata (aka .torrent files) while downloading using a magnet link. The bencoded metadata is split into virtual chunks (each 16384 bytes): A client requests these chunks from a peer that already has the metadata and will be able to verify the received data by recalculating the info_hash. This is documented at http://bittorrent.org/beps/bep_0009.html. Bitflu’s implementation can be found by searching for UtMeta functions in this file.

ut_comment

Extensions to exchange comments between BitTorrent peers. Currently undocumented, the Bitflu implementation can be found here.

Comments have to be requested (uTorrent and Bitflu do this each 20 minutes (per peer)), a request looks like this:

{ num=>20, msg_type=>0, filter=>"\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0....(64 bytes)" }

A peer will reply with an array of hashes like this (msg_type will be set to ‘1’):

{ like=>2, owner=>"", timestamp=>300, text=>"bla" }

• like should be called rating: It’s an integer between 0 (= unrated) and 5 (= 5 stars).
• owner seems to be unused. (always an empty string)
• timestamp isn’t a timestamp: It’s the age of the comment in seconds
• text is an UTF8 encoded string

DHT votes

Yet another uTorrent extension to implement ‘dht votes’.

The ‘vote id’ that belongs to an ‘info_hash id’ can be calculated via:

$vote_id = sha1($info_hash."rating"); # Note: "rating" is just a string

The client should then send a ‘vote’ query to the K closest nodes: (Bitflu’s implementation).

The value of ‘vote’ can be between 0-5 (0 means: just query).