Külföldi torrent oldalak RedActed (PTH) | ReD Tracker Traffic

[ This is going to be a tad technical. ]

While you have been ripping, tagging and seeding, we have been grappling with infrastructure challenges of the sort that arise in heavy traffic. We know our users — good-looking and intellectually curious as you are — enjoy an occasional peek under the hood. That is the goal of today's post: a glimpse into the technical aspects of tracker management.

As per this external tally, we have the privilege to connect a lot of peers. To contextualize that spreadsheet, our peer count exceeds the next largest (MAM) by ~50% and the next most frequented music tracker (OPS) by ~3x. In the words of poet Kendrick Lamar, they not like us.

StarLord wrote:Because our torrents have a smaller size, our announces arrive in bursts; with growth, these bursts compound. We track more torrents, more peers, more snatches, more announces.What does this mean in practice? Well, in a recent one-hour window, ocelot processed 37,156,462 requests originating from 20,432 different IPs on behalf of 23,776 users. Traffic patterns vary but this is reasonably representative, if a touch high. More than ½ of all announces arrive via HTTP/1.1, roughly ⅓ via HTTP/2 and just a few via HTTP/3 or HTTP/1.0 (yes, really 1.0). In a proximate one-hour window, ~123GiB of data traveled on the wire between your torrent clients and the tracker. This amount includes not just application-level data (more on that below), but also TCP overhead and the TLS handshake:


Source: Cloudflare

How does that handshake look vis-à-vis RED?


That is 3,694 bytes of just TLS + TCP overhead, i.e. the bandwidth cost of initiating then closing a secure connection. For comparison, at the application layer during a typical announce, the client might send 362 bytes to receive a 392 byte response from the tracker, incurring some additional TCP overhead along the way. In other words, for BitTorrent, the data transfer for simply "speaking" TLS far exceeds the application payload. To avoid over-paying this toll on announces occurring in bursts, clients ought to send multiple requests over a single connection, utilize multiplexing and/or TLS session resumption.

Currently, five clients make the bulk of all requests; remarkably, just one (Transmission) tends to reuse connections! In the case of popular clients Deluge and qBittorrent, both of which announce via libtorrent-rasterbar, there is no reuse at all. It appears this problem was acknowledged by the developer in 2020 but unfortunately not prioritized. rTorrent uses a separate libtorrent library under the hood and appears to send announces via libcurl.

In the table below, the small percentage of connection reuse associated with qBittorrent reflects a custom reverse proxy. If you are interested in configuring that to conserve bandwidth (both ours and yours), please get in touch. Better still, if someone could convince the developers of libtorrent-rasterbar to adopt modern protocols and best practices.

What are the most common (top five) torrent clients?
client | IPs | users | requests | app_traffic | connection_reuse
-------------+------------+--------------+----------------+---------------+---------------------------
qbittorrent | 11294 | 10792 | 14891274 | 10.70 GiB | 874020/14891274 (5.87%)
rtorrent | 3587 | 6909 | 11272780 | 5.77 GiB | 1814458/11272780 (16.10%)
transmission | 4213 | 4185 | 8110603 | 4.08 GiB | 7384086/8110603 (91.04%)
deluge | 2342 | 2656 | 1599822 | 1.17 GiB | 0/1599822 (0.00%)
utorrent | 1028 | 894 | 934025 | 719.48 MiB | 960/934025 (0.10%)
[ this and all other data based on same one-hour window ]

Some of you are still announcing in plaintext ...
tls_protocol | requests |
-------------+---------------
TLSv1.3 | 32663134 |
Plaintext | 2368765 |
TLSv1.2 | 2124563 |

Which client versions are most common?
+-------------------------------------------+-------+------------+--------------+
| Client_Version | Users | % of Total | Multi-Client |
+-------------------------------------------+-------+------------+--------------+
| rtorrent/0.9.8 | 5088 | 21.8 | 763 |
| qBittorrent/5.1.0 | 2299 | 9.9 | 305 |
| qBittorrent/5.1.2 | 1795 | 7.7 | 256 |
| Transmission/4.0.5 | 1498 | 6.4 | 291 |
| rtorrent/0.9.6 | 1336 | 5.7 | 206 |
| Deluge/2.1.1 | 950 | 4.1 | 285 |
| Transmission/3.00 | 944 | 4.1 | 241 |
| Transmission/4.0.6 | 867 | 3.7 | 147 |
| qBittorrent/5.0.1 | 835 | 3.6 | 155 |
| Deluge/2.2.0 | 813 | 3.5 | 203 |
+-------------------------------------------+-------+------------+--------------+
[ Multi-Client = number of users using that client that also use at least one other ]

Largest seeders?
+---------------------------------------+--------+
| User | TiB |
+---------------------------------------+--------+
| https://redacted.sh/user.php?id=32427 | 331.87 |
| https://redacted.sh/user.php?id=00000 | 161.53 |
| https://redacted.sh/user.php?id=1738 | 70.35 |
| https://redacted.sh/user.php?id=27212 | 62.64 |
| https://redacted.sh/user.php?id=00000 | 39.37 |
| https://redacted.sh/user.php?id=4204 | 32.26 |
| https://redacted.sh/user.php?id=31077 | 31.73 |
| https://redacted.sh/user.php?id=00000 | 31.26 |
| https://redacted.sh/user.php?id=00000 | 28.71 |
| https://redacted.sh/user.php?id=38673 | 28.09 |
| https://redacted.sh/user.php?id=00000 | 27.30 |
| https://redacted.sh/user.php?id=53567 | 27.19 |
| https://redacted.sh/user.php?id=00000 | 27.18 |
| https://redacted.sh/user.php?id=54778 | 25.59 |
| https://redacted.sh/user.php?id=00000 | 22.40 |
| https://redacted.sh/user.php?id=33185 | 21.64 |
| https://redacted.sh/user.php?id=00000 | 20.92 |
| https://redacted.sh/user.php?id=00000 | 20.18 |
| https://redacted.sh/user.php?id=59792 | 18.98 |
| https://redacted.sh/user.php?id=5169 | 18.73 |
+---------------------------------------+--------+
These numbers are impressive but potentially misleading. Underlying data does not determine announce (and thus, traffic) patterns; rather, it is the number of seeds. In the latter context, there is incomplete overlap with the above list of largest (data) seeders.
+---------------------------------------+---------+
| User | Value |
+---------------------------------------+---------+
| https://redacted.sh/user.php?id=32427 | 862,697 |
| https://redacted.sh/user.php?id=00000 | 359,534 |
| https://redacted.sh/user.php?id=1738 | 289,518 |
| https://redacted.sh/user.php?id=00000 | 138,782 |
| https://redacted.sh/user.php?id=53567 | 107,345 |
| https://redacted.sh/user.php?id=27212 | 89,112 |
| https://redacted.sh/user.php?id=00000 | 81,649 |
| https://redacted.sh/user.php?id=00000 | 70,602 |
| https://redacted.sh/user.php?id=31077 | 66,266 |
| https://redacted.sh/user.php?id=112 | 64,234 |
| https://redacted.sh/user.php?id=00000 | 62,268 |
| https://redacted.sh/user.php?id=00000 | 62,202 |
| https://redacted.sh/user.php?id=5169 | 58,725 |
| https://redacted.sh/user.php?id=00000 | 56,262 |
| https://redacted.sh/user.php?id=00000 | 52,880 |
| https://redacted.sh/user.php?id=00000 | 52,694 |
| https://redacted.sh/user.php?id=33185 | 52,679 |
| https://redacted.sh/user.php?id=00000 | 47,624 |
| https://redacted.sh/user.php?id=4204 | 47,283 |
| https://redacted.sh/user.php?id=64486 | 45,204 |
+---------------------------------------+---------+
[ 00000 UIDs correspond to users who declined attribution ]
The user (Complex) seeding 862,697 torrents generates less traffic with the tracker than those seeding much less. Complex uses qBittorrent, routing via Caddy's reverse proxy, which converts HTTP/1.1 → HTTP/2 and pushes thousands of announces over a single connection. The second largest seeder uses Transmission, which fortunately reuses connections out of the box.

What's the bottom line?
Security is paramount: HTTPS trumps HTTP. We have previously indicated that RED will not forever support plaintext. Given the nature and amount of traffic we receive, TLS skews up the overheadayload ratio. Connection reuse, multiplexing and generally modern techniques push that ratio down. Security, performance and efficiency are not mutually exclusive; we can pursue them all.

Discuss this post here