I'm using libtorrent 0.15.5 and qBittorrent 2.6.6. At the end of downloading (nearly 99.5%) a speed rapidly falls to several kbytes per second. Sometimes it's need to restart a torrent to download it completely. I asked another qBittorrent user. He said that he also noticed it.

I believe this has been fixed in svn (branches/RC_0_15). If you have a chance, please confirm this. If anyone can confirm seeing this fixed I will cut 0.15.6 out of svn.

Maybe I'm a bad tester, but svn version works worse :( I tested two torrents. Both of them was already partly-downloaded. I marked some (not all) files, which wasn't downloaded before, to be downloaded. Maybe this is an illusion, but I noticed that in this situation problem affects harder. First torrent: At first all worked ok, the speed was normal. but nearly at 90% the speed started to fall smoothly. Nearly at 95% the speed felt twenty times comparing to a normal speed. At 99,8% a download process stopped fully, the speed was 0. libtorrent found another peer to download, the speed rose to 1,5 kbytes per second (very-very bad speed, a normal speed is nearly 700 kbytes per second), but felt to 0 soon. I waited two minites, but the speed didn't changed, a download process stopped fully at 99.9%. Second torrent: At first all worked ok, the speed was normal. but nearly at 99% the speed started to fall rapidly. At 99,9% a download process stopped fully, the speed was 0. I waited one minites, but the speed didn't changed, a download process stopped fully again. Both torrents have a lot of peers.

Ok. When I fixed this I was working out trunk, I should take another look at RC_0_15.

I haven't been able to observer this. Are there any particular circumstances where you see it?

I tried to catch you in IRC channel, but couldn't :) I asked another qBittorrent/libtorrent user some time ago. He said: "I usually don't restart torrents. After some time of being non-active (download speed is zero) they downloads fully. Also sometimes Transmission have a similar problem". I tried another torrent. Speed falls rapidly to zero at 99.9%, but after some time (10-15 minutes) I noticed that it's being seeded.

Are there any particular circumstances where you see it?

Only one. As I wrote before, if torrent is already partly-downloaded and I am marking some (not all) files, which wasn't downloaded before, to be downloaded, this problem affects much harder. If downloading single-file torrent (linux distro, for example) problem appears, but it affects slightly.

I take it it's not caused by lack of seeds, is it? Do you know if you're connected to any seeds and if they've unchoked you?

I take it it's not caused by lack of seeds, is it? There is a lot of seeds, nearly 50. Do you know if you're connected to any seeds Yes. During a normal download libtorrent makes a lot of connections. If speed falls down, a number of connections is falling too. At the end (~99.9%) there is usually only several connections (so the speed is horrible) or none (so the speed is zero). if they've unchoked you? No, qBittorrent doesn't shows it.

It doesn't sounds like a piece picker issue then, but rather something that makes the peers be disconnected. Do you see anything in the log about why peers are being disconnected?

Sorry for my delay. Where can I find libtorrent's log? qBittorrent's log is not helpful.

you have to build libtorrent with verbose logging enabled (that's an option in the Jamfile and configure script).

libtorrent will then produce quite verbose logs, every peer gets its own log and every message passed back and forth is logged. It also tells you why it disconnects peers.

http://forum.deluge-torrent.org/viewtopic.php?f=7&t=29545 Simular problem. But in my case torrents are much smaller, nearly 2-3 Gb.

Is it the case that the dropping download rate is the result of peers suddenly being banned once end-game is entered?

I've been trying to reproduce this kind of behavior without success. What are the characteristics of your download? for instance, total download rate, how evenly distributed is the download rate from your peers (is it mostly one fast peer, two fast peers, or half of the peers?). Are they pieces very large? (8 MB or up for instance). I take it there's no web server involved. Any other properties of the swarm and your connection you can think of?

What are the characteristics of your download? It's http://thepiratebay.se/torrent/6850782/Nightwish_-Imaginaerum%282011%29

total download rate At the middle of the download it's OK, 1,1 megabyte per second. At the end - nearly 100 bytes per second.

how evenly distributed is the download rate from your peers (is it mostly one fast peer, two fast peers, or half of the peers?) Normal situation, at the middle of the downloading: http://storage5.static.itmages.ru/i/12/0325/h_1332669795_7708569_2a81ccd6a5.png Speed rapidly felt: http://storage9.static.itmages.ru/i/12/0325/h_1332670021_7454314_15190efa9d.png Soon after this: http://storage2.static.itmages.ru/i/12/0325/h_1332670112_5545733_15522bfe6e.png Finally, it stalled: http://storage4.static.itmages.ru/i/12/0325/h_1332670182_8838735_e298a0b769.png A number of connections is ~60 all the time. I waited nearly 15 minutes, there was no progress at all, so I stopped a client. Here is log: http://ompldr.org/vZDUyOA/libtorrent_logs54100.7z . There is also a log of a previous attempt, I forgot to delete it, sorry.

I take it there's no web server involved. Yes, there are no web seeds. Any other properties of the swarm and your connection you can think of? I have a 10 mbit/s download speed, port is opened.

If there are no other users with this problem, don't pay much attention to this bug. It's not critical for me.

as far as I can tell, this particular swarm seems to have a lot of peers sending corrupt data. I see a similar amount of hash failures in other clients as well.

The corrupt data appears to be valid mp3 data but from the wrong place in the file. My guess would be that some people's audio players prepend an ID3 tag, shifting the file data out, making it not align to the original torrent anymore.

I've seen this problem too, it looks like torrents do not enter end-game mode if parts of the torrent have priority zero. See this line in policy.cpp:

    bool dont_pick_busy_blocks = (ses.m_settings.strict_end_game_mode
        && p.num_have() + int(p.get_download_queue().size())
            < t.torrent_file().num_pieces())
        || dq.size() + rq.size() > 0;

If a significant part of the torrent has priority zero, then num_pieces() will be much greater than num_have(). The torrent never enters end-game mode, so each block is requested only once. If there's one block remaining, and the peer is no good, libtorrent just sits and waits until it times out, and then retries.

ah, that makes perfect sense. This should be a simple fix. hopefully it will take care of it. thanks!

I've submitted a fix to this now, which will be in 0.16.2. Thanks.

Thanks for the quick response. However, although the fix helps in some cases, it isn't quite right. Specifically, end-game is still not being entered when downloading a subset of a torrent, after a different subset was already downloaded.

Suppose I have a torrent with 100 pieces, and I download just 10 of them. Then, at some later time, I restart it, but ask for a different 10 pieces. This second time around, num_filtered = 80, since it only counts the pieces that we don't have yet. So num_want = 100 - 80 = 20, which num_have will never approach.

It works for me if I change the num_want calculation to:

    int num_want = t.torrent_file().num_pieces() - p.num_filtered()
        - p.num_have_filtered();

Does that look reasonable?

thanks for testing. The num_filtered() and num_have_filtered() counters are supposed to be the number of pieces selected not to download, and num_have_filtered is the number of those pieces we already have, even though they're not selected for downloading (anymore).

The new logic I put in there is supposed to take the case you mention into account in the num_have counter, by subtracting the filtered pieces we have from it.

I don't think it's correct to subtract it from num_want as well, since those pieces have already been subtracted with "- p.num_filtered()" (num_filtered() is always greater than or equal to num_have_filtered()).

Is it possible that subtracting it again covers up a mistake elsewhere? say, the downloading pieces counter may be too low.

You of course know your codebase better than I do, but the comments in piece_picker.hpp suggest a different relationship between num_filtered() and num_have_filtered():

            // the number of filtered pieces we don't have
            int num_filtered() const { return m_num_filtered; }

            // the number of filtered pieces we already have
            int num_have_filtered() const { return m_num_have_filtered; }

If we've filtered some pieces, all of which we already have, num_filtered() would then be zero and num_have_filtered() greater than that.

Maybe the comments are out of date or something? Let me know if that's the case, and I'll try to figure out what's really going on.

you're right. Thanks for being persistent. I've added another function to the piece picker to further document this semantics and simplify the code a bit further. I'll put it in trunk and RC_0_16.

thanks again!

(and please let me know if I still screwed up somehow).

This is the patch btw:

Index: src/policy.cpp

--- src/policy.cpp (revision 7115) +++ src/policy.cpp (revision 7116) @@ -236,15 +236,8 @@ // and we're not strictly speaking in end-game mode yet // also, if we already have at least one outstanding

// request, we shouldn't pick any busy pieces either

• // the number of pieces we want (i.e. not including the filtered pieces)
• int num_want = t.torrent_file().num_pieces() - p.num_filtered();
• // the number of pieces we already have (not counting pieces that are filtered

• // that we might have still for some reason)

  - int num_have = p.num_have() - p.num_have_filtered();

  bool dont_pick_busy_blocks = (ses.m_settings.strict_end_game_mode
  

• && num_have + p.num_downloading_pieces() < num_want)

• && p.num_downloading_pieces() < p.num_want_left()) || dq.size() + rq.size() > 0;

  // this is filled with an interesting piece
  

  Index: include/libtorrent/piece_picker.hpp

--- include/libtorrent/piece_picker.hpp (revision 7115) +++ include/libtorrent/piece_picker.hpp (revision 7116) @@ -341,6 +341,9 @@

    int num_have() const { return m_num_have; }

• // the number of pieces we want and don't have
• int num_want_left() const { return num_pieces() - m_num_have - m_num_filtered; } +

  ifdef TORRENT_DEBUG

  // used in debug mode
  void verify_priority(int start, int end, int prio) const;
  

Looks good this time! I wish I was as good at keeping up with my issue trackers as you are, thanks for the quick fix.