[tin-dev] Bug#832397: tin: fails after reconnection, losing all downloaded data

[Urs Janßen]

On Mon, Jul 25, 2016 at 03:48:57AM +0200, Vincent Lefevre wrote:
> Package: tin
> Version: 1:2.3.2-1
> Severity: important
> 
> On a large newsgroup, tin starts to download the headers, but at
> some point, there's a disconnection. And tin always fails after a
> reconnection. However, if I start tin again, the connection succeeds,
> showing the problem occurs only for a reconnection.
> 
> With "tin -D 1", here's how /tmp/NNTP looks like.
> 
> nntp_open() BEGIN
> nntp_open() news.gandi.net:119
> <<< [01:26:48.881604] 200 groups.gandi.net Papercut server ready (posting allowed)
> nntp_open() groups.gandi.net Papercut server ready (posting allowed)
[...]
> >>> [01:26:50.929886] LIST
> <<< [01:26:51.136585] 215 list of newsgroups follows
> nntp_command(LIST) OK
> <<< [01:26:51.136650] gandi.en.api 166 1 y
[...]
> <<< [01:26:51.136861] gandi.fr.domaine 8478 1 y
[...]
> nntp_command(LISTGROUP gandi.fr.domaine)
> >>> [01:26:54.579698] LISTGROUP gandi.fr.domaine
> <<< [01:26:54.935979] 211 8317 1 8478 gandi.fr.domaine Article numbers follow (multiline)
> nntp_command(LISTGROUP gandi.fr.domaine) OK
> setup_hard_base(LISTGROUP gandi.fr.domaine)
> nntp_command(XOVER 1-8478)
> >>> [01:26:54.980474] XOVER 1-8478
> <<< [01:27:06.850014] 224 Overview information follows
> nntp_command(XOVER 1-8478) OK

the XOVER response from the server is empty, thus tin tries XHDR XREF

> nntp_command(XHDR XREF 1-8478)
> >>> [01:27:06.850150] XHDR XREF 1-8478
> <<< [01:27:06.860971] 501 command syntax error (or un-implemented option)
> nntp_command(XHDR XREF 1-8478) NOT_OK

which fails so it uses HEAD (damned slow) instead

> nntp_command(HEAD 1)
> >>> [01:27:06.861101] HEAD 1
> <<< [01:27:06.920651] 221 1 <45ca0f06$0$22737$afc38c87 at groups.gandi.net> article retrieved - head follows
> nntp_command(HEAD 1) OK
> nntp_command(HEAD 2)
> >>> [01:27:06.920916] HEAD 2
> <<< [01:27:06.980840] 221 2 <45ca0f7a$0$22731$afc38c87 at groups.gandi.net> article retrieved - head follows
> nntp_command(HEAD 2) OK
[...]
> >>> [01:31:51.906590] HEAD 4702
> <<< [01:31:51.967943] 221 4702 <95d6f1cc7df4d5dac6dd6736fc53e047 at groups.gandi.net> article retrieved - head follows
> nntp_command(HEAD 4702) OK
> nntp_command(HEAD 4703)
> >>> [01:31:51.968080] HEAD 4703

and here the connection is closed by the server for whatever reason

(verified manually via:
~ > telnet groups.gandi.net 119
Trying 217.70.184.33...
Connected to groups.gandi.net.
Escape character is '^]'.
200 groups.gandi.net Papercut server ready (posting allowed)
GROUP gandi.fr.domaine
211 8317 1 8478 gandi.fr.domaine group selected
head 4703
Connection closed by foreign host.

so tin reconnects and retires the last HEAD again:

> >>> [01:31:56.123696] GROUP gandi.fr.domaine
> <<< [01:31:56.252672] 211 8317 1 8478 gandi.fr.domaine group selected
> 
> >>> [01:31:56.252717] HEAD 4703

till it reaches its reconnection limit, and then quits with (undocumented)
NNTP_ERROR_EXIT (3)

> And then tin quits with no error messages, and exit status 3, which is

there should have been a "NNTP connection error. Exiting..."
message (shown for 2 seconds but then [n]curses is switched off ...).
I've added a log message to the NNTP-log when started with -D 1 now:
"reconnect(3) limit 2 reached, giving up".

> If I restart tin and retry on the same newsgroup, it starts again
> to download the headers from 1! Thus it is impossible to read this
> newsgroup.

you can use "-G positive_number" to download only the last n articles
(i.e. tin -G 2500 -g groups.gandi.net) - but with that buggy server there is
currently no way to read all articles (except 4703).

HTH,
urs
-- 
"Only whimps use tape backup: _real_ men just upload their important stuff
 on ftp, and let the rest of the world mirror it ;)" - Linus