[tin-bugs] Tin incorrectly parses comment blocks in CFWS blocks (per RFC5322 3.2.2)

[Stephen Hurd]

Sorry about the formatting, I'm between mail clients right now, and 
using Thunderbird.

On 2024-11-11 18:39, Nicholas Boel wrote:
> Either way, I've included Stephen, one of the developers of the server 
> software I'm using. As we were in a lengthy conversation last night 
> about this and I'm not much of a programmer, so I don't want any 
> confusion here while trying to pass on what he says.

Hey, just popping in to assert that:

From: "Some Body" (Cool Guy) <somebody at example.com>

Is legal but sketchy.  The " (Cool Guy) " (including the spaces before 
and after) is CFWS (Comment + Folding White Space) as allowed in the 
RFCs since 822. " (Cool Guy)" is a comment.

Section 3.2.2 in RFC-5322 covers this in detail and contains the CFWS 
ABNF grammar... the angle-addr grammar starts with optional CFWS, which 
makes it legal to put a comment (surrounded by parentheses) before 
there.  Most importantly, is the final paragraph which states "Runs of 
FWS, comment, or CFWS that occur between lexical tokens in a structured 
header field are semantically interpreted as a single space character."

The RFC does mention in 3.4 that comments SHOULD NOT be used in address 
fields (specifically to avoid confusing software that interprets 
comments), but it's strictly allowed by the standard.

Generally, unless you want to specifically keep the comments in for a 
specific reason, the best thing to do with comments it to just fold them 
with the rest of the field and replace the whole CFWS run with a single 
space at the same time lines are being unwrapped prior to parsing.

Basically, translate:
From: "Some Body" (Cool Guy) <somebody at example.com>

To:
From: "Some Body" <somebody at example.com>

What tin apparently does currently is changes it to:
From: "\"Some Body\" (Cool Guy)" <somebody at example.com>

Which places the CFWS inside the display-name.  This is very close to 
being the same as converting:
From: "Some Body"
     <somebody at example.com>

To:
From: "\"Some Body\"\r\n  " <somebody at example.com>

Horrifyingly, CFWS is used liberally in the grammar, so unfolding and 
stripping comments is an important part of parsing RFC-822 messages.