|
Startseite · Das Institut · ChemNet · Fachinformationen · Internet · Index · Benutzerseiten |
|
|
Startseite · Das Institut · ChemNet · Fachinformationen · Internet · Index · Benutzerseiten |
This document is intended to become an Internet Draft. Internet Drafts are working documents of the Internet Engineering Task Force (IETF), its Areas, and its Working Groups. Note that other groups may also distribute working documents as Internet Drafts.
Internet Drafts are draft documents valid for a maximum of six months. Internet Drafts may be updated, replaced, or obsoleted by other documents at any time. It is not appropriate to use Internet Drafts as reference material or to cite them other than as a "working draft" or "work in progress".
Please check the I-D abstract listing contained in each Internet Draft directory to learn the current status of this or any other Internet Draft. (Actually, this draft is at too early a stage to even be listed there yet.)
It is hoped that a later version of this Draft will obsolete RFC 1036 and will become an Internet standard.
References to the "successor to this Draft" refer not to later versions of this draft, but to a hypothetical future rewrite of this Draft (in the same way that this Draft is a rewrite of RFC 1036 ).
Distribution of this memo is unlimited.
This Draft defines the format and procedures for interchange of network news articles. It is hoped that a later version of this Draft will obsolete RFC 1036 , reflecting more recent experience and accommodating future directions.
Network news articles resemble mail messages but are broadcast to potentially-large audiences, using a flooding algorithm that propagates one copy to each interested host (or group thereof), typically stores only one copy per host, and does not require any central administration or systematic registration of interested users. Network news originated as the medium of communication for Usenet, circa 1980. Since then Usenet has grown explosively, and many Internet sites participate in it. In addition, the news technology is now in widespread use for other purposes, on the Internet and elsewhere.
This Draft primarily codifies and organizes existing practice. A few small extensions have been added in an attempt to solve problems that are considered serious. Major extensions (e.g. cryptographic authentication) that need significant development effort are left to be undertaken as independent efforts.
Network news articles resemble mail messages but are broadcast to potentially-large audiences, using a flooding algorithm that propagates one copy to each interested host (or groups thereof), typically stores only one copy per host, and does not require any central administration or systematic registration of interested users. Network news originated as the medium of communication for Usenet, circa 1980. Since then Usenet has grown explosively, and many Internet sites participate in it. In addition, the news technology is now in widespread use for other purposes, on the Internet and elsewhere.
The earliest news interchange used the so-called "A News" article format. Shortly thereafter, an article format vaguely resembling Internet mail was devised and used briefly. Both of those formats are completely obsolete; they are documented in appendix A for historical reasons only. With publication of RFC 850 [rrr] in 1983, news articles came to closely resemble Internet mail messages, with some restrictions and some additional headers. RFC 1036 [rrr] in 1987 updated RFC 850 without making major changes.
In the intervening five years, the RFC 1036 article format has proven quite satisfactory, although minor extensions appear desirable to match recent developments in areas such as multi-media mail. RFC 1036 itself has not proven quite so satisfactory. It is often rather vague and does not address some issues at all; this has caused significant interoperability problems at times, and implementations have diverged somewhat. Worse, although it was intended primarily to document existing practice, it did not precisely match existing practice even at the time it was published, and the deviations have grown since.
This Draft attempts to specify the format of articles, and the procedures used to exchange them and process them, in sufficient detail to allow full interoperability. In addition, some tentative suggestions are made about directions for future development, in an attempt to avert unnecessary divergence and consequent loss of interoperability. Major extensions (e.g. cryptographic authentication) that need significant development effort are left to be undertaken as independent efforts.
As in this Draft's predecessors, the exact means used to transmit articles from one host to another is not specified. NNTP [rrr] is probably the most common transmission method on the Internet, but a number of others are known to be in use, including the UUCP protocol [rrr] extensively used in the early days of Usenet and still much used on its fringes today.
Several of the mechanisms described in this Draft may seem somewhat strange or even bizarre at first reading. As with Internet mail, there is no reasonable possibility of updating the entire installed base of news software promptly, so interoperability with old software is crucial and will remain so. Compatibility with existing practice and robustness in an imperfect world necessarily take priority over elegance.
Throughout this Draft, "MAIL" is short for " RFC 822 [rrr] as amended by RFC 1123 [rrr]". ( RFC 1123 's amendments are mostly relatively small, but they are not insignificant.) See also the discussion in section 3 about this Draft's relationship to MAIL. "MIME" is short for " RFCs 2045 , 2046 , 2047 , 2048 , and 2049 " (or their updated replacements).
"ASCII" is short for "the ANSI X3.4 character set" [rrr]. While "ASCII" is often misused to refer to various character sets somewhat similar to X3.4, in this Draft, "ASCII" means X3.4 and only X3.4.
Certain words used to define the significance of individual requirements are capitalized. "MUST" means that the item is an absolute requirement of the specification. "SHOULD" means that the item is a strong recommendation: there may be valid reasons to ignore it in unusual circumstances, but this should be done only after careful study of the full implications and a firm conclusion that it is necessary, because there are serious disadvantages to doing so. "MAY" means that the item is truly optional, and implementors and users are warned that conformance is possible but not to be relied on.
The term "compliant", applied to implementations etc., indicates satisfaction of all relevant "MUST" and "SHOULD" requirements. The term "conditionally compliant" indicates satisfaction of all relevant "MUST" requirements but violation of at least one relevant "SHOULD" requirement.
This Draft contains explanatory notes using the following format. These may be skipped by persons interested solely in the content of the specification. The purpose of the notes is to explain why choices were made, to place them in context, or to suggest possible implementation techniques.
All numeric values are given in decimal unless otherwise indicated. Octets are assumed to be unsigned values for this purpose. Large numbers are written using the North American convention, in which "," separates groups of three digits but otherwise has no significance.
Although the mechanisms specified in this Draft are all described in prose, most are also described formally in the modified BNF notation of RFC 822 . Implementors will need to be familiar with this notation to fully understand this specification, and are referred to RFC 822 for a complete explanation of the modified BNF notation. Here is a brief illustrative example:
sentence = clause *( punct clause ) "."
punct = ":" / ";"
clause = 1*word [ "(" clause ")" / "," 1*word ]
word = <any English word>
This defines a sentence as some clauses separated by puncts and ended by a period, a punct as a colon or semicolon, a clause as at least one <word> optionally followed by either a parenthesized clause or a comma and at least one more <word>, and a <word> as (informally) any English word. <> are used to enclose names when (and only when) distinguishing them from surrounding text is useful. The full form of the repetition notation is <m>"*"<n><thing>, denoting <m> through <n> repetitions of <thing>; <m> defaults to zero, <n> to infinity, and the "*" and <n> can be omitted if <m> and <n> are equal, so 1*word is one or more words, 1*5word is one through five words, and 2word is exactly two words.
The character "\" is not special in any way in this notation.
This Draft is intended to be self-contained; all syntax rules used in it are defined within it, and a rule with the same name as one found in MAIL does not necessarily have the same definition. The lexical layer of MAIL is NOT, repeat NOT, used in this Draft, and its presence must not be assumed; notably, this Draft spells out all places where white space is permitted/required and all places where constructs resembling MAIL comments can occur.
The term "character set", wherever it is used in this Draft, refers to a coded character set, in the sense of ISO character set standardization work, and must not be misinterpreted as meaning merely "a set of characters".
In this Draft, ASCII character 32 is referred to as "blank"; the word "space" has a more generic meaning.
An "article" is the unit of news, analogous to a MAIL "message".
A "poster" is a human being (or software equivalent) submitting a possibly-compliant article to be "posted": made available for reading on all relevant hosts. A "posting agent" is software that assists posters to prepare articles, including determining whether the final article is compliant, passing it on to a relayer for posting if so, and returning it to the poster with an explanation if not. A "relayer" is software which receives allegedly-compliant articles from posting agents and/or other relayers, files copies in a "news database", and possibly passes copies on to other relayers.
A "reader" is a human being reading news articles. A "reading agent" is software which presents articles to a reader.
A "newsgroup" is a single news forum, a logical bulletin board, having a name and nominally intended for articles on a specific topic. An article is "posted to" a single newsgroup or several newsgroups. When an article is posted to more than one newsgroup, it is said to be "cross-posted"; note that this differs from posting the same text as part of each of several articles, one per newsgroup. A "hierarchy" is the set of all newsgroups whose names share a first component (see the name syntax in section 5.5 ).
A newsgroup may be "moderated", in which case submissions are not posted directly, but mailed to a "moderator" for consideration and possible posting. Moderators are typically human but may be implemented partially or entirely in software.
A "followup" is an article containing a response to the contents of an earlier article (the followup's "precursor"). A "followup agent" is a combination of reading agent and posting agent that aids in the preparation and posting of a followup.
Text comparisons are "case-sensitive" if they consider uppercase letters (e.g. "A") different from lowercase letters (e.g. "a"), and "case-insensitive" if letters differing only in case (e.g. "A" and "a") are considered identical. Categories of text are said to be case-(in)sensitive if comparisons of such texts to others are case-(in)sensitive.
A "cooperating subnet" is a set of news-exchanging hosts which is sufficiently well-coordinated (typically via a central administration of some sort) that stronger assumptions can be made about hosts in the set than about news hosts in general. This is typically used to relax restrictions which are otherwise required for worst-case interoperability; members of a cooperating subnet MAY interchange articles that do not conform to this Draft's specifications, provided all members have agreed to this and provided the articles are not permitted to leak out of the subnet. The word "subnet" is used to emphasize that a cooperating subnet is typically not an isolated universe; care must be taken that traffic leaving the subnet complies with the restrictions of the larger net, not just those of the cooperating subnet.
A "message ID" is a unique identifier for an article, usually supplied by the posting agent which posted it. It distinguishes the article from every other article ever posted anywhere (in theory). Articles with the same message ID are treated as identical copies of the same article even if they are not in fact identical.
A "gateway" is software which receives news articles and converts them to messages of some other kind (e.g. mail to a mailing list), or vice-versa; in essence it is a translating relayer that straddles boundaries between different methods of message exchange. The most common type of gateway connects newsgroup(s) to mailing list(s), either unidirectionally or bidirectionally, but there are also gateways between news networks using this Draft's news format and those using other formats.
A "control message" is an article which is marked as containing control information; a relayer receiving such an article will (subject to permissions etc.) take actions beyond just filing and passing on the article.
An article's "reply address" is the address to which mailed replies should be sent. This is the address specified in the article's From header (see section 5.2 ), unless it also has a Reply-To header (see section 6.3 ).
The notation (e.g.) "(ASCII 17)" following a name means "this name refers to the ASCII character having value 17". An "ASCII printable character" is an ASCII character in the range 33-126. An "ASCII control character" is an ASCII character in the range 0-31, or the character DEL (ASCII 127). A "non-ASCII character" is a character having a value exceeding 127.
How the end of a text line is represented depends on the context and the implementation. For Internet transmission via protocols such as SMTP [rrr], an end-of-line is a CR (ASCII 13) followed by an LF (ASCII 10). ISO C [rrr] and many modern operating systems indicate end-of-line with a single character, typically ASCII LF (aka "newline"), and this is the normal convention when news is transmitted via UUCP. A variety of other methods are in use, including outof-band methods in which there is no specific character that means end-of-line.
This Draft does not constrain how end-of-line is represented in news, except that characters other than CR and LF MUST not be usurped for use in end-of-line representations. Also, obviously, all software dealing with a particular copy of an article must agree on the convention to be used. "EOL" is used to mean "whatever end-of-line representation is appropriate"; it is not necessarily a character or sequence of characters.
Text in newsgroup names, header parameters, etc. is casesensitive unless stated otherwise.
Various constant strings in this Draft, such as header names and month names, are derived from English words. Despite their derivation, these words do NOT change when the poster or reader employing them is interacting in a language other than English. Posting and reading agents SHOULD translate as appropriate in their interaction with the poster or reader, but the forms that actually appear in articles are always the English-derived ones defined in this Draft.
The primary intent of this Draft is to completely describe the news article format as a subset of MAIL's message format augmented by some new headers. Unless explicitly noted otherwise, the intent throughout is that an article MUST also be a valid MAIL message.
Given that this Draft attempts to be self-contained, it inevitably contains considerable repetition of information found in MAIL. This raises the possibility of unintentional conflicts. Unless specifically noted otherwise, any wording in this Draft which permits behavior that is not MAILcompliant is erroneous and should be followed only to the extent that the result remains compliant with MAIL.
Implementors and users should note that MAIL is deliberately an extensible standard, and most extensions devised for mail are also relevant to (and compatible with) news. Note particularly MIME [rrr], summarized briefly in appendix B, which extends MAIL in a number of useful ways that are definitely relevant to news. Also of note is the work in progress on reconciling PEM (Privacy Enhanced Mail, which defines extensions for authentication and security) with MIME, after which this may also be relevant to news.
Similarly, descriptions here of MIME facilities should be considered correct only to the extent that they do not require or legitimize practices that would violate those RFCs. (Note that this Draft does extend the application of some MIME facilities, but this is an extension rather than an alteration.)
The overall syntax of a news article is:
article = 1*header separator body
header = start-line *continuation
start-line = header-name ":" space [ nonblank-text ] eol
continuation = space nonblank-text eol
header-name = 1*name-character *( "-" 1*name-character )
name-character = letter / digit
letter = <ASCII letter A-Z or a-z>
digit = <ASCII digit 0-9>
separator = eol
body = *( [ nonblank-text / space ] eol )
eol = <EOL>
nonblank-text = [ space ] text-character *( space-or-text )
text-character = <any ASCII character except NUL (ASCII 0),
HT (ASCII 9), LF (ASCII 10), CR (ASCII 13),
or blank (ASCII 32)>
space = 1*( <HT (ASCII 9)> / <blank (ASCII 32)> )
space-or-text = space / text-character
An article consists of some headers followed by a body. An empty line separates the two. The headers contain structured information about the article and its transmission. A header begins with a header name identifying it, and can be continued onto subsequent lines by beginning the continuation line(s) with white space. (Note that section 4.2.3 adds some restrictions to the header syntax indicated here.) The body is largely-unstructured text significant only to the poster and the readers.
Note that the separator line must be truly empty, not just a line containing white space. Further empty lines following it are part of the body, as are empty lines at the end of the article.
Implementors are warned that trailing white space, whether alone on the line or not, MAY be significant in the body, notably in early versions of the "uuencode" encoding for binary data. Trailing white space MUST be preserved unless the article is known to have originated within a cooperating subnet that avoids using significant trailing white space, and SHOULD be preserved regardless. Posters SHOULD avoid using conventions or encodings which make trailing white space significant; for encoding of binary data, MIME's "base64" encoding is recommended. Implementors are warned that ISO C implementations are not required to preserve trailing white space, and special precautions may be necessary in implementations which do not.
Posters are warned that some very old relayer software misbehaves when the first non-empty line of an article body begins with white space.
Despite the restrictions on header-name syntax imposed by the grammar, relayers and reading agents SHOULD tolerate header names containing any ASCII printable character other than colon (":", ASCII 58).
Relayers MUST disregard headers not described in this Draft (that is, with header names not mentioned in this Draft), and pass them on unaltered.
Posters wishing to convey non-standard information in headers SHOULD use header names beginning with "X-". No standard header name will ever be of this form. Reading agents SHOULD ignore "X-" headers, or at least treat them with great care.
The order of headers in an article is not significant. However, posting agents are encouraged to put mandatory headers (see section 5 ) first, followed by optional headers (see section 6 ), followed by headers not defined in this Draft.
Header names are case-insensitive. There is a preferred case convention, which posters and posting agents SHOULD use: each hyphen-separated "word" has its initial letter (if any) in uppercase and the rest in lowercase, except that some abbreviations have all letters uppercase (e.g. "Message-ID" and "MIME-Version"). The forms used in this Draft are the preferred forms for the headers described herein. Relayers and reading agents are warned that articles might not obey this convention.
In general, a header can consist of several lines, with each continuation line beginning with white space. The EOLs preceding continuation lines are ignored when processing such a header, effectively combining the start-line and the continuations into a single logical line. The logical line, less the header name, colon, and any white space following the colon, is the "header content".
A header whose content is empty is said to be an empty header. Relayers and reading agents SHOULD not consider presence or absence of an empty header to alter the semantics of an article (although syntactic rules, such as requirements that certain header names appear at most once in an article, MUST still be satisfied). Posting agents SHOULD delete empty headers from articles before posting them.
Headers that merely state defaults explicitly (e.g., a Followup-To header with the same content as the Newsgroups header, or a MIME Content-Type header with contents "text/plain; charset=us-ascii") or state information that reading agents can typically determine easily themselves (e.g. the length of the body in octets) are redundant, conveying no information whatsoever. Headers that state information which cannot possibly be of use to a significant number of relayers, reading agents, or readers (e.g., the name of the software package used as the posting agent) are useless and pointless. Posters and posting agents SHOULD avoid including redundant or useless headers in articles.
The colon following the header name on the start-line MUST be followed by white space, even if the header is empty. If the header is not empty, at least some of the content MUST appear on the start-line. Posting agents MUST enforce these restrictions, but relayers (etc.) SHOULD accept even articles that violate them.
In general, posters and posting agents SHOULD use blank (ASCII 32), not tab (ASCII 9), where white space is desired in headers. Existing software does not consistently accept tab as synonymous with blank in all contexts. In particular, RFC 1036 appeared to specify that the character immediately following the colon after a header name was required to be a blank, and some news software insists on that, so this character MUST be a blank. Again, posting agents MUST enforce these restrictions but relayers SHOULD be more tolerant.
Since the white space beginning a continuation line remains a part of the logical line, headers can be "broken" into multiple lines only at white space. Posting agents SHOULD not break headers unnecessarily. Relayers SHOULD preserve existing header breaks, and SHOULD not introduce new breaks. Breaking headers SHOULD be a last resort; relayers and reading agents SHOULD handle long header lines gracefully. (See the discussion of size limits in section 4.6 .)
Although the article body is unstructured for most of the purposes of this Draft, structure MAY be imposed on it by other means, notably MIME headers (see appendix B).
The body of an article MAY be empty, although posting agents SHOULD consider this an error condition (meriting returning the article to the poster for revision). A posting agent which does not reject such an article SHOULD issue a warning message to the poster and supply a non-empty body. Note that the separator line MUST be present even if the body is empty.
Note that an article body is a sequence of lines terminated by EOLs, not arbitrary binary data, and in particular it MUST end with an EOL. However, relayers SHOULD treat the body of an article as an uninterpreted sequence of octets (except as mandated by changes of EOL representation and by control-message processing) and SHOULD avoid imposing constraints on it. See also section 4.6 .
Although body lines can in principle be very long (see section 4.6 for some discussion of length limits), posters SHOULD restrict body line lengths to circa 70-75 characters. On systems where text is conventionally stored with EOLs only at paragraph breaks and other "hard return" points, with software breaking lines as appropriate for display or manipulation, posting agents SHOULD insert EOLs as necessary so that posted articles comply with this restriction.
Reading agents confronted with body lines much longer than the available output-device width SHOULD break lines as appropriate. Posters are warned that such breaks may not occur exactly where the poster intends.
Although styles vary widely, for plain text it is usual to use no left margin, leave the right edge ragged, use a single empty line to separate paragraphs, and employ normal natural-language usage on matters such as upper/lowercase. (In particular, articles SHOULD not be written entirely in uppercase. In environments where posters have access only to uppercase, posting agents SHOULD translate it to lowercase.)
Tone of voice does not carry well in written text, and misunderstandings are common when sarcasm, parody, or exaggeration for humorous effect is attempted without explicit warning. It has become conventional to use the sequence ":-)", which (on most output devices) resembles a rotated "smiley face" symbol, as a marker for text not meant to be taken literally, especially when humor is intended. This practice aids communication and averts unintended ill-will; posters are urged to use it. A variety of analogous sequences are used with less-standardized meanings [Sanderson].
The order of arrival of news articles at a particular host depends somewhat on transmission paths, and occasionally articles are lost for various reasons. When responding to a previous article, posters SHOULD not assume that all readers understand the exact context. It is common to quote some of the previous article to establish context. This SHOULD be done by prefacing each quoted line (even if it is empty) with the character ">". This will result in multiple levels of ">" when quoted context itself contains quoted context.
Readability is enhanced if quoted text and new text are separated by an empty line.
Posters SHOULD edit quoted context to trim it down to the minimum necessary. However, posting agents SHOULD not attempt to enforce this by imposing overly-simplistic rules like "no more than 50% of the lines should be quotes".
Some followup agents supply "attribution" lines for quoted context, indicating where it first appeared and under whose name. When multiple levels of quoting are present and quoted context is edited for brevity, "inner" attribution lines are not always retained. The editing process is also somewhat error-prone. Reading agents (and readers) are warned not to assume that attributions are accurate.
Early difficulties in inferring return addresses from article headers led to "signatures": short closing texts, automatically added to the end of articles by posting agents, identifying the poster and giving his network addresses etc. If a poster or posting agent does append a signature to an article, the signature SHOULD be preceded with a delimiter line containing (only) two hyphens (ASCII 45) followed by one blank (ASCII 32). Posting agents SHOULD limit the length of signatures, since verbose excess bordering on abuse is common if no restraint is imposed; 4 lines is a common limit.
Header and body lines MAY contain any ASCII characters other than CR (ASCII 13), LF (ASCII 10), and NUL (ASCII 0).
However, posters SHOULD avoid using ASCII control characters except for tab (ASCII 9), formfeed (ASCII 12), and backspace (ASCII 8). Tab signifies sufficient horizontal white space to reach the next of a set of fixed positions; posters are warned that there is no standard set of positions, so tabs should be avoided if precise spacing is essential. Formfeed signifies a point at which a reading agent SHOULD pause and await reader interaction before displaying further text . Backspace SHOULD be used only for underlining, done by a sequence of underscores (ASCII 95) followed by an equal number of backspaces, signifying that the same number of text characters following are to be underlined. Posters are warned that underlining is not available on all output devices and is best not relied on for essential meaning. Reading agents SHOULD recognize underlining and translate it to the appropriate commands for devices that support it.
Cooperating subnets which wish to employ non-ASCII character sets by using escape sequences (employing, e.g., ESC (ASCII 27), SO (ASCII 14), and SI (ASCII 15)) to alter the meaning of superficially-ASCII characters MAY do so, but MUST use MIME headers to alert reading agents to the particular character set(s) and escape sequences in use. A reading agent SHOULD not pass such an escape sequence through, unaltered, to the output device unless the agent confirms that the sequence is one used to affect character sets and has reason to believe that the device is capable of interpreting that particular sequence properly.
Articles MUST not contain any octet with value exceeding 127, i.e. any octet that is not an ASCII character.
In anticipation of the day when it is possible to use nonASCII characters safely anywhere, and to provide for the (substantial) cooperating subnets that are already using them, transmission paths SHOULD treat news articles as uninterpreted sequences of octets (except perhaps for transformations between EOL representations) and relayers SHOULD treat non-ASCII characters in articles as ordinary characters.
Except where cooperating subnets permit more direct approaches, MIME [rrr] headers and encodings SHOULD be used to transmit non-ASCII content using ASCII characters; see section 4.5 , appendix B, and the MIME RFCs for details. If article content can be expressed in ASCII, it SHOULD be. Failing that, the order of preference for character sets is that described in MIME [rrr].
Articles containing non-ASCII characters, articles using ASCII characters (values 0 through 127) to refer to nonASCII symbols, and articles using escape sequences to shift character sets SHOULD include MIME headers indicating which character set(s) and conventions are being used, and MUST do so unless such articles are strictly confined to a cooperating subnet which has its own pre-agreed conventions. MIME encodings are preferred over all these techniques. If it comes to a relayer's attention that it is being asked to pass an article using such techniques outward across what it knows to be the boundary of such a cooperating subnet, it MUST report this error to its administrator, and MAY refuse to pass the article beyond the subnet boundary. If it does pass the article, it MUST re-encode it with MIME encodings to make it conform to this Draft.
Reading agents SHOULD note MIME headers and attempt to show the reader the closest possible approximation to the intended content. They SHOULD not just send the octets of the article to the output device unaltered, unless there is reason to believe that the output device will indeed interpret them correctly. Reading agents MUST not pass ASCII control characters or escape sequences, other than as discussed above, unaltered to the output device; only by chance would the result be the desired one, and there is serious potential for harmful side effects, either accidental or malicious.
Followup agents MUST be careful to apply appropriate transformations of representation to the outbound followup as well as the inbound precursor. A followup to an article containing non-ASCII material is very likely to contain nonASCII material itself.
All octets found in headers MUST be ASCII characters. However, it is desirable to have a way of encoding non-ASCII characters, especially in "human-readable" headers such as Subject. MIME [rrr] provides a way to do this. Full details may be found in the MIME specifications; herewith a quick summary to alert software authors to the issues...
encoded-word = "=?" charset "?" encoding "?" codes "?=" charset = 1*tag-char encoding = 1*tag-char tag-char = <ASCII printable character except !()<>@,;:\"[]/?=> codes = 1*code-char code-char = <ASCII printable character except ?>
An encoded word is a sequence of ASCII printable characters that specifies the character set, encoding method, and bits of (potentially) non-ASCII characters. Encoded words are allowed only in certain positions in certain headers. Specific headers impose restrictions on the content of encoded words beyond that specified in this section . Posting agents MUST ensure that any material resembling an encoded word (complete with all delimiters), in a context where encoded words may appear, really is an encoded word.
An encoded word MUST not be more than 75 octets long. Each line of a header containing encoded word(s) MUST be at most 76 octets long, not counting the EOL.
The details of charsets and encodings are defined by MIME [rrr]; the sequence of preferred character sets is the same as MIME's. Encoded words SHOULD not be used for content expressible in ASCII.
When an encoded word is used, other than in a newsgroup name (see section 5.5 ), it MUST be separated from any adjacent non-space characters (including other encoded words) by white space. Reading agents displaying the contents of encoded words (as opposed to their encoded form) should ignore white space adjacent to encoded words.
Reading-agent implementors are warned that although this Draft completely specifies where encoded words may appear in the headers it defines, there are other headers (e.g. the MIME Content-Description header) that MAY contain them.
Implementations SHOULD avoid fixed constraints on the sizes of lines within an article and on the size of the entire article.
Relayers SHOULD treat the body of an article as an uninterpreted sequence of octets (except as mandated by changes of EOL representation and processing of control messages), not to be altered or constrained in any way.
If it is absolutely necessary for an implementation to impose a limit on the length of header lines, body lines, or header logical lines, that limit shall be at least 1000 octets, including EOL representations. Relayers and transmission paths confronted with lines beyond their internal limits (if any) MUST not simply inject EOLs at random places; they MAY break headers (as described in 4.2.3) as a last resort, and otherwise they MUST either pass the long lines through unaltered, or refuse to pass the article at all (see section 9.1 for further discussion).
All implementations MUST be able to handle an article totalling at least 65,000 octets, including headers and EOL representations, gracefully and efficiently. All implementations SHOULD be able to handle an article totalling at least 1,000,000 (one million) octets, including headers and EOL representations, gracefully and efficiently. "Gracefully and efficiently" is intended to preclude not only failures, but also major loss of performance, serious problems in error recovery, or resource consumption beyond what is reasonably necessary.
Posters SHOULD limit posted articles to at most 60,000 octets, including headers and EOL representations, unless the articles are being posted only within a cooperating subnet which is known to be capable of handling larger articles gracefully. Posting agents presented with a large article SHOULD warn the poster and request confirmation.
Posters using non-ASCII characters in their text MUST take into account the overhead involved in MIME encoding, unless the article's propagation will be entirely limited to a cooperating subnet which does not use MIME encodings for non-ASCII characters. For example, MIME base64 encoding involves growth by a factor of approximately 4/3, so an article which would likely have to use this encoding should be at most about 45,000 octets before encoding.
Posters SHOULD use MIME "message/partial" conventions to facilitate automatic reassembly of a large document split into smaller pieces for posting. It is recommended that the content identifier used should be a message ID, generated by the same means as article message IDs (see section 5.3 ), and that all parts should have a See-Also header (see section 6.16) giving the message IDs of at least the previous parts and preferably all the parts.
To repeat: implementations SHOULD avoid fixed constraints on the sizes of lines within an article and on the size of the entire article.
Here is a sample article:
From: jerry@eagle.ATT.COM (Jerry Schwarz) Path: cbosgd!mhuxj!mhuxt!eagle!jerry Newsgroups: news.announce Subject: Usenet Etiquette -- Please Read Message-ID: <642@eagle.ATT.COM> Date: Mon, 17 Jan 1994 11:14:55 -0500 (EST) Followup-To: news.misc Expires: Wed, 19 Jan 1994 00:00:00 -0500 Organization: AT&T Bell Laboratories, Murray Hill
body body body
An article MUST have one, and only one, of each of the following headers: Date, From, Message-ID, Subject, Newsgroups, Path.
Note also that there are situations, discussed in the relevant parts of section 6 , where References, Sender, or Approved headers are mandatory.
In the discussions of the individual headers, the content of each is specified using the syntax notation. The convention used is that the content of, for example, the Subject header is defined as <Subject-content>.
The Date header contains the date and time when the article was submitted for transmission:
Date-content = [ weekday "," space ] date space time
weekday = "Mon" / "Tue" / "Wed" / "Thu"
/ "Fri" / "Sat" / "Sun"
date = day space month space year
day = 1*2digit
month = "Jan" / "Feb" / "Mar" / "Apr" / "May" / "Jun"
/ "Jul" / "Aug" / "Sep" / "Oct" / "Nov" / "Dec"
year = 4digit / 2digit
time = hh ":" mm [ ":" ss ] space timezone
timezone = "UT" / "GMT"
/ ( "+" / "-" ) hh mm [ space "(" zone-name ")" ]
hh = 2digit
mm = 2digit
ss = 2digit
zone-name = 1*( <ASCII printable character except ()\> / space )
This is a restricted subset of the MAIL date format.
If a weekday is given, it MUST be consistent with the date. The modern Gregorian calendar is used, and dates MUST be consistent with its usual conventions; for example, if the month is May, the day must be between 1 and 31 inclusive. The year SHOULD be given as four digits, and posting agents SHOULD enforce this; however, relayers MUST accept the twodigit form, and MUST interpret it as having the implicit prefix "19".
The time is given on the 24-hour clock, e.g. two hours before midnight is "22:00" or "22:00:00". The hh must be between 00 and 23 inclusive, the mm between 0 and 59 inclusive, and the ss between 0 and 61 inclusive.
The date and time SHOULD be given in the poster's local timezone, including a specification of that timezone as a numeric offset (which SHOULD include the timezone name, e.g. "EST", supplied in parentheses like a MAIL comment). If not, they MUST be given in Universal Time (abbreviated "UT"; "GMT" is a historical synonym for "UT"). The timezone name in parentheses, if present, is a comment; software MUST ignore it, except that reading agents might wish to display it to the reader. Timezone names other than "UT" and "GMT" MUST appear only in the comment.
The From header contains the electronic address, and possibly the full name, of the article's author:
From-content = address [ space "(" paren-phrase ")" ]
/ [ plain-phrase space ] "<" address ">"
paren-phrase = 1*( paren-char / space / encoded-word )
paren-char = <ASCII printable character except ()<>\>
plain-phrase = plain-word *( space plain-word )
plain-word = unquoted-word / quoted-word / encoded-word
unquoted-word = 1*unquoted-char
unquoted-char = <ASCII printable character except !()<>@,;:\".[]>
quoted-word = quote 1*( quoted-char / space ) quote
quote = <" (ASCII 34)>
quoted-char = <ASCII printable character except "()<>\>
address = local-part "@" domain
local-part = unquoted-word *( "." unquoted-word )
domain = unquoted-word *( "." unquoted-word )
(Encoded words are described in section 4.5 .) The full name is distinguished from the electronic address either by enclosing the former in parentheses (making it resemble a MAIL comment, after the address) or by enclosing the latter in angle brackets. The second form is preferred. In the first form, encoded words inside the full name MUST be composed entirely of <paren-char>s. In the second form, encoded words inside the full name may not contain characters other than letters (of either case), digits, and the characters "!", "*", "+", "-", "/", "=", and "_". The local part is case-sensitive (except that all case counterparts of "postmaster" are deemed equivalent), the domain is caseinsensitive, and all other parts of the From content are comments which MUST be ignored by news software (except insofar as reading agents may wish to display them to the reader). Posters and posting agents MUST restrict themselves to this subset of the MAIL From syntax; relayers MAY accept a broader subset, but see the discussion in section 9.1 .
The address SHOULD be a valid and complete Internet domain address, capable of being successfully mailed to by an Internet host (possibly via an MX record and a forwarder). The pseudo-domain ".uucp" MAY be used for hosts registered in the UUCP maps (e.g. name "xyz.uucp" for registered site "xyz"), but such hosts SHOULD discontinue this usage (either by arranging a proper Internet address and forwarder, or by using the "% hack" (see below)), as soon as possible. Bitnet hosts SHOULD use Internet addresses, avoiding the obsolescent ".bitnet" pseudo-domain. Other forms of address MUST not be used.
If it is necessary to use the local part to specify a routing relative to the nearest Internet host, this MUST be done using the "% hack", using "%" as a secondary "@". For example, to specify that mail to the address should go to Internet host "foo.bar.edu", then to non-Internet host "ein", then to non-Internet host "deux", for delivery there to mailbox "fred", a suitable address would be:
Analogous forms using "!" in the local part MUST not be used, as they are ambiguous; they should be expressed in the "%" form.
Relayers MUST not, repeat MUST not, repeat MUST not, rewrite From lines, in any way, however minor or innocent-seeming. Trying to "fix" a non-conforming address has a very high probability of making things worse. Either pass it along unchanged, or reject the article.
Posters and posting agents SHOULD avoid use of the characters "!" and "@" in full names, as they may trigger unwanted header rewriting by old, simple-minded news software.
From: "John W. Campbell, Jr." <editor@analog.com>
The Message-ID header contains the article's message ID, a unique identifier distinguishing the article from every other article:
Message-ID-content = message-id message-id = "<" local-part "@" domain ">"
As with From addresses, a message ID's local part is casesensitive and its domain is case-insensitive. The "<" and ">" are parts of the message ID, not peculiarities of the Message-ID header.
The domain in the message ID SHOULD be the full Internet domain name of the posting agent's host. Use of the ".uucp" pseudo-domain (for hosts registered in the UUCP maps) or the ".bitnet" pseudo-domain (for Bitnet hosts) is permissible, but SHOULD be avoided.
Posters and posting agents MUST generate the local part of a message ID using an algorithm which obeys the specified syntax (words separated by ".", with certain characters not permitted) (see section 5.2 for details), and will not repeat itself (ever). The algorithm SHOULD not generate message IDs which differ only in case of letters. Note the specification in section 6.5 of a recommended convention for indicating subject changes. Otherwise the algorithm is up to the implementor.
The local part of a message ID MUST not be "postmaster" or any other string that would compare equal to "postmaster" in a case-insensitive comparison. Message IDs MUST be no longer than 250 octets, including the "<" and ">".
The Subject header's content (the "subject" of the article) is a short phrase describing the topic of the article:
Subject-content = [ "Re: " ] nonblank-text
Encoded words MAY appear in this header.
If the article is a followup, the subject SHOULD begin with "Re: " (a "back reference"). If the article is not a followup, the subject MUST not begin with a back reference. Back references are case-insensitive, although "Re: " is the preferred form. A followup agent assisting a poster in preparing a followup SHOULD prepend a back reference, UNLESS the subject already begins with one. If the poster determines that the topic of the followup differs significantly from what is described in the subject, a new, more descriptive, subject SHOULD be substituted (with no back reference). An article whose subject begins with a back reference MUST have a References header referencing the precursor.
new topic (was: old topic)
For historical reasons, the subject MUST not begin with "cmsg " (note that this sequence ends with a blank).
The subject SHOULD be terse. Posters SHOULD avoid trying to cram their entire article into the headers; even the simplest query usually benefits from a sentence or two of elaboration and context, and the details of header display vary widely among reading agents.
The Newsgroups header's content specifies which newsgroup(s) the article is posted to:
Newsgroups-content = newsgroup-name *( ng-delim newsgroup-name ) newsgroup-name = plain-component *( "." component ) component = plain-component / encoded-word plain-component = component-start *13component-rest component-start = lowercase / digit lowercase = <letter a-z> component-rest = component-start / "+" / "-" / "_" ng-delim = ","
Encoded words used in newsgroup names MUST not contain characters other than letters, digits, "+", "-", "/", "_", "=", and "?" (although they may encode them).
A newsgroup name consists of one or more components, which may be plain components or (except for the first) encoded words. A plain component MUST contain at least one letter, MUST begin with a letter or digit, and MUST not be longer than 14 characters. The first component MUST begin with a letter; subsequent components SHOULD begin with a letter. Newsgroup names MUST not contain uppercase letters, except where required by encodings in encoded words. The sequences "all" and "ctl" MUST not be used as components.
ng-delim = "," [ space ]
Encoded words are allowed in newsgroup names ONLY where nonASCII characters are necessary to the name, and must use the "b" encoding [rrr] and the first suitable character set in the MIME order of preferred character sets [rrr].
Posters SHOULD use only the names of existing newsgroups in the Newsgroups header, because newsgroups are NOT created simply by being posted to. However, it is legitimate to cross-post to newsgroup(s) which do not exist on the posting agent's host, provided that at least one of the newsgroups DOES exist there, and followup agents MUST accept this (posting agents MAY accept it, but SHOULD at least alert the poster to the situation and request confirmation). Relayers MUST not rewrite Newsgroups headers in any way, even if some or all of the newsgroups do not exist on the relayer's host.
Cross-posting an article to several relevant newsgroups is far superior to posting separate articles with duplicated content to each newsgroup, because reading agents can detect the situation and show the article to a reader only once. Posters SHOULD cross-post rather than duplicate-post.
A newsgroup SHOULD not appear more than once in the Newsgroups header.
Newsgroup names having only one component are reserved for newsgroups whose propagation is restricted to a single host (or the administrative equivalent). It is inadvisable to name a newsgroup "poster" because that word has special meaning in the Followup-To header (see section 6.1 ). The names "control" and "junk" are frequently used for pseudonewsgroups internal to relayer implementations, and hence are also best avoided.
It is conventional to reserve newsgroup names beginning with "to." for test messages sent on an essentially point-topoint basis (see also the ihave/sendme protocol described in section 7.2 ); newsgroup names beginning with "to." SHOULD not be used for any other purpose. The second (and possibly later) components of such a name should, together, comprise the relayer name (see section 5.6 ) of a relayer. The newsgroup exists only at the named relayer and its neighbors. The neighbors all pass that newsgroup to the named relayer, while the named relayer does not pass it to anyone.
The order of newsgroup names in the Newsgroups header is not significant.
The Path header's content indicates which relayers the article has already visited, so that unnecessary redundant transmission can be avoided:
Path-content = [ path-list path-delimiter ] local-part path-list = relayer-name *( path-delimiter relayer-name ) relayer-name = 1*rn-char rn-char = letter / digit / "." / "-" / "_" path-delimiter = "!"
The Path content is a list of relayer names, separated by path delimiters, followed (after a final delimiter) by the local part of a mailing address. Each relayer MUST prepend its name, and a delimiter, to the Path content in all articles it processes. A relayer MUST not pass an article to a neighboring relayer whose name is already mentioned in an article's path list, unless this is explicitly requested by the neighbor in some way. The Path content is casesensitive.
Path: fee!fie!foe!fum
path-delimiter = "!" [ space ]
Because an article will not propagate to a relayer already mentioned in its path list, the path list MUST not contain any names other than those of relayers the article has passed through AS NEWS. This is trivially obvious for normal news articles, but requires attention from the moderators of moderated newsgroups and the implementors and maintainers of gateways.
Relayer names need to be unique among all relayers which will ever see the articles using them. A relayer name is normally either an "official" name for the host the relayer runs on, or some other "official" name controlled by the same organization. Except in cooperating subnets that agree to some other convention, and don't let articles using it escape beyond the subnet, a relayer name MUST be either a UUCP name registered in the UUCP maps (without any domain suffix such as ".UUCP"), or a complete Internet domain name. Use of a (registered) UUCP name is recommended, where practical, to keep the length of the path list down.
The use of Internet domain names in the path list presents one problem: domain names are case-insensitive, but the path list is case-sensitive. Relayers using domain names as their relayer names MUST pick a standard form for the name, and use that form consistently to the exclusion of all others. The preferred form for this purpose, which relayers SHOULD use, is the all-lowercase form.
In the ordinary case, where the poster is the author of the article, the local part following the path list SHOULD be the local part of the poster's full Internet domain mailing address.
The Path content somewhat resembles a mailing address, particularly in the UUCP world with its manual routing and "!" address syntax. Historically, this resemblance was important, and the Path content was often used as a reply address. This practice has always been somewhat unreliable, since news paths are not always mail paths and news relayer names are not always recognized by mail handlers, and its reliability has generally worsened in recent times. The widespread use of and recognition of Internet domain addresses, even outside the actual Internet, has largely eliminated the problem. Readers SHOULD not use the Path content as a reply address. On the other hand, relayer administrators are urged not to break this usage without good reason; where practical, paths followed by news SHOULD be traversable by mail, and mail handlers SHOULD recognize relayer names as host names.
It will typically be difficult or impractical for gateways and moderators to supply a Path content that is useful as a reply address for the author, bearing in mind that the path list they supply will normally be empty. (To reiterate: the path list MUST not contain any names other than those of relayers the article has passed through AS NEWS.) They SHOULD supply a local part that will result in replies to a Path-derived address being returned to the sender with a brief explanation. Software permitting, the local part "not-for-mail" is recommended.
Many MAIL headers, and many of those specified in present and future MAIL extensions, are potentially applicable to news. Headers specific to MAIL's point-to-point transmission paradigm, e.g. To and Cc, SHOULD not appear in news articles. (Gateways wishing to preserve such information for debugging probably SHOULD hide it under different names; prefixing "X-" to the original headers, resulting in e.g. "X-To", is suggested.)
The following optional headers are either specific to news or of particular note in news articles; an article MAY contain some or all of them. (Note that there are some circumstances in which some of them are mandatory; these are explained under the individual headers.) An article MUST not contain two or more headers with any one of these header names.
The Followup-To header contents specify which newsgroup(s) followups should be posted to:
Followup-To-content = Newsgroups-content / "poster"
The syntax is the same as that of the Newsgroups content, with the exception that the magic word "poster" means that followups should be mailed to the article's reply address rather than posted. In the absence of Followup-To, the default newsgroup(s) for a followup are those in the Newsgroups header.
Although it is generally desirable to limit followups to the smallest reasonable set of newsgroups, especially when the precursor was cross-posted widely, posting agents SHOULD not supply a Followup-To header except at the poster's explicit request.
The Expires header content specifies a date and time when the article is deemed to be no longer useful and should be removed ("expired"):
Expires-content = Date-content
The content syntax is the same as that of the Date content. In the absence of Expires, the default is decided by the administrators of each host the article reaches, who MAY also restrict the extent to which the Expires header is honored.
The Expires header has two main applications: removing articles whose utility ends on a specific date (e.g., event announcements which can be removed once the day of the event is past) and preserving articles expected to be of prolonged usefulness (e.g., information aimed at new readers of a newsgroup). The latter application is sometimes abused. Since individual hosts have local policies for expiration of news (depending on available disk space, for instance), posters SHOULD not provide Expires headers for articles unless there is a natural expiration date associated with the topic. Posting agents MUST not provide a default Expires header. Leave it out and allow local policies to be used unless there is a good reason not to. Expiry dates are properly the decision of individual host administrators; posters and moderators SHOULD set only expiry dates that most administrators would agree with.
The Reply-To header content specifies a reply address different from the author's address given in the From header:
Reply-To-content = From-content
In the absence of Reply-To, the reply address is the address in the From header.
Use of a Reply-To header is preferable to including a similar request in the article body, because reply-preparation software can take account of Reply-To automatically.
The Sender header identifies the poster, in the event that this differs from the author identified in the From header:
Sender-content = From-content
In the absence of Sender, the default poster is the author (named in the From header).
If the poster supplies a From header, the posting agent MUST ensure that a Sender header is present, unless it can verify that the mailing address in the From header is a valid mailing address for the poster. A poster-supplied Sender header MAY be used, if its mailing address is verifiably a valid mailing address for the poster; otherwise the posting agent MUST supply a Sender header and delete (or rename, e.g. to X-Unverifiable-Sender) any poster-supplied Sender header.
Sender: fred@zoo.toronto.edu (RFC-1413@reverse-lookup; not verified)
Sender: somebody@zoo.toronto.edu (user name unknown)
The References header content lists message IDs of precursors:
References-content = message-id *( space message-id )
A followup MUST have a References header, and an article which is not a followup MUST not have a References header. In a followup, if the precursor had a References header, the message ID of the precursor is appended to the end of the precursor's References-content to form the followup's References-content. a References header containing the precursor's message ID. A followup to an article which had a References header MUST have a References header containing the precursor's References content, plus the precursor's message ID appended to the end of the list.
Followup agents SHOULD not shorten References headers. If it is absolutely necessary to shorten the header, as a desperate last resort, a followup agent MAY do this by deleting some of the message IDs. However, it MUST not delete the first message ID, the last three message IDs (including that of the immediate precursor), or any message ID mentioned in the body of the followup. If it is possible for the followup agent to determine the Subject content of the articles identified in the References header, it MUST not delete the message ID of any article where the Subject content changed (other than by prepending of a back reference). The followup agent MUST not delete any message ID whose local part ends with "_-_" (underscore (ASCII 95), hyphen (ASCII 45), underscore); followup agents are urged to use this form to mark subject changes, and to avoid using it otherwise.
When a References header is shortened, at least three blanks SHOULD be left between adjacent message IDs at each point where deletions were made. Software preparing new References headers SHOULD preserve multiple blanks in older References content.
To repeat: followup agents SHOULD not shorten References headers.
The Control header content marks the article as a control message, and specifies the desired actions (other than the usual ones of filing and passing on the article):
Control-content = verb *( space argument ) verb = 1*( letter / digit ) argument = 1*<ASCII printable character>
The verb indicates what action should be taken, and the argument(s) (if any) supply details. In some cases, the body of the article may also contain details. Section 7 describes the standard verbs. See also the Also-Control header ( section 6.15 ).
An article with a Control header MUST not have an Also-Control or Supersedes header.
The Distribution header content specifies geographic or organizational limits on an article's propagation:
Distribution-content = distribution *( dist-delim distribution ) dist-delim = "," distribution = plain-component
A distribution is syntactically identical to a one-component newsgroup name, and must satisfy the same rules and restrictions. In the absence of Distribution, the default distribution is "world".
dist-delim = "," [ space ]
A relayer MUST not pass an article to another relayer unless configuration information specifies transmission to that other relayer of BOTH (a) at least one of the article's newsgroup(s), and (b) at least one of the article's distribution(s). In effect, the only role of distributions is to limit propagation, by preventing transmission of articles that would have been transmitted had the decision been based solely on newsgroups.
A posting agent might wish to present a menu of possible distributions, or suggest a default, but normally SHOULD not supply a default without giving the poster a chance to override it. A followup agent SHOULD initially supply the same Distribution header as found in the precursor, although the poster MAY alter this if appropriate.
Despite the syntactic similarity and some historical confusion, distributions are NOT newsgroup names. The whole point of putting a distribution on an article is that it is DIFFERENT from the newsgroup(s). In general, a meaningful distribution corresponds to some sort of region of propagation: a geographical area, an organization, or a cooperating subnet.
The Keywords header content is one or more phrases intended to describe some aspect of the content of the article:
Keywords-content = plain-phrase *( "," [ space ] plain-phrase )
Keywords, separated by commas, each follow the <plainphrase> syntax defined in section 5.2 . Encoded words in keywords MUST not contain characters other than letters (of either case), digits, and the characters "!", "*", "+", "-", "/", "=", and "_".
Keywords: Thompson Ritchie Multics Linux
Keywords: Thompson, Ritchie, Multics, Linux
The Summary header content is a short phrase summarizing the article's content:
Summary-content = nonblank-text
As with the subject, no restriction is placed on the content since it is intended solely for display to humans.
The summary SHOULD be terse. Posters SHOULD avoid trying to cram their entire article into the headers; even the simplest query usually benefits from a sentence or two of elaboration and context, and not all reading agents display all headers.
The Approved header content indicates the mailing addresses (and possibly the full names) of the persons or entities approving the article for posting:
Approved-content = From-content *( "," [ space ] From-content )
An Approved header is required in all postings to moderated newsgroups; the presence or absence of this header allows a posting agent to distinguish between articles posted by the moderator (which are normal articles to be posted normally) and attempted contributions by others (which should be mailed to the moderator for approval). An Approved header is also required in certain control messages, to reduce the probability of accidental posting of same; see the relevant parts of section 7.
The Lines header content indicates the number of lines in the body of the article:
Lines-content = 1*digit
The line count includes all body lines, including the signature if any, including empty lines (if any) at beginning or end of the body. (The single empty separator line between the headers and the body is not part of the body.) The "body" here is the body as found in the posted article, AFTER all transformations such as MIME encodings.
Reading agents SHOULD not rely on the presence of this header, since it is optional (and some posting agents do not supply it). They MUST not rely on it being precise, since it frequently is not.
Relayers SHOULD discard this header if they find it necessary to re-encode the article in such a way that the original Lines header would be rendered incorrect.
The Xref header content indicates where an article was filed by the last relayer to process it:
Xref-content = relayer 1*( space location ) relayer = relayer-name location = newsgroup-name ":" article-locator article-locator = 1*<ASCII printable character>
The relayer's name is included so that software can determine which relayer generated the header (and specifically, whether it really was the one that filed the copy being examined). The locations specify what newsgroups the article was filed under (which may differ from those in the Newsgroups header) and where it was filed under them. The exact form of an article locator is implementation-specific.
A relayer inserting an Xref header into an article MUST delete any previous Xref header. A relayer which is not inserting its own Xref header SHOULD delete any previous Xref header. A relayer MAY delete the Xref header when passing an article on to another relayer.
A relayer MUST use the same name in Xref headers as it uses in Path headers. Reading agents MUST ignore an Xref header containing a relayer name that differs from the one that begins the path list.
The Organization header content is a short phrase identifying the poster's organization:
Organization-content = nonblank-text
This header is typically supplied by the posting agent. The Organization content SHOULD mention geographical location (e.g. city and country) when it is not obvious from the organization's name.
The Organization content is provided for identification only, and does not imply that the poster speaks for the organization or that the article represents organization policy. Posting agents SHOULD permit the poster to override a local default Organization header.
The Supersedes header content specifies articles to be cancelled on arrival of this one:
Supersedes-content = message-id *( space message-id )
Supersedes is equivalent to Also-Control ( section 6.15 ) with an implicit verb of "cancel" ( section 7.1 ).
An article with a Supersedes header MUST not have an Also-Control or Control header.
The Also-Control header content marks the article as being a control message IN ADDITION to being a normal news article, and specifies the desired actions:
Also-Control-content = Control-content
An article with an Also-Control header is filed and passed on normally, but the content of the Also-Control header is processed as if it were found in a Control header.
An article with an Also-Control header MUST not have a Control or Supersedes header.
The See-Also header content lists message IDs of articles that are related to this one but are not its precursors:
See-Also-content = message-id *( space message-id )
See-Also resembles References, but without the restrictions imposed on References by the followup rules.
The Article-Names header content indicates any special significance the article may have in particular newsgroups:
Article-Names-content = 1*( name-clause space ) name-clause = newsgroup-name ":" article-name article-name = letter 1*( letter / digit / "-" )
Each name clause specifies a newsgroup (which SHOULD be among those in the Newsgroups header) and an article name local to that newsgroup. Article names MAY be used by relayers to file the article in special ways, or they MAY just be noted for possible special attention by reading agents. Article names are case-sensitive.
The Article-Names header SHOULD be ignored unless the article also contains an Approved header.
The presence of an Article-Names header does not necessarily imply that the article will be retained unusually long before expiration, or that previous article(s) with similar Article-Names headers will be cancelled by its arrival. Posters preparing special postings SHOULD include appropriate other headers, such as Expires and Supersedes, to request such actions.
Different networks MAY establish different sets of article names for the special postings they deem significant; it is preferable for usage to be standardized within networks, although it might be desirable for individual newsgroups to have different naming conventions in some situations. Article names MUST be 14 characters or less. The following names are suggested but are not mandatory:
intro Introduction to the newsgroup for newcomers.
charter Charter, rules, organization, moderation policies, etc.
background Biographies of special participants, history of the newsgroup, notes on related newsgroups, etc.
subgroups Descriptions of sub-newsgroups under this newsgroup, e.g. "sci.space.news" under "sci.space".
facts Information relating to the purpose of the newsgroup, e.g. an acronym glossary in "sci.space".
references Where to get more information: books, journals, FTP repositories, etc.
faq Answers to frequently-asked questions.
menu If present, a list of all the other article names local to this newsgroup, with brief descriptions of their contents.
Such articles may be divided into subsections using the MIME "multipart/mixed" conventions. If size considerations make it necessary to split such articles, names ending in a hyphen and a part number are suggested; for example, a three-part frequently-asked-questions list could have article names "faq-1", "faq-2", and "faq-3".
The Article-Updates header content indicates what previous articles this one is deemed (by the poster) to update:
Article-Updates-content = message-id *( space message-id )
Each message ID identifies a previous article that this one is deemed to update. This MUST not cause the previous article(s) to be cancelled or otherwise altered, unless this is implied by other headers (e.g. Supersedes); Article-Updates is merely an advisory which MAY be noted for special attention by reading agents.
The following sections document the currently-defined control messages. "Message" is used herein as a synonym for "article" unless context indicates otherwise.
Posting agents are warned that since certain control messages require article bodies in quite specific formats, signatures SHOULD not be appended to such articles, and it may be wise to take greater care than usual to avoid unintended (although perhaps well-meaning) alterations to text supplied by the poster. Relayers MUST assume that control messages mean what they say; they MAY be obeyed as is or rejected, but MUST not be reinterpreted.
The execution of the actions requested by control messages is subject to local administrative restrictions, which MAY deny requests or refer them to an administrator for approval. The descriptions below are generally phrased in terms suggesting mandatory actions, but any or all of these MAY be subject to local administrative approval (either as a class or case-by-case). Analogously, where the description below specifies that a message or portion thereof is to be ignored, this action MAY include reporting it to an administrator.
Relayers MUST propagate even control messages they do not understand.
In the following sections, each type of control message is defined syntactically by defining its arguments and its body. For example, "cancel" is defined by defining cancelarguments and cancel-body.
The cancel message requests that one or more previous articles be "cancelled":
cancel-arguments = message-id *( space message-id ) cancel-body = body
The argument(s) identify the articles to be cancelled, by message ID. The body is a comment, which software MUST ignore, and SHOULD contain an indication of why the cancellation was requested. The cancel message SHOULD be posted to the same newsgroup(s), with the same distribution(s), as the article(s) it is attempting to cancel.
When an article (the "target article") is to be cancelled, there are four cases of interest: the article hasn't arrived yet, it has arrived and been filed and is available for reading, it has expired and been archived on some lessaccessible storage medium, or it has expired and been deleted. The next few paragraphs discuss each case in turn (in reverse order, which is convenient for the explanation).
EXPIRED AND DELETED. Take no action.
EXPIRED AND ARCHIVED. If the article is readily accessible and can be deleted or made unreadable easily, treat as under AVAILABLE below. Otherwise treat as under EXPIRED AND DELETED.
AVAILABLE. Compare the mailing addresses from the From lines of the cancel message and the target article, bearing in mind that local parts (except for "postmaster") are casesensitive and domains are case-insensitive. If they do not match, either refer the issue to an administrator for a case-by-case decision, or treat as if they matched.
If the addresses match, then if technically possible, the relayer MUST delete the target article completely and immediately. Failing that, it MUST make the target article unreadable (preferably to everyone, minimally to everyone but the administrator) and either arrange for it to be deleted as soon as possible or notify an administrator at once.
NOT ARRIVED YET. If practical, retain the cancel message until the target article does arrive, or until there is no further possibility of it arriving and being accepted (see section 9.2 ), and then treat as under AVAILABLE. Failing that, arrange for the target article to be rejected and discarded if it does arrive.
The cancel message MUST be propagated onward in the usual fashion, regardless of which of the four cases applied, so that the target article will be cancelled everywhere even if cancellation and target article follow different routes.
Posting agents meant for use by ordinary posters SHOULD reject an attempt to post a cancel message if the target article is available and the mailing address in its From header does not match the one in the cancel message's From header.
The ihave and sendme control messages implement a crude batched predecessor of the NNTP [rrr] protocol. They are largely obsolete in the Internet, but still see use in the UUCP environment, especially for backup feeds that normally are active only when a primary feed path has failed.
The two messages share the same syntax:
ihave-arguments = *( message-id space ) relayer-name sendme-arguments = ihave-arguments ihave-body = *( message-id eol ) sendme-body = ihave-body
Message IDs MUST appear in either the arguments or the body, but not both. Relayers SHOULD generate the form putting message IDs in the body, but the other form MUST be supported for backward compatibility.
The ihave message states that the named relayer has filed articles with the specified message IDs, which may be of interest to the relayer(s) receiving the ihave message. The sendme message requests that the relayer receiving it send the articles having the specified message IDs to the named relayer.
These control messages are normally sent essentially as point-to-point messages, by using "to." newsgroups (see section 5.5 ) that are sent only to the relayer the messages are intended for. The two relayers MUST be neighbors, exchanging news directly with each other. Each relayer advertises its new arrivals to the other using ihave messages, and each uses sendme messages to request the articles it lacks.
To reduce overhead, ihave and sendme messages SHOULD be sent relatively infrequently and SHOULD contain substantial numbers of message IDs. If ihave and sendme are being used to implement a backup feed, it may be desirable to insert a delay between reception of an ihave and generation of a sendme, so that a slightly slow primary feed will not cause large numbers of articles to be requested unnecessarily via sendme.
The newgroup control message requests that a new newsgroup be created:
newgroup-arguments = newsgroup-name [ space moderation ]
moderation = "moderated" / "unmoderated"
newgroup-body = body
/ [ body ] descriptor [ body ]
descriptor = descriptor-tag eol description-line eol
descriptor-tag = "For your newsgroups file:"
description-line = newsgroup-name space description
description = nonblank-text [ " (Moderated)" ]
The first argument names the newsgroup to be created, and the second one (if present) indicates whether it is moderated. If there is no second argument, the default is "unmoderated".
The body is a comment, which software MUST ignore, except that if it contains a descriptor, the description line is intended to be suitable for addition to a list of newsgroup descriptions. The description cannot be continued onto later lines, but is not constrained to any particular length. Moderated newsgroups have descriptions that end with the string " (Moderated)" (note that this string begins with a blank).
The remainder of the body SHOULD contain an explanation of the purpose of the newsgroup and the decision to create it.
A newgroup message which lacks an Approved header MUST be ignored.
It would be desirable to provide some way of supplying a moderator's address in a newgroup message for a moderated newsgroup, but this will cause problems unless effective authentication is available, so it is left for future work.
A newgroup message naming a newsgroup that already exists is requesting a change in the moderation status or description of the newsgroup. The same rules apply.
The rmgroup message requests that a newsgroup be deleted:
rmgroup-arguments = newsgroup-name rmgroup-body = body
The sole argument is the newsgroup name. The body is a comment, which software MUST ignore; it SHOULD contain an explanation of the decision to delete the newsgroup.
A rmgroup message which lacks an Approved header MUST be ignored.
Unexpected deletion of a newsgroup being a disruptive action, implementations are strongly advised to refer rmgroup messages to an administrator by default, unless perhaps the message can be determined to have originated within a cooperating subnet whose members are considered trustworthy. Abuses have occurred.
The sendsys message requests that a description of the relayer's news feeds to other relayers be mailed to the article's reply address:
sendsys-arguments = [ relayer-name ] sendsys-body = body
If there is an argument, relayers other than the one named by the argument MUST not respond. The body is a comment, which software MUST ignore; it SHOULD contain an explanation of the reason for the request.
The version message requests that the name and version of the relayer software be mailed to the reply address:
version-arguments = version-body = body
There are no arguments. The body is a comment, which software MUST ignore; it SHOULD contain an explanation of the reason for the request.
The whogets message requests that a description of the relayer and its news feeds to other relayers be mailed to the article's reply address:
whogets-arguments = newsgroup-name [ space relayer-name ] whogets-body = body
The first argument is the name of the "target newsgroup", specifying the newsgroup for which propagation information is desired. This MUST be a complete newsgroup name, not the name of a hierarchy or a portion of a newsgroup name that is not itself the name of a newsgroup. If there is a second argument, only the relayer named by that argument should respond. The body is a comment, which software MUST ignore; it SHOULD contain an explanation of the reason for the request.
Any of these messages lacking an Approved header MUST be ignored. Response to any of these messages SHOULD be delayed for at least 24 hours, and no response should be attempted if the message has been cancelled in that time. Also, no response SHOULD be attempted unless the local part of the destination address is "newsmap". News administrators SHOULD arrange for mail to "newsmap" on their systems to be discarded (without reply) unless legitimate use is in progress.
The body of the reply to a sendsys message SHOULD be of the form:
sendsys-reply = responder 1*sys-line responder = "Responding-System:" space domain eol sys-line = relayer-name ":" newsgroup-patterns [ ":" text ] eol newsgroup-patterns = newsgroup-name *( "," newsgroup-name )
The first line identifies the responding system, using a syntax resembling a header (but note that it is part of the BODY). Remaining lines indicate what newsgroups are sent to what other systems. The syntax of newsgroup patterns is not well standardized; the form described is common (often with newsgroup names only partially given, denoting all names starting with a particular set of components) but not universal. The whogets message provides a better-defined alternative.
The reply to a version message is of somewhat ill-defined form, with a body normally consisting of a single line of text that somehow describes the version of the relayer software. The whogets message provides a better-defined alternative.
The body of the reply to a whogets message MUST be of the form:
whogets-reply = responder-domain responder-relayer response-date
responding-to arrived-via responder-version
whogets-delimiter *pass-line
responder-domain = "Responding-System:" space domain eol
responder-relayer = "Responding-Relayer:" space relayer-name eol
response-date = "Response-Date:" space date eol
responding-to = "Responding-To:" space message-id eol
arrived-via = "Arrived-Via:" path-list eol
responder-version = "Responding-Version:" space nonblank-text eol
whogets-delimiter = eol
pass-line = relayer-name [ space domain ] eol
The first six lines identify the responding relayer by its Internet domain name (use of the ".uucp" and ".bitnet" pseudo-domains is permissible, for registered hosts in them, but discouraged) and its relayer name, specify the date when the reply was generated and the message ID of the whogets message being replied to, give the path list (from the Path header) of the whogets message (which MAY, if absolutely necessary, be truncated to a convenient length, but MUST contain at least the leading three relayer names), and indicate the version of relayer software responding. Note that these lines are part of the BODY even though their format resembles that of headers. Despite the apparently-fixed order specified by the syntax above, they can appear in any order, but there must be exactly one of each.
After those preliminaries, and an empty line to unambiguously define their end, the remaining lines are the relayer names (which MAY be accompanied by the corresponding domain names, if known) of systems which the responding system passes the target newsgroup to. Only the names of news relayers are to be included.
A relayer which is unaware of the existence of the target newsgroup MUST not reply to a whogets message at all, although this MUST not influence decisions on whether to pass the article on to other relayers.
Different networks set different rules for the legitimacy of these messages, given that they may reveal details of organization-internal topology that are sometimes considered proprietary.
The checkgroups control message contains a supposedly authoritative list of the valid newsgroups within some subset of the newsgroup name space:
checkgroups-arguments =
checkgroups-body = [ invalidation ] valid-groups
/ invalidation
invalidation = "!" plain-component *( "," plain-component ) eol
valid-groups = 1*( description-line eol )
There are no arguments. The body lines (except possibly for an initial invalidation) each contain a description line for a newsgroup, as defined under the newgroup message ( section 7.3 ).
The checkgroups message applies to all hierarchies containing any of the newsgroups listed in the body. The checkgroups message asserts that the newsgroups it lists are the only newsgroups in those hierarchies. If there is an invalidation, it asserts that the hierarchies it names no longer contain any newsgroups.
Processing a checkgroups message MAY cause a local list of newsgroup descriptions to be updated. It SHOULD also cause the local lists of newsgroups (and their moderation statuses) in the mentioned hierarchies to be checked against the message. The results of the check MAY be used for automatic corrective action, or MAY be reported to the news administrator in some way.
While this Draft does not specify transmission methods except to place a few constraints on them, there are some data formats used only for transmission that are unique to news.
For efficient bulk transmission and processing of news articles, it is often desirable to transmit a number of them as a single block of data, a "batch". The format of a batch is:
batch = 1*( batch-header article ) batch-header = "#! rnews " article-size eol article-size = 1*digit
A batch is a sequence of articles, each prefixed by a header line that includes its size. The article size is a decimal count of the octets in the article, counting each EOL as one octet regardless of how it is actually represented.
When transmitting news, especially over communications links that are slow or are billed by the bit, it is often desirable to batch news and apply data compression to the batches. Transmission links sending compressed batches SHOULD use out-of-band means of communication to specify the compression algorithm being used. If there is no way to send out-of-band information along with a batch, the following encapsulation for a compressed batch MAY be used:
ec-batch = "#! " compression-keyword eol compressed-batch compression-keyword = "cunbatch"
A line containing a keyword indicating the type of compression is followed by the compressed batch. The only truly widespread compression keyword at present is "cunbatch", indicating compression using the widely-distributed "compress" program. Other compression keywords MAY be used by mutual agreement between the hosts involved.
rnews -d decompressor
It is often desirable to transmit news as mail, either for the convenience of a human recipient or because that is the only type of transmission available on a restrictive communication path.
Given the similarity between the news format and the MAIL format, it is superficially attractive to just send the news article as a mail message. This is typically a mistake: mail-handling software often feels free to manipulate various headers in undesirable ways (in some cases, such as Sender, such manipulation is actually mandatory), and mail transmission problems etc. MUST be reported to the administrators responsible for the mail transmission rather than to the article's author. In general, news sent as mail should be encapsulated to separate the mail headers and the news headers.
When the intended recipient is a human, any convenient form of encapsulation may be used. Recommended practice is to use MIME encapsulation with a content type of "message/news", given that news articles have additional semantics beyond what "message/rfc822" implies.
When mail is being used as a transmission path between two relayers, however, a standard method is desirable. Currently the standard method is to send the mail to an address whose local part is "rnews", with whatever mail headers are necessary for successful transmission. The news article (including its headers) is sent as the body of the mail message, with an "N" prepended to each line.
This method has its weaknesses. In particular, it assumes that the mail transmission channel can transmit nearlyarbitrary body text undamaged. When mail is being used as a transmission path of last resort, however, the mail system often has inconvenient preconceived notions about the format of message bodies. Various ad-hoc encoding schemes have been used to avoid such problems. The recommended method is to send a news article or batch as the body of a MIME mail message, using content type "application/news-transmission" and MIME's "base64" encoding (which is specifically designed to survive all known major mail systems).
Most aspects of news propagation and processing are implementation-specific. The basic propagation algorithms, and certain details of how they are implemented, nevertheless need to be standard.
There are two important principles that news implementors (and administrators) need to keep in mind. The first is the well-known Internet Robustness Principle:
However, in the case of news there is an even more important principle, derived from a much older code of practice, the Hippocratic Oath (we will thus call this the Hippocratic Principle):
It is VITAL to realize that decisions which might be merely suboptimal in a smaller context can become devastating mistakes when amplified by the actions of thousands of hosts within a few hours.
Relayers MUST not alter the content of articles unnecessarily. Well-intentioned attempts to "improve" headers, in particular, typically do more harm than good. It is necessary for a relayer to prepend its own name to the Path content (see section 5.6 ) and permissible for it to rewrite or delete the Xref header (see section 6.12 ). Relayers MAY delete the thoroughly-obsolete headers described in appendix A.3, although this behavior no longer seems useful enough to encourage. Other alterations SHOULD be avoided at all costs, as per the Hippocratic Principle.
Relayers MUST not pass non-conforming articles on to other relayers, except perhaps in a cooperating subnet that has agreed to permit certain kinds of non-conforming behavior. This is a direct consequence of the Internet Robustness Principle.
The two preceding paragraphs may appear to be in conflict. What is to be done when a non-conforming article is received? The Robustness Principle argues that it should be accepted but must not be passed on to other relayers while still non-conforming, and the Hippocratic Principle strongly discourages attempts at repair. The conclusion that this appears to lead to is correct: a non-conforming article MAY be accepted for local filing and processing, or it MAY be discarded entirely, but it MUST not be passed on to other relayers.
A relayer MUST not respond to the arrival of an article by sending mail to any destination, other than a local administrator, except by explicit prearrangement with the recipient. Neither posting an article (other than certain types of control message, see section 7.5 ) nor being the moderator of a moderated newsgroup constitutes such prearrangement. UNDER NO CIRCUMSTANCES WHATSOEVER may a relayer attempt to send mail to either an article's originator or a moderator.
Notification of problems in incoming articles MAY go to local administrators, or at most (by prearrangement!) to the administrators of the neighboring relayer(s) that passed on the problematic articles.
If it is necessary to alter an article, e.g. translate it to another character set or alter its EOL representation, strenuous efforts should be made to ensure that such transformations are reversible, and that relayers or other software that might wish to reverse them know exactly how to do so.
When a relayer first receives an article, it must decide whether to accept it. (This applies regardless of whether the article arrived by itself or as part of a batch, and in principle regardless of whether it originated as a local posting or as traffic from another relayer.) In a cooperating subnet with well-controlled propagation paths, some of the tests specified here MAY be delegated to centrallylocated relayers; that is, relayers that can receive news ONLY via one of the central relayers might simplify acceptance testing based on the assumption that incoming traffic has already passed the full set of tests at a central relayer.
The wording that follows is based on a model in which articles arrive on a relayer's host before acceptance tests are done. However, depending on the degree of integration of the transport mechanisms and the relayer, some or all of these tests MAY be done before the article is actually transmitted, so that articles which definitely will not be accepted need not be transmitted at all.
The wording that follows also specifies a particular order for the acceptance tests. While this order is the obvious one, the tests MAY be done in any order.
First, the relayer MUST verify that the article is a legal news article, with all mandatory headers present with legal contents.
Second, the relayer MUST determine whether it has already seen this article (identified by its message ID). This is normally done by retaining a history of all article message IDs seen in the last N days, where the value of N is decided by the relayer's administrator but SHOULD be at least 7. Since N cannot practically be infinite, articles whose Date content indicates that they are older than N days are declared "stale" and are deemed to have been seen already.
Third, the relayer MUST determine whether any of the article's newsgroups are "subscribed to" by the host, i.e. fit a description of what hierarchies or newsgroups the site wants to receive.
Once an article has been accepted, it may be passed on to other relayers. The fundamental news propagation rule is a flooding algorithm: on receiving and accepting an article, send it to all neighboring relayers not already in its path list that are sent its newsgroup(s) and distribution(s).
In general, relayers SHOULD not make propagation decisions by "anticipation": relayer X, noting that the article's path list already contains relayer Y, decides not to send it to relayer Z because X anticipates that Z will get the article by a better path. If that is generally true, then why is there a news feed from X to Z at all? In fact, the "better path" may be running slowly or may be down. News propagation is very robust precisely because some redundant transmission is done "just in case". If it is imperative to limit unnecessary traffic on a path, use of NNTP [rrr] or ihave/sendme (see section 7.2 ) to pass articles only when necessary is better than arbitrary decisions not to pass articles at all.
Anticipation is occasionally justified in special cases. Such cases should involve both (1) a cooperating subnet whose propagation paths are well-understood and wellmonitored, with failures and slowdowns noticed and dealt with promptly, and (2) a persistent pattern of heavy unnecessary traffic on a path that is either slow or costly. In addition, there should be some reason why neither NNTP nor ihave/sendme is suitable as a solution to the problem.
It is desirable to have a standardized contact address for a relayer's administrators, in the spirit of the "postmaster" address for mail administrators. Mail addressed to "newsmaster" on a relayer's host MUST go to the administrator(s) of that relayer. Mail addressed to "usenet" on the relayer's host SHOULD be handled likewise. Mail addressed to either address on other hosts using the same news database SHOULD be handled likewise.
Gatewaying of traffic between news networks using this Draft and those using other exchange mechanisms can be useful, but must be done cautiously. Gateway administrators are taking on significant responsibilities, and must recognize that the consequences of error can be quite serious.
This section will primarily address the problems of gatewaying traffic INTO news networks. Little can be said about the other direction without some specific knowledge of the network(s) involved. However, the two issues are not entirely independent: if a non-news network is gatewayed into a news network at more than one point, traffic injected into the non-news network by one gateway may appear at another as a candidate for injection back into the news network.
This raises a more general principle, the single most important issue for gatewaying:
The normal loop prevention of news transmission is vitally dependent on the Message-ID header. Any gateway which finds it necessary to remove this header, alter it, or supersede it (by moving it into the body), MUST take equally effective precautions against looping.
Gateway implementors should realize that gateways have all the responsibilities of relayers, plus the added complications introduced by transformations between different information formats. Much of section 9 's discussion of relayer issues is relevant to gateways as well. In particular, gateways SHOULD keep a history of recently-seen articles, as described in section 9.2 , and not assume that articles will never reappear. This is particularly important for networks that have their own concept analogous to message IDs: a gateway should keep a history of traffic seen from BOTH directions.
If at all possible, articles entering the non-news network SHOULD be marked in some way so that they will NOT be regatewayed back into news. Multiple gateways obviously must agree on the marking method used; if it is done by having them know each others' names, name changes MUST be coordinated with great care. If marking cannot be done, all transformations MUST be reversible so that a re-gatewayed article is identical to the original (except perhaps for a longer Path header).
Gateways MUST not pass control messages (articles containing Control, Also-Control, or Supersedes headers) without removing the headers that make them control messages, unless there are compelling reasons to believe that they are relevant to both sides and that conventions are compatible. If it is truly desirable to pass them unaltered, suitable precautions MUST be taken to ensure that there is NO POSSIBILITY of a looping control message.
Gateways, like relayers, SHOULD make determined efforts to avoid mangling articles unnecessarily. In the case of gateways, some transformations may be inevitable, but keeping them to a minimum and ensuring that they are reversible is still highly desirable.
Gateways MUST avoid destroying information. In particular, the restrictions of section 4.2.2 are best taken with a grain of salt in the context of gateways. Information that does not translate directly into news headers SHOULD be retained, perhaps in "X-" headers, both because it may be of interest to sophisticated readers and because it may be crucial to tracing propagation problems.
Gateway implementors should take particular note of the discussion of mailed replies, or more precisely the ban on same, in section 9.1 . Gateway problems MUST be reported to the local administration, not to the innocent originator of traffic. "Gateway problems" here includes all forms of propagation anomaly on the non-news side of the gateway, e.g. unreachable addresses on a mailing list. Note that this requires consideration of possible misbehavior of "downstream" hosts, not just the gateway host.
News articles prepared by gateways MUST be legal news articles. In particular, they MUST include all of the mandatory headers (see section 5 ) and MUST fully conform to the restrictions on said headers. This often requires that a gateway function not only as a relayer, but also partly as a posting agent, aiding in the synthesis of a conforming article from non-conforming input.
The mandatory headers generally present few problems.
If no date information is available, the gateway should supply a Date header with the gateway's current date. If only partial information is available (e.g. date but not time), this should be fleshed out to a full Date header by adding default values, not by mixing in parts of the gateway's current date. (Defaults should be chosen so that fleshed-out dates will not be in the future!) It may be necessary to map timezone information to the restricted forms permitted in the news Date header. See section 5.1.
If the author's address as supplied in the original message is not suitable for inclusion in a From header, the gateway MUST transform it so it is, e.g. by use of the "% hack" and the domain address of the gateway. The desire to preserve information is NOT an excuse for violating the rules. If the transformation is drastic enough that there is reason to suspect loss of information, it may be desirable to include the original form in an X- header, but the From header's contents MUST be as specified in section 5.2 .
If the message contains a Message-ID header, the contents should be dealt with as discussed in section 10.3 . If there is no message ID present, it will be necessary to synthesize one, following the news rules (see section 5.3 ).
Every effort should be made to produce a meaningful Subject header; see section 5.4 . Many news readers select articles to read based on Subject headers, and inserting a placeholder like "<no subject available>" is considered highly objectionable. Even synthesizing a Subject header by picking out the first half-dozen nouns and adjectives in the article body is better than using a placeholder, since it offers SOME indication of what the article might contain.
The contents of the Newsgroups header ( section 5.5 ) are usually predetermined by gateway configuration, but a gateway to a network that has its own concept of newsgroups or discussions might have to make transformations. Such transformations should be reversible; otherwise confusion is likely on both sides.
It will rarely be possible for gateways to provide a Path header that is both an accurate history of the relayers the article has passed through AS NEWS and a usable reply address. The history function MUST be given priority; see the discussion in section 5.6 . It will usually be necessary for a gateway to supply an empty path list, abandoning the reply function.
It is desirable for gatewayed articles to convey as much useful information as possible, e.g. by use of optional news headers (see section 6 ) when the relevant information is available. Synthesis of optional headers can generally follow similar rules.
Software synthesizing References headers should note the discussion in section 6.5 concerning the incompatibility between MAIL and news. Also of interest is the possibility of incorporating information from In-Reply-To headers and from attribution lines in the body; an incomplete or somewhat conjectural References header is much better than none at all, and reading agents already have to cope with incomplete or slightly erroneous References lists.
This section, like the previous one, is phrased in terms of mail being gatewayed into news, but most of the discussion should be more generally applicable.
A particularly sticky problem of gatewaying mail into news is supplying legal news message IDs. Note, in particular, that not all MAIL message IDs are legal in news; the news syntax (specified in section 5.3 , with related material in 5.2) is more restrictive. Generating a fully-conforming news article from a mail message may require transforming the message ID somewhat.
Generation and transformation of message IDs assumes particular importance if a given mailing list (or whatever) is being handled by more than one gateway. It is highly desirable that the same article contents not appear twice in the same newsgroup, which requires that they receive the same message ID from all gateways. Gateways SHOULD use the following algorithm (possibly modified by the later discussion of gatewaying into more than one newsgroup) unless local considerations dictate another:
1. Separate message ID from surroundings, if necessary. A plausible method for this is to start at the first "<", end at the next ">", and reject the message if no ">" is found or a second "<" is seen before the ">". Also reject the message if the message ID contains no "@" or more than one "@", or if it contains no ".". Also reject the message if the message ID contains non-ASCII characters, ASCII control characters, or white space.
2. Delete the leading "<" and trailing ">". Separate message ID into local part and domain at the "@".
3. In both components, transliterate leading dots (".", ASCII 46), trailing dots, and dots after the first in sequences of two or more consecutive dots, into underscores (ASCII 95).
4. In both components, transliterate disallowed characters other than dots (see the definition of <unquoted-char> in section 5.2 ) to underscores (ASCII 95).
5. Form the message ID as
"<" local-part "@" domain ">"
Despite the desire to keep message IDs consistent across multiple gateways, there is also a more subtle issue that can require a different approach. If the same articles are being gatewayed into more than one newsgroup, and it is not possible to arrange that all gateways gateway them to the same cross-posted set of newsgroups, then the message IDs in the different newsgroups MUST be DIFFERENT.
In such cases, it is suggested that the newsgroup name, or an agreed-on abbreviation thereof, be prepended to the local part of the message ID (with a separating ".") by the gateway. This will ensure that multiple gateways generate the same message ID, while also ensuring that different newsgroups can be read independently.
Gatewaying mail to news, and vice-versa, is the most obvious form of news gatewaying. It is common to set up gateways between news and mail rather too casually.
It is hard to go very wrong in gatewaying news into a mailing list, except for the non-trivial matter of making sure that error reports go to the local administration rather than to the authors of news articles. (This requires attention to the "envelope address" as well as to the message headers.) Doing the reverse connection correctly is much harder than it looks.
It is useful to distinguish between two different forms of mail-to-news gatewaying: gatewaying a mailing list into a newsgroup, and operating a "post-by-mail" service in which individual articles can be posted to a newsgroup by mailing them to a specific address. In the first case, the message is already being "broadcast", and the situation can be viewed as gatewaying one form of news into another. The second case is closer to that of a moderator posting submissions to a moderated newsgroup.
In either case, the discussions in the preceding two sections are relevant, as is the Hippocratic Principle of section 9. However, some additional considerations are specific to mail-to-news gatewaying.
As mentioned in section 6 , point-to-point headers like To and Cc SHOULD not appear as such in news, although it is suggested that they be transformed to "X-" headers, e.g. XTo and X-Cc, to preserve their information content for possible use by readers or troubleshooters. The Received header is entirely specific to MAIL and SHOULD be deleted completely during gatewaying, except perhaps for the Received header supplied by the gateway host itself.
The Sender header is a tricky case, one where mailing-list and post-by-mail practice should differ. For gatewaying mailing lists, the mailing-list host should be considered a relayer, and the From and Sender headers supplied in its transmissions left strictly untouched. For post-by-mail, as for a moderator posting a mailed submission, the Sender header should reflect the poster rather than the author. If a post-by-mail gateway receives a message with its own Sender header, it might wish to preserve the content in an X-Sender header.
It will generally be necessary to transform between mail's In-Reply-To/References convention and news's References/SeeAlso convention, to preserve correct semantics of cross references. This also requires attention when going the other way, from news to mail. See the discussion of the difference in section 6.5 .
Any news system will benefit from an attentive administrator, preferably assisted by automated monitoring for anomalies. This is particularly true of gateways. Gateway software SHOULD be instrumented so that unusual occurrences, such as sudden massive surges in traffic, are reported promptly. It is desirable, in fact, to go further: gateway software SHOULD endeavour to limit damage in the event that the administrator does not respond promptly.
Traffic gatewayed into a news network SHOULD include a suitable header, perhaps X-Gateway-Administrator, giving an electronic address that can be used to report problems. This SHOULD be an address that goes direct to a human, not to a "routine administrative issues" mailbox that is examined only occasionally, since the point is to be able to reach the administrator quickly in an emergency. Gateway administrators SHOULD arrange substitutes to cover gateway operation (with suitable redirection of mail) when they are on vacation etc.
Although the interchange format itself raises no significant security issues, the wider context does.
The most obvious form of security problem with news is "leakage" of articles which are intended to have only restricted circulation. The flooding algorithm is EXTREMELY good at finding any path by which articles can leave a subnet with supposedly-restrictive boundaries. Substantial administrative effort is required to ensure that local newsgroups remain local, unless connections to the outside world are tightly restricted.
A related problem is that the sendme control message can be used to ask for any article by its message ID. The usefulness of this has declined as message-ID generation algorithms have become less predictable, but it remains a potential problem for "secure" newsgroups. Hosts with such newsgroups may wish to disable the sendme control message entirely. The sendsys, version, and whogets control messages also allow "outsiders" to request information from "inside", which may reveal details of internal topology (etc.) that are considered confidential. (Note that at least limited openness about such matters may be a condition of membership in such networks, e.g. Usenet.)
Organizations wishing to control these forms of leakage are strongly advised to designate a small number of "official gateway" hosts to handle all news exchange with the outside world, so that a bounded amount of administrative effort is needed to control propagation and eliminate problems. Attempts to keep news out entirely, by refusing to support an official gateway, typically result in large numbers of unofficial partial gateways appearing over time. Such a configuration is much more difficult to troubleshoot.
A somewhat-related problem is the possibility of proprietary material being disclosed unintentionally by a poster who does not realize how far his words will propagate, either from sheer misunderstanding or because of errors made (by human or software) in followup preparation. There is little that can be done about this except education.
Although the limitations of the medium restrict what can be done to attack a host via news, some possibilities exist, most of them problems news shares with mail.
If reading agents are careless about transmitting nonprintable characters to output devices, malicious posters may post articles containing control sequences ("letterbombs") meant to have various destructive effects on output devices. Possible effects depend on the device, but they can include hardware damage (e.g. by repeated writing of values into configuration memories that can tolerate only a limited number of write cycles) and security violation (e.g. by reprogramming function keys potentially used by privileged readers).
A more sophisticated variation on the letterbomb is inclusion of "Trojan horses" in programs. Obviously, readers must be cautious about using software found in news, but more subtly, reading agents must also exercise care. MIME messages can include material that is executable in some sense, such as PostScript documents (which are programs!), and letterbombs may be introduced into such material.
Given the presence of finite resources and other software limitations, some degree of system disruption can be achieved by posting otherwise-innocent material in great volume, either in single huge articles (see section 4.6 ) or in a stream of modest-sized articles. (Some would say that the steady growth of Usenet volume constitutes a subtle and unintentional attack of the latter type; certainly it can have disruptive effects if administrators are inattentive.) Systems need some ability to cope with surges, because single huge articles occur occasionally as the result of software error, innocent misunderstanding, or deliberate malice, and downtime at upstream hosts can cause droughts, followed by floods, of legitimate articles. (There is also a certain amount of normal variation; for example, Usenet traffic is noticeably lighter on weekends and during Christmas holidays, and rises noticeably at the start of the school term of North American universities.) However, a site that normally receives little traffic may be quite vulnerable to "swamping" attack if its software is insufficiently careful.
In general, careless implementation may open doors that are not intrinsic to news. In particular, implementation of control messages (see sections 6.6 and 7) and unbatchers (see section 8.1 and 8.2) via a command interpreter requires substantial precautions to ensure that only the intended capabilities are available. Care must also be taken that article-supplied text is not fed to programs that have escapes to command interpreters.
Finally, there is considerable potential for malice in the sendsys, version, and whogets control messages. They are not harmful to the hosts receiving them as news, but they can be used to enlist those hosts (by the thousands) as unwitting allies in a mail-swamping attack on a victim who may not even receive news. The precautions discussed in section 7.5 can reduce the potential for such attacks considerably, but the hazard cannot be eliminated as long as these control messages exist.
The highly distributed nature of news propagation, and the lack of adequate authentication protocols (especially for use over the less-interactive transport mechanisms such as UUCP), make article forgery relatively straightforward. It may be possible to at least track a forgery to its source, once it is recognized as such, but clever forgers can make even that relatively difficult. The assumption that forgeries will be recognized as such is also not to be taken for granted; readers are notoriously prone to blindly assuming authenticity. If a forged article's initial path list includes the relayer name of the supposed poster's host, the article will never be sent to that host, and the alleged author may learn about the forgery secondhand or not at all.
A particularly noxious form of forgery is the forged "cancel" control message. Notably, it is relatively straightforward to write software that will automatically send out a (forged) cancel message for any article meeting some criterion, e.g. written by a specific author. The authentication problems discussed in section 7.1 make it difficult to solve this without crippling cancel's important functionality.
A related problem is the possibility of disagreements over newsgroup creation, on networks where such things are not decided by central authorities. There have been cases of "rmgroup wars", where one poster persistently sends out newgroup messages to create a newsgroup and another, equally persistently, sends out rmgroup messages asking that it be removed. This is not particularly damaging, if relayers are configured to be cautious, but can cause serious confusion among innocent third parties who just want to know whether they can use the newsgroup for communication or not.
News shares the legal uncertainty surrounding other forms of electronic communication: what rules apply to this new medium of information exchange? News is a particularly problematic case because it is a broadcast medium rather than a point-to-point one like mail, and analogies to older forms of communication are particularly weak.
Are news-carrying hosts common carriers, like the phone companies, providing communications paths without having either authority over or responsibility for content? Or are they publishers, responsible for the content regardless of whether they are aware of it or not? Or something in between? Such questions are particularly significant when the content is technically criminal, e.g. some types of sexually-oriented material in some jurisdictions, in which case ignorance of its presence may not be an adequate defence.
Even in milder situations such as libel or copyright violation, the responsibilities of the poster, his host, and other hosts carrying the traffic are unclear. Note, in particular, the problems arising when the article is a forgery, or when the alleged author claims it is a forgery but cannot prove this.
The obsolete "A News" article format consisted of exactly five lines of header information, followed by the body. For example:
Aeagle.642 news.misc cbosgd!mhuxj!mhuxt!eagle!jerry Fri Nov 19 16:14:55 1982 Usenet Etiquette - Please Read body body body
The first line consisted of an "A" followed by an article ID (analogous to a message ID and used for similar purposes). The second line was the list of newsgroups. The third line was the path. The fourth was the date, in the format above (all fields fixed width), resembling an Internet date but not quite the same. The fifth was the subject.
This format is documented for archeological purposes only. Do not generate articles in this format.
The obsolete pseudo-Internet article format, used briefly during the transition between the A News format and the modern format, followed the general outline of a MAIL message but with some non-standard headers. For example:
From: cbosgd!mhuxj!mhuxt!eagle!jerry (Jerry Schwarz) Newsgroups: news.misc Title: Usenet Etiquette -- Please Read Article-I.D.: eagle.642 Posted: Fri Nov 19 16:14:55 1982 Received: Fri Nov 19 16:59:30 1982 Expires: Mon Jan 1 00:00:00 1990
body body body
The From header contained the information now found in the Path header, plus possibly the full name now typically found in the From header. The Title header contained what is now the Subject content. The Posted header contained what is now the Date content. The Article-I.D. header contained an article ID, analogous to a message ID and used for similar purposes. The Newsgroups and Expires headers were approximately as now. The Received header contained the date when the latest relayer to process the article first saw it. All dates were in the above format, with all fields fixed width, resembling an Internet date but not quite the same.
This format is documented for archeological purposes only. Do not generate articles in this format.
Early versions of news software following the modern format sometimes generated headers like the following:
Relay-Version: version B 2.10 2/13/83; site cbosgd.UUCP Posting-Version: version B 2.10 2/13/83; site eagle.UUCP Date-Received: Friday, 19-Nov-82 16:59:30 EST
Relay-Version contained version information about the relayer that last processed the article. Posting-Version contained version information about the posting agent that posted the article. Date-Received contained the date when the last relayer to process the article first saw it (in a slightly nonstandard format).
These headers are documented for archeological purposes only. Do not generate articles using them.
There once was a senduuname control message, resembling sendsys but requesting transmission of the list of hosts that the receiving host had UUCP connections to. This rapidly ceased to be of much use, and many organizations consider information about their internal connectivity to be confidential.
Historically, a checkgroups body consisting of one or two lines, the first of the form "-n newsgroup", caused checkgroups to apply to only that single newsgroup. This form is documented for archeological purposes only; do not use it.
Historically, an article posted to a newsgroup whose name had exactly three components of which the third was "ctl" signified that article was to be taken as a control message. The Subject header specified the actions, in the same way the Control header does now. This form is documented for archeological purposes only; do not use it; do not implement it.
(The editor wishes to thank Luc Rooijakkers; most of this appendix is a lightly-edited version of a summary he kindly supplied.)
MIME (Multipurpose Internet Mail Extensions) is an upwardcompatible set of extensions to RFC 822 , currently documented in RFCs 2045 , 2046 , 2047 , 2048 , and 2049 . This appendix summarizes these documents. See the MIME RFCs for more information; they are very readable.
MIME defines the following new headers:
MIME-Version Content-Type Content-Transfer-Encoding Content-ID Content-Description
The MIME-Version header is mandatory for all messages conforming to the MIME specification and carries the version number of the MIME specification. Example:
MIME-Version: 1.0
The Content-Type header indicates the content type of the message. Content types are split into a top-level type and a subtype, separated by a slash. Auxiliary information can also be supplied, using an attribute-value notation. Example:
Content-Type: text/plain; charset=us-ascii
(In the absence of a Content-Type header this is in fact the default content type.)
Important type/subtype combinations are
Some of the above types require the ability to transport binary data. Since the existing message systems usually do not support this, MIME provides a Content-Transfer-Encoding header to indicate the kind of encoding used. The possible encodings are:
The Content-Description header allows further description of a body part, analogous to the use of Subject for messages.
Finally, the Content-ID header can be used to assign an identification to body parts, analogous to the assignment of identifications to messages by Message-ID. Note that most of these headers are structured header fields, as defined in RFC 822 . Consequently, comments are allowed in their values. The following is a legal MIME header:
Content-Type: (a comment) text (yeah) /
plain (and now some params:) ; charset= (guess what)
iso-8859-1 (we don't have iso-10646 yet, pity)
The second part of MIME, RFC 1522 (Representation of NonASCII Text in Internet Message Headers), addresses the problem of non-ASCII characters in headers. An example of a header using the RFC 1522 mechanism is
From: =?ISO-8859-1?Q?Andr=E9_?= Pirard <PIRARD@vm1.ulg.ac.be>
Such encodings are allowed in selected headers, subject to the restrictions listed in RFC 1522 .
The MIME effort has also produced an RFC defining a ContentMD5 header [rrr 1544], containing an MD5-based "checksum" of the contents of an article or body part, giving high confidence of detecting accidental modifications to the contents.
The "metamail" software package [rrr] helps provide MIME support with minimal changes to mailers, and may also be relevant to news reading agents.
The PEM (Privacy Enhanced Mail) effort is pursuing analogous facilities to offer stronger guarantees against malicious modifications, unauthorized eavesdropping, and forgery. This work too may be applicable to news, once it is reconciled with MIME (by efforts now underway).
This Draft is much longer than RFC 1036 , so there is obviously much change in content. Much of this is just increased precision and rigor. Noteworthy changes and additions include:
Most of this Draft merely documents existing practice, but there are a few attempts to extend it. These are:
TBW
E. Summary of Differences From RFC 822 + 1123
The following are noteworthy differences between this Draft's articles and MAIL messages:
[Sanderson] "Smileys", David Sanderson, O'Reilly & Associates Ltd., 1993.
TBW
Section 11 discusses security considerations in detail.
Henry Spencer (henry@zoo.toronto.edu)
SP Systems
Box 280 Stn. A
Toronto, Ont. M5W1B2 Canada
Internet Draft to be for NEWS (Henry Spencer), 02.06.94
HTML-Version: Vera Heinau (25.03.94, updated 28.03.97)
Changes by V. Heinau:
|
|
| www@chemie.fu-berlin.de | Letzte Änderung: 2003-07-31 |