yEnc

yEnc is a binary-to-text encoding scheme for transferring binary files in messages on Usenet or via e-mail. It reduces the overhead over previous US-ASCII-based encoding methods by using an 8-bit Extended ASCII encoding method. yEnc's overhead is often (if each byte value appears approximately with the same frequency on average) as little as 1–2%,[1] compared to 33%–40% overhead for 6-bit encoding methods like uuencode and Base64. yEnc was initially developed by Jürgen Helbing and its first release was early 2001. By 2003 yEnc became the de facto standard encoding system for binary files on Usenet.[2] The name yEncode is actually a wordplay on "Why encode?", since the idea is to only encode characters if it is absolutely required to adhere to the message format standard.[3]

With decreased overhead, the encoded message body is smaller. Therefore, the message can be delivered faster and requires less storage space.

An additional advantage of yEnc over previous encoding methods, such as uuencode and Base64, is the inclusion of a CRC checksum to verify that the decoded file has been delivered intact.

How yEnc works

Usenet and email message bodies were intended to contain only ASCII characters (RFC 822 or RFC 2822). Most competing encodings represent binary files by converting them into printable ASCII characters, because the range of printable ASCII characters is supported by most operating systems. However, since this reduces the available character set considerably, there is significant overhead (wasted bandwidth) over 8bit-byte networks. For example, in uuencode and Base64, three bytes of data are encoded into four printable ASCII characters, which equals four bytes, a 33% overhead (not including the overhead from headers). yEnc uses one character (one byte) to represent one byte of the file, with a few exceptions.

The RFCs that define Internet messages still require that carriage returns and line feeds have special meaning in a mail message. Therefore, yEnc escapes the carriage return and line feed characters in the encoded body.

There is no RFC or other standards document describing yEnc. The yEnc homepage contains a draft informal specification and a grammar (which contradict RFC 2822 and RFC 2045), although neither has been submitted to the Internet Engineering Task Force.

As with uuencoding, despite its flaws, yEnc remains active and effective on Usenet. The yEnc homepage states that "all major newsreaders have been extended to yEnc support". Microsoft's Outlook Express, Windows Mail and Windows Live Mail do not provide yEnc support for either news or mail, but there are plug-ins available. Mozilla Thunderbird will decode single-part yEnc files, but is not able to combine multi-part binaries.[4]

Problems

Many programmers and news admins have outlined the weaknesses of yEnc.[5][6][7][8] It suffers from many of the same flaws as uuencode does, a number of which had already been solved years before by MIME (which addressed the same flaws in uuencode). For example, yEnc requires the strings "=ybegin" and "=yend" to be placed around the encoded file in the message body.[3] Although this is an improvement over uuencode's "begin" and "end", which occur more frequently in normal text, message readers can still encounter attachments where those strings are present (most frequently in discussions about yEnc itself). yEnc and uuencode also attempt to reassemble files split into multiple messages by using the subject line, which is unreliable.

Moreover, yEnc adds a few new flaws of its own. It attempts to turn unstructured fields into structured ones, which is unreliable, given that no constraints can be placed upon the unstructured use of the fields by non-yEnc uses. Most notably, the subject line of the message is supposed to contain the string "yEnc", the filename, and the part number. (The yEnc homepage chastises yEnc article posters for themselves not observing these constraints.) MIME places all such information in the message headers, which is far more reliable.

Uuencode was careful to support Internet messages as streams of text, which yEnc does not support. Software that supports yEnc encoding must know the size of the original file in advance, because the file size is specified in the yEnc header that precedes the encoded file.

Not all transports can handle the 8-bit characters employed by yEnc, which may cause data corruption. yEnc can also be mangled by different character sets. It works poorly with the increasingly popular UTF-8 character set, for instance. Moreover, some article transports may, on the grounds of enforcing compliance with the Internet message format standard, automatically convert any message using 8-bit characters to either Base64 or quoted-printable, entirely nullifying the overhead advantage.

Critics also take issue with the lack of formal standardization.

Some have suggested including yEnc as part of MIME, which would solve nearly all of its problems and retain the low encoding overhead. However, no formal or informal standard has been reached.

yEncode adoption

The yEncode draft proposal document was made available on 31 July 2001.[9] A reference encoder and decoder was included in the MyNews 1.9 freeware version in November that year.[10] yDec, a freeware win32 decoder came on 14 November 2001. On 21 March 2002, Agent supported yEnc with version 1.91.[11][12] Due to feedback of Juergen Helbing, the release was postponed by one week.[13][14] A couple of days after the release Jürgen Helbing wrote that Forté implemented yEnc in the best way imaginable.[15]

Stuffit Deluxe added yEnc support with version 8.0 in 2003.[16][17] PowerArchiver 9.2 added yEnc support in May 2005.[18]

See also

References

  1. ↑ Helbing, Juergen (2002-02-28). "yEncode - A quick and dirty encoding for binaries". Retrieved 2014-07-26.
  2. ↑ Fellows, G. (2006). "Newsgroups reborn – The binary posting renaissance". Digital Investigation 3 (2): 73–78. doi:10.1016/j.diin.2006.04.006. ISSN 1742-2876.
  3. ↑ 3.0 3.1 Kim, Juhoon; Schneider, Fabian; Ager, Bernhard; Feldmann, Anja (2010). "Today's Usenet Usage: NNTP Traffic Characterization". pp. 1–6. doi:10.1109/INFCOMW.2010.5466665.
  4. ↑ "Yenc support in Thunderbird?". org.mozilla.lists.support-thunderbird (Mailing list). 2006-05-18.
  5. ↑ Helbing, Jürgen (2002-06-10). "Opponents to yEnc". yenc.org. Archived from the original on 2013-08-28.
  6. ↑ Nixon, Jeremy (2002-03-17). "Why yEnc is bad for Usenet". Archived from the original on 2013-08-29.
  7. ↑ Welch, Curt (2002-09-19). "What's wrong with yEnc?". Archived from the original on 2013-08-11.
  8. ↑ Färber, Claus (2002-03-04). "yEnc considered harmful".
  9. ↑ "The original draft yEnc 1.0". 2001-07-31.
  10. ↑ Juergen "The Archiver" Helbing. "New features in 1.9". winews.net.
  11. ↑ "Agent 1.91 is Released". forteinc.com. Agent 1.91 provides full support for yEnc, a new Usenet encoding algorithm for binaries.
  12. ↑ Spanbauer, Scott (August 2002). "Revision control - Latest Software Tweaks (Listen to a world of radio stations on the Internet)". PC World 20 (8): 138–139. Version 1.92 of Forté's Usenet newsreader adds a trash folder, improves some existing features, and takes care of various bugs; but more important than the fixes and enhancements is the application's added support for the YEnc binary encoding algorithm.
  13. ↑ "Agent 1.91 needs one more week". Forté. 2002-03-15.
  14. ↑ "Juergen Helbing's feedback on yEnc and Agent 1.91". Forté. 2002-03-17.
  15. ↑ Helbing, Jürgen (2002-03-22). "Forte Agent 1.91 supports yEnc".
  16. ↑ Sellers, Dennis (2003-09-22). "StuffIt Deluxe 8.0 gets new plug-ins, performance boost". Macworld.
  17. ↑ Breen, Christopher (July 2004). "Stufflt Deluxe 8.0". Macworld 21 (7): 40.
  18. ↑ Richard V. Dragan (2005-05-04). "File Compression: PowerArchiver 9.2".

External links