ZMODEM
ZMODEM is a file transfer protocol developed by Chuck Forsberg in 1986, in a project funded by Telenet in order to improve file transfers on their X.25 network. In addition to dramatically improved performance compared to older protocols, ZMODEM also offered restartable transfers, auto-start by the sender, an expanded 32-bit CRC, and control character quoting, allowing it to be used on networks that might "eat" control characters. ZMODEM became extremely popular on bulletin board systems (BBS) in the early 1990s, displacing earlier protocols such as XMODEM and YMODEM.
The key improvement in ZMODEM was the introduction of sliding window support for improved performance. Generally file transfer protocols break down a file into a series of packets, and then send them one-at-a-time to the receiver. If the packet is received correctly an ACK message is sent and the sender then starts sending the next packet.
However, the telephone system introduces a small delay known as latency that interferes with this process. Even if the receiver sends the ACK immediately, the delay in the phone lines means there will always be some time before the sender receives it and sends the next packet. As modem speeds increase, this delay represents a larger and larger number of packets that could be sent during the delay, decreasing the overall performance.
Sliding window protocols avoid this problem by allowing the sending machine to move on to the next packet without waiting for an ACK. Instead, the receiver sends both an ACK (or NAK if there was an error) along with the packet number it is confirming. The sender can process these at its leisure, re-sending packets as required when need be. This effectively reduces the latency to zero at the cost of the very small overhead data. This is similar to TCP's approach to sending ACKs.
ZMODEM's performance was so improved over previous common protocols that it generally replaced even special protocols such as YMODEM-g, which included no error correction at all and instead relied on error-free links maintained by the modems. Although YMODEM-g was noticeably faster, the lack of other features such as restartable transfers made it less interesting.
Auto-starting simplified management by allowing the sending machine to start the transfer. Previously the user had to first request the file from the sender, placing it into a "waiting" state, then return to their local programs and invoke a command to start the transfer. With auto-transfer, they simply requested the file, the sender would then automatically trigger the transfer in the user's program.
Restarts were another addition, allowing transfers to be restarted after the line was dropped or the user ran out of time and was kicked off.
A number of modified versions of ZMODEM appeared. ZedZap was a variant of ZMODEM with 8 kbyte blocks for better performance on high-speed modems. LeechZmodem was a mischievous ZMODEM variant (among similar XMODEM and YMODEM derivatives) that cheated BBS download quotas. A backwards compatible extension of ZMODEM with 32 kbyte and 64 kbyte block lengths was created by ADONTEC in 2002 and 2007 to increase performance on high-speed error free connections like ISDN or TCP/IP networks (intranet, internet). Both extensions are backward compatible.
The most notable ZMODEM implementations were from Chuck Forsberg's Omen Technology, Inc. These included DSZ (DOS Send ZMODEM), GSZ (Graphical Send ZMODEM), and the ubiquitous (l)rzsz for Unix variants.
In more current times, the developers of Synchronet have created a modern X/Y/ZMODEM implementation named SEXYZ, loosely based on the zmtx/zmrx package, which runs natively on Windows and Unix variants, supports long filenames and faster, more reliable data transfers. The ZMODEM implementation from SEXYZ has also been incorporated into the SyncTERM project. Synchronet, SEXYZ, and SyncTERM are all open-source, cross-platform, BBS-centric projects.
Forsberg himself collected a number of improvements into ZMODEM-90. The first of these is MobyTurbo, which removed control quoting to further improve performance, about 15%. Even on networks that "eat" control characters, ZMODEM-90 can be tailored to quote only those characters the network actually eats, as opposed to every possible one. A similar improvement allows ZMODEM-90 to work on 7-bit networks, whereas earlier protocols (with the notable exception of Kermit) had all demanded 8-bits to one degree or another. Finally, ZMODEM-90 includes a basic Run-length encoding compression system to further improve performance on uncompressed files.
Limitations
- Some of the ZMODEM packets (e.g. ZACK, ZRPOS) embed a byte-offset within the transferred file as a 32-bit unsigned integer. This design limits the feasibility of ZMODEM to only reliably transfer files that are under 4GB in size.
- Even though the protocol could permit it, the reference (l)rzsz implementation cannot encode arbitrary non-control characters (e.g. '~') which are often used by TCP/IP connection programs like telnet and ssh as client-side "terminal escape" characters. Users must disable the terminal escape feature to achieve reliable transfers over these kinds of links, e.g. ssh -e none user@hostname.
References
- The ZMODEM Inter Application File Transfer Protocol
- Synchronet External X/Y/Zmodem (SEXYZ) File Transfer Protocol Driver