MPQ

From Wikipedia, the free encyclopedia

MPQ (Mo'PaQ, short for Mike O'Brien Pack, named after its creator), is an archiving file format used in several of Blizzard Entertainment's games.

MPQs used in Blizzard's games generally contain a game's data files, including graphics, sounds, and level data. The format's capabilities include compression, encryption, file segmentation, extensible file metadata, cryptographic signature and the ability to store multiple versions of the same file for internationalization and platform-specific differences. MPQ archives can use a variety of compression algorithms which may also be combined.

Contents

[edit] File indexing

In order to meet the requirements of speed generally demanded by a computer game, files are indexed in a hash table using a quick, low-collision hashing algorithm. The index of a specific file within the hash table is the hash of the uppercased filename modulo the size of the hash table, allowing for quick verification of a file's existence within the archive. If multiple files within the archive have the same hash, colliding entries will follow each other in increasing index order (forming a colliding hash cluster). In order to identify the exact entry for the requested file within a colliding hash cluster, each hash table entry stores 2 additional hashes of the uppercased filename, each using the same hashing algorithm but with a different seed value, as well as a locale code and platform code. The end of a colliding hash cluster is detected either by encountering an empty hash table entry or by traversing the entire hash table (including the modulo loopback) back to the initial hash table index.

[edit] Archive metadata

MPQ archives do not have specific structures to store metadata beyond what is absolutely necessary to access archived files. Instead, the convention is to use regular files whose filename is enclosed by parentheses.

Below are known metadata files.

  • (listfile): Contains a list of the archive's files, one filename per line. May or may not be exhaustive.
  • (signature): Contains the weak cryptographic signature of the archive. This type of signature is deprecated.
  • (attributes): Contains extended file metadata. Currently known attributes are file creation date, CRC32 checksum and MD5 checksum.

[edit] Compression

In modern MPQ archives, each segment (or sector) of a file can be compressed using a combination of compression algorithms. A header byte is prepended to every compressed sector to indicate which compressions were used. The order in which those compressors are applied is hardcoded.

The following algorithms are currently in use by Blizzard games:

  • PKZIP (licensed from PKWARE). The first compression algorithm available.
  • Huffman tree compression (introduced in StarCraft). Always combined with ADPCM compression.
  • ADPCM 4:1 compression (introduced in StarCraft). Always combined with Huffman tree compression. This algorithm is lossy and only suitable for raw PCM input data.
  • zlib (introduced in Warcraft III).
  • bzip2 (introduced in World of Warcraft).

Since there was only one compression algorithm available when MPQs were first deployed in Diablo 1, those archives used a different archive file metadata flag to indicate compression and did not use a compression header byte.

[edit] Warcraft III cinematics

Cutscene cinematics with the MPQ extension are included with Warcraft III and, despite the file extension, are not actual MPQ files. Rather they are AVI files compressed with Blizzard's renamed MPEG-4 codec, BLZ0 (which actually is DivX). These files are playable in ordinary media players, provided the proper codecs are installed. It is not known why Blizzard used the MPQ extension on those files. In addition, World of Warcraft cinematics use the AVI extension.

DBC (WDBC) files are collections of data records used in World of Warcraft's client-side database. They are found within the MPQ files in the game. .DBC files organize the client-side information necessary to run the game correctly[1]

[edit] Usage in gaming

Blizzard has utilized the MPQ file format for archiving game files in a number of their games and a few other games, including:

[edit] Icons

The following two icons represent MPQ files on Mac OS X with Warcraft III and World of Warcraft installed. These icons are associated with the .MPQ file extension upon installation of the Mac OS X versions of their respective games. The Windows OS versions of these games do not associate any icon with the file extensions.

[edit] References

  1. ^ DBC - WoWDev

[edit] External links

[edit] Original Site

[edit] Specifications

[edit] Libraries

  • libmpq is a cross-platform C library (and associated tools) for extracting files from MPQ archives.
  • jlibmpq java wrapper for libmpq.
  • MPQKit is a Mac OS X Objective-C framework for manipulating MPQ archives. It is the only third-party library that supports signature verification and full support for extended MPQ archives (introduced in The Burning Crusade).

[edit] Programs

  • MPQDraft, an open source program hosted on Sourceforge that allows users to work with MPQ files.
  • MPQ Extractor — A Mac OS X extractor utility.
  • MPQ Archives — Overview, background, MPQEditor, a .MPQ file editor for Windows, Storm.dll, StormLib, libraries for reading and writing .MPQ files.
v • d • e
The Warcraft series and related topics
Games
Books
Professional
Competition
Other