From 3935fdfd4791e3c13822661ba400723ddaef127a Mon Sep 17 00:00:00 2001 From: thomaslaurenson Date: Tue, 28 Apr 2026 20:10:13 +1200 Subject: [PATCH 1/4] Added mdbook implementation for website-based docs --- .github/workflows/docs.yml | 53 +++++++ book.toml | 9 ++ docs/SUMMARY.md | 19 +++ docs/advanced.md | 50 +++++++ docs/building.md | 59 ++++++++ docs/changelog.md | 283 +++++++++++++++++++++++++++++++++++++ docs/commands.md | 3 + docs/commands/about.md | 15 ++ docs/commands/add.md | 69 +++++++++ docs/commands/create.md | 50 +++++++ docs/commands/extract.md | 51 +++++++ docs/commands/info.md | 37 +++++ docs/commands/list.md | 89 ++++++++++++ docs/commands/read.md | 46 ++++++ docs/commands/remove.md | 21 +++ docs/commands/verify.md | 27 ++++ docs/commands/version.md | 8 ++ docs/commands_list.md | 19 +++ docs/contributing.md | 123 ++++++++++++++++ docs/installation.md | 41 ++++++ docs/introduction.md | 16 +++ docs/js/copy-command.js | 110 ++++++++++++++ 22 files changed, 1198 insertions(+) create mode 100644 .github/workflows/docs.yml create mode 100644 book.toml create mode 100644 docs/SUMMARY.md create mode 100644 docs/advanced.md create mode 100644 docs/building.md create mode 100644 docs/changelog.md create mode 100644 docs/commands.md create mode 100644 docs/commands/about.md create mode 100644 docs/commands/add.md create mode 100644 docs/commands/create.md create mode 100644 docs/commands/extract.md create mode 100644 docs/commands/info.md create mode 100644 docs/commands/list.md create mode 100644 docs/commands/read.md create mode 100644 docs/commands/remove.md create mode 100644 docs/commands/verify.md create mode 100644 docs/commands/version.md create mode 100644 docs/commands_list.md create mode 100644 docs/contributing.md create mode 100644 docs/installation.md create mode 100644 docs/introduction.md create mode 100644 docs/js/copy-command.js diff --git a/.github/workflows/docs.yml b/.github/workflows/docs.yml new file mode 100644 index 0000000..a477fcc --- /dev/null +++ b/.github/workflows/docs.yml @@ -0,0 +1,53 @@ +name: docs + +on: + push: + branches: + - main + paths: + - 'docs/**' + - 'book.toml' + +# Allow only one concurrent deployment +concurrency: + group: "pages" + cancel-in-progress: false + +# Required permissions for GitHub Pages deployment +permissions: + contents: read + pages: write + id-token: write + +jobs: + build: + runs-on: ubuntu-24.04 + steps: + - name: Checkout + uses: actions/checkout@v6 + + - name: Install mdBook + run: | + mkdir -p $HOME/.local/bin + curl -sSL https://github.com/rust-lang/mdBook/releases/download/v0.5.2/mdbook-v0.5.2-x86_64-unknown-linux-gnu.tar.gz \ + | tar -xz --directory=$HOME/.local/bin + echo "$HOME/.local/bin" >> $GITHUB_PATH + + - name: Build docs + run: mdbook build + + - name: Upload Pages artifact + uses: actions/upload-pages-artifact@v3 + with: + path: book/ + + deploy: + needs: build + runs-on: ubuntu-24.04 + environment: + name: github-pages + url: ${{ steps.deployment.outputs.page_url }} + steps: + - name: Deploy to GitHub Pages + id: deployment + uses: actions/deploy-pages@v4 diff --git a/book.toml b/book.toml new file mode 100644 index 0000000..7c30575 --- /dev/null +++ b/book.toml @@ -0,0 +1,9 @@ +[book] +title = "mpqcli" +description = "A command-line tool to create, add, remove, list, extract, read, and verify MPQ archives." +src = "docs" + +[output.html] +git-repository-url = "https://github.com/thegraydot/mpqcli" +git-repository-icon = "fab-github" +additional-js = ["docs/js/copy-command.js"] diff --git a/docs/SUMMARY.md b/docs/SUMMARY.md new file mode 100644 index 0000000..9e02607 --- /dev/null +++ b/docs/SUMMARY.md @@ -0,0 +1,19 @@ +# Summary + +- [Introduction](./introduction.md) +- [Installation](./installation.md) +- [Commands](./commands_list.md) + - [version](./commands/version.md) + - [about](./commands/about.md) + - [info](./commands/info.md) + - [create](./commands/create.md) + - [add](./commands/add.md) + - [remove](./commands/remove.md) + - [list](./commands/list.md) + - [extract](./commands/extract.md) + - [read](./commands/read.md) + - [verify](./commands/verify.md) +- [Advanced Examples](./advanced.md) +- [Building](./building.md) +- [Contributing](./contributing.md) +- [Changelog](./changelog.md) diff --git a/docs/advanced.md b/docs/advanced.md new file mode 100644 index 0000000..a44118a --- /dev/null +++ b/docs/advanced.md @@ -0,0 +1,50 @@ +# Advanced Examples + +## Search and extract files on Linux + +The `mpqcli` tool has no native search feature - instead, it is designed to be integrated with other, external operating system tools. For example, `mpqcli list` can be "piped" to `grep` in Linux or `Select-String` in Windows PowerShell to perform searching. + +The following command lists all files in an MPQ archive, and each filename is filtered using `grep` - selecting files ending in `.exe` (note: `.` is a regex wildcard matching any character; use `\.exe` for a strictly literal match) - which is then passed back to `mpqcli extract`. The result: search and extract all `exe` files. + +```bash +$ mpqcli list wow-patch.mpq | grep -i \.exe | xargs -I@ mpqcli extract -f "@" wow-patch.mpq +[*] Extracted: Launcher.exe +[*] Extracted: BackgroundDownloader.exe +[*] Extracted: WoW.exe +[*] Extracted: BNUpdate.exe +[*] Extracted: Repair.exe +[*] Extracted: WowError.exe +``` + +Note that directories are specified with backslashes, which need to be escaped (due to how grep and xargs handles them). The following example extracts all `dat` files in the `arr` directory from the `StarDat.mpq` archive and handles escaping via `sed`. + +```bash +$ mpqcli list -l scbw.txt StarDat.mpq | grep -i "arr\\\.*dat$" | sort | sed 's|\\|\\\\|g' | \ + xargs -I@ mpqcli extract -f "@" -k StarDat.mpq +[*] Extracted: arr/flingy.dat +[*] Extracted: arr/images.dat +[*] Extracted: arr/mapdata.dat +[*] Extracted: arr/orders.dat +[*] Extracted: arr/portdata.dat +[*] Extracted: arr/sfxdata.dat +[*] Extracted: arr/sprites.dat +[*] Extracted: arr/techdata.dat +[*] Extracted: arr/units.dat +[*] Extracted: arr/upgrades.dat +[*] Extracted: arr/weapons.dat +``` + +## Search and extract files on Windows + +The following command lists all files in an MPQ archive, and each filename is filtered using `Select-String` - selecting files ending in `.exe` (note: `.` is a regex wildcard matching any character; use `\.exe` for a strictly literal match) - which is then passed back to `mpqcli extract`. The result: search and extract all `exe` files. + +```powershell +PS> mpqcli.exe list wow-patch.mpq | Select-String -Pattern \.exe | ` + ForEach-Object { mpqcli.exe extract -f $_ wow-patch.mpq } +[*] Extracted: Launcher.exe +[*] Extracted: BackgroundDownloader.exe +[*] Extracted: WoW.exe +[*] Extracted: BNUpdate.exe +[*] Extracted: Repair.exe +[*] Extracted: WowError.exe +``` diff --git a/docs/building.md b/docs/building.md new file mode 100644 index 0000000..4d23b07 --- /dev/null +++ b/docs/building.md @@ -0,0 +1,59 @@ +# Building + +## Requirements + +- cmake +- C++ 17 compiler +- StormLib (provided as Git Submodule) +- CLI11 (provided as Git Submodule) + +## Linux + +```bash +$ git clone --recursive https://github.com/TheGrayDot/mpqcli.git +$ cd mpqcli +$ cmake -B build +$ cmake --build build +``` + +The `mpqcli` binary will be available in: `./build/bin/mpqcli` + +## Windows + +```bash +$ git clone --recursive https://github.com/TheGrayDot/mpqcli.git +$ cd mpqcli +$ cmake -B build +$ cmake --build build --config Release +``` + +The `mpqcli.exe` binary will be available in: `.\build\bin\Release\mpqcli.exe` + +## Dependencies + +### StormLib + +This project requires the [StormLib](https://github.com/ladislav-zezula/StormLib) library. Many thanks to [Ladislav Zezula](https://github.com/ladislav-zezula) for authoring such a good library and releasing the code under an open-source license. The StormLib library has a number of requirements. However, the build method specifies using the libraries bundled with StormLib. + +### CLI11 + +This project also uses the [CLI11](https://github.com/CLIUtils/CLI11) command line parser for C++11 and beyond. It provides simple and easy-to-use CLI arguments. + +## Tests + +This project implements End-to-end (E2E) testing, sometimes referred to as system testing or integration testing. This methodology is used because it verifies application functionality by simulating actual usage by an end user. Testing includes creating a variety of MPQ archives, as well as dynamically downloading some small (~1-5MB) MPQ archives from the Internet Archive. The Python programming language coupled with the [pytest framework](https://github.com/pytest-dev/pytest) is used to implement testing, mainly due to ease of implementation. + +To configure the testing environment you will need Python installed, as well as the required `pytest` package. On a Debian-based Linux system, the following will configure the environment: + +```bash +$ sudo apt install python3-venv python3-pip +$ python3 -m venv test/.venv +$ source test/.venv/bin/activate +$ pip3 install -r test/requirements.txt +``` + +Then you can run the tests using: + +```bash +$ python3 -m pytest test -s +``` diff --git a/docs/changelog.md b/docs/changelog.md new file mode 100644 index 0000000..474e4d2 --- /dev/null +++ b/docs/changelog.md @@ -0,0 +1,283 @@ +# Changelog + +## 0.9.9 - 2026-04-05 + +### Fixed + +- Potential path traversal attack in extract subcommand +- Empty MPQ archive name when directory supplied with trailing slash +- Bug where MPQ internal files were added with create command +- Signature being added automatically when not requested +- Potential DWORD overflow in `AddFiles` + +## 0.9.8 - 2026-03-22 + +### Added + +- The `create` subcommand now supports a single file +- The `add` and `create` subcommands have additional arguments for specifiying the file location in the MPQ archive (`name-in-archive` and `dir-in-archive`) +- The `add` subcommand can now overwrite existing files if requested (`overwrite`) + +### Fixed + +- Several memory leaks + +### Thanks + +- Thanks to @sjoblomj for adding more features and fixes in this release + +## 0.9.7 - 2026-02-23 + +### Added + +- Game profile support for `create` and `add` subcommands via `-g/--game` option +- Game profiles for Diablo, StarCraft, Warcraft, WoW, and more +- Game setting override options for `create` subcommand +- Compression override options for `add` subcommand +- Support for unfamiliar/unknown locales via raw hex LCID +- Additional built-in locales: `ptBR`, `esMX`, `ptPT` + +### Fixed + +- Subcommands now properly verify locale before operating on files +- Error messages now include locale context in `remove`, `extract`, and `read` +- MPQ archives are now properly closed across all subcommands +- Non-zero exit codes returned on failure in `extract` and `remove` +- `list` subcommand now shows long output when properties are requested + +### Changed + +- Improved locale printing and display throughout + +### Thanks + +- A big thank you to @sjoblomj for adding so many features in this release + +## 0.9.6 - 2025-12-23 + +### Added + +- Proper support for locales in MPQ files + +### Fixed + +- Fixed bug where MPQ signature only printed on success + +## 0.9.5 - 2025-10-23 + +### Changed + +- Long (detailed) list format to output ISO 8601 timestamps + +### Fixed + +- Message prefix format consistency + +## 0.9.4 - 2025-10-12 + +### Added + +- Filename added to program output +- Added simple wrapper around verify command + +### Fixed + +- Show help menu when no arguments provided + +## 0.9.3 - 2025-10-03 + +### Added + +- Support for `add` subcommand to have path inside MPQ archive + +### Fixed + +- Bug in `add` subcommand when exceeding max file count + +## 0.9.2 - 2025-08-21 + +### Fixed + +- Bug in `create` subcommand resulting in incorrect max file count + +## 0.9.1 - 2025-08-19 + +### Fixed + +- Bug in Dockerfiles resulting in missing commit hash from version + +## 0.9.0 - 2025-08-19 + +### Added + +- Linux binaries for amd64 and arm64 +- Linux binaries built with musl and glibc + +### Updated + +- Install script for Linux to provided better binary download options + +### Changed + +- Linux binary release naming convention + +## 0.8.1 - 2025-08-15 + +### Added + +- Added print raw bytes for `read` and `verify` subcommands + +### Fixed + +- Tests for `read` and `verify` subcommands + +### Removed + +- Ability to print hex or plaintext in `read` and `verify` subcommands +- Removed messages when printing signature value + +## 0.8.0 - 2025-08-11 + +### Added + +- Added `add`/`remove` subcommands working +- Added tests for `add`/`remove` subcommands +- Added `max-files` property to `info` subcommand + +### Changed + +- Change for default maximum file count when creating MPQ archives + +## 0.7.0 - 2025-08-09 + +### Added + +- Added detailed (long) listing + +### Fixed + +- Updated create subcommand to always include attributes +- Updated Info function to support archive/file property fetching + +## 0.6.2 - 2025-07-11 + +### Added + +- Added tests for `extract` subcommand +- Added dependabot configuration for Python and Docker + +### Fixed + +- Fixed deprecated Windows Action running to Server 2022 + +## 0.6.1 - 2025-06-27 + +### Fixed + +- Updated readme and help menu to match application functionality + +## 0.6.0 - 2025-06-02 + +### Added + +- Added the `read` subcommand +- Added tests for `read` subcommand +- Added new helper functions to print ASCII or hexidecimal output + +### Fixed + +- Fixed line endings in Windows pytest cases + +## 0.5.0 - 2025-05-05 + +### Added + +- Added the `verify` subcommand +- Included argument to print digital signature +- Added ability to sign archive when creating MPQ +- Added tests for `verify` subcommand + +## 0.4.4 - 2025-05-05 + +### Fixed + +- Refactored `create` subcommand tests + +## 0.4.3 - 2025-05-01 + +### Added + +- Refactored `info` subcommand +- Added test for `info` subcommand + +### Fixed + +- Updated tests to be multi-platform (Windows) + +### Removed + +- Removed exe patch split functionality + +## 0.4.2 - 2025-04-22 + +### Added + +- Added Dockerfile +- Added Docker build and release workflows +- Added `add` and `remove` subcommand placeholders +- Added `about` subcommand + +## 0.4.1 - 2025-04-22 + +### Added + +- Added Python testing environment +- Added tests for `list` subcommand +- Added test workflow + +## 0.4.0 - 2025-04-12 + +### Added + +- Added Windows binary build to releases + +## 0.3.2 - 2025-03-29 + +### Added + +- External listfile support for `list` and `extract` by @sjoblomj +- Added functions for add and remove files + +### Fixed + +- Fixed path extraction on Linux to create nested folder structure + +## 0.3.1 - 2025-03-03 + +### Fixed + +- Fixed path extraction on Linux to create nested folder structure + +## 0.3.0 - 2025-02-24 + +### Added + +- Added `create` subcommand +- Added download information to readme + +### Fixed + +- Improved Linux binary compatibility + +## 0.2.3 - 2025-02-01 + +### Fixed + +- Improved Linux binary compatibility + +## 0.2.2 - 2025-02-01 + +### Added + +- Added the `extract`, `verify` and `list` subcommands +- Added `release.yml` workflow for Linux binary diff --git a/docs/commands.md b/docs/commands.md new file mode 100644 index 0000000..9bf37da --- /dev/null +++ b/docs/commands.md @@ -0,0 +1,3 @@ +# Commands + +- `add` \ No newline at end of file diff --git a/docs/commands/about.md b/docs/commands/about.md new file mode 100644 index 0000000..bf9393d --- /dev/null +++ b/docs/commands/about.md @@ -0,0 +1,15 @@ +# about + +Print information about the tool. + +```bash +$ mpqcli about +Name: mpqcli +Version: 0.9.8-041480a92e698514d7938426587e93582b336b7d +Author: Thomas Laurenson +License: MIT +GitHub: https://github.com/TheGrayDot/mpqcli +Dependencies: + - StormLib (https://github.com/ladislav-zezula/StormLib) + - CLI11 (https://github.com/CLIUtils/CLI11) +``` diff --git a/docs/commands/add.md b/docs/commands/add.md new file mode 100644 index 0000000..b5534c5 --- /dev/null +++ b/docs/commands/add.md @@ -0,0 +1,69 @@ +# add + +Add a file to an existing MPQ archive. + +## Add a file to an existing archive + +Add a local file to an already existing MPQ archive. + +```bash +$ echo "For The Horde" > fth.txt +$ mpqcli add fth.txt wow-patch.mpq +[+] Adding file: fth.txt +``` + +Alternatively, you can add a file under a specific file name using the `-f` or `--filename-in-archive` argument. + +```bash +$ echo "For The Alliance" > fta.txt +$ mpqcli add fta.txt wow-patch.mpq --filename-in-archive "alliance.txt" +[+] Adding file: alliance.txt +``` + +Alternatively, you can add a file to a specific subdirectory using the `-d` or `--directory-in-archive` argument. + +```bash +$ echo "For The Swarm" > fts.txt +$ mpqcli add fts.txt wow-patch.mpq --directory-in-archive texts +[+] Adding file: texts\fts.txt +``` + +Alternatively, you can add a file under a specific directory and filename using the `-p` or `--path` argument. + +```bash +$ echo "For The Swarm" > fts.txt +$ mpqcli add fts.txt wow-patch.mpq --path "texts\swarm.txt" +[+] Adding file: texts\swarm.txt +``` + +To overwrite a file in an MPQ archive, set the `-w` or `--overwrite` flag: + +```bash +$ echo "For The Horde" > allegiance.txt +$ mpqcli add allegiance.txt wow-patch.mpq +[+] Adding file: allegiance.txt +$ echo "For The Alliance" > allegiance.txt +$ mpqcli add allegiance.txt wow-patch.mpq +[!] File already exists in MPQ archive: allegiance.txt - Skipping... +$ mpqcli add allegiance.txt wow-patch.mpq --overwrite +[+] File already exists in MPQ archive: allegiance.txt - Overwriting... +[+] Adding file: allegiance.txt +``` + +## Add a file to an MPQ archive with a given locale + +Use the `--locale` argument to specify the locale that the added file will have in the archive. Note that subsequent added files will have the default locale unless the `--locale` argument is specified again. + +```bash +$ mpqcli add allianz.txt wow-patch.mpq --locale deDE +[+] Adding file for locale deDE: allianz.txt +``` + +## Add a file with game-specific properties + +Target a specific game version by using the `-g` or `--game` argument. This will automatically set the correct encryption rules and MPQ flags, although they can be overridden. + +```bash +$ mpqcli add khwhat1.wav archive.mpq --game wc2 +[+] Adding file: khwhat1.wav +``` diff --git a/docs/commands/create.md b/docs/commands/create.md new file mode 100644 index 0000000..d517e42 --- /dev/null +++ b/docs/commands/create.md @@ -0,0 +1,50 @@ +# create + +Create an MPQ archive from a target directory or a single file. + +## Create an MPQ archive from a target directory + +Create an MPQ file from a target directory. Automatically adds `(listfile)` to the archive, and will skip this file if it exists in the target directory. + +```bash +$ mpqcli create +``` + +The default mode of operation for the `create` subcommand is to take everything from the "target" directory (and below) and recursively add it to the archive. The directory structure is retained. Windows-style backslash path separators are used (`\`), as per the observed behavior in most MPQ archives. + +## Create an MPQ archive for a specific game + +Target a specific game version by using the `-g` or `--game` argument. This will automatically set the correct archive format version and settings, although they can be overridden. + +```bash +$ mpqcli create -g starcraft +$ mpqcli create --game wow-wotlk \ + --sector-size 16384 \ + --version 3 +``` + +## Create an MPQ archive from a single file + +Create an MPQ file from a single file. + +```bash +$ mpqcli create --game diablo2 +``` + +This will put the given file in the root of the MPQ archive. By optionally providing a path in the `--name-in-archive` parameter, the name that the file has in the MPQ archive can be changed, and it can be put in a directory. + +## Create and sign an MPQ archive + +Use the `-s` or `--sign` argument to cryptographically sign an MPQ archive with the Blizzard weak signature. + +```bash +$ mpqcli create --version 1 --sign +``` + +## Create an MPQ archive with a given locale + +Use the `--locale` argument to specify the locale that all added files will have in the archive. Note that subsequent added files will have the default locale unless the `--locale` argument is specified again. + +```bash +$ mpqcli create --locale koKR +``` diff --git a/docs/commands/extract.md b/docs/commands/extract.md new file mode 100644 index 0000000..9efc885 --- /dev/null +++ b/docs/commands/extract.md @@ -0,0 +1,51 @@ +# extract + +Extract one or all files from a target MPQ archive. + +## Extract all files from an MPQ archive + +The output will be saved in a folder with the same name as the target MPQ file, without the extension. + +```bash +$ mpqcli extract wow-patch.mpq +[*] Extracted: BM_COKETENT01.BLP +[*] Extracted: Blizzard_CraftUI.xml +[*] Extracted: CreatureSoundData.dbc +... +[*] Extracted: Blizzard_CraftUI.lua +[*] Extracted: 30ee7bd3959906e358eff01332cf045e.blp +[*] Extracted: realmlist.wtf +``` + +## Extract all files to a target directory + +Extract files to a specific target directory, which will be created if it doesn't already exist. In this example, `patch-1.10` is the user-specified output directory. + +```bash +$ mpqcli extract -o patch-1.10 wow-patch.mpq +``` + +## Extract all files with an external listfile + +Older MPQ archives do not contain (complete) file paths of their content. By providing an external listfile that lists the content of the MPQ archive, the extracted files will have the correct names and paths. Listfiles can be downloaded on [Ladislav Zezula's site](http://www.zezula.net/en/mpq/download.html). + +```bash +$ mpqcli extract -l path/to/listfile War2Dat.mpq +``` + +## Extract one specific file + +Extract a single file using the `-f` option. If the target file in the MPQ archive is nested (in a directory) you need to include the full path. Similar to the examples above, you can use the `-o` argument to specify the output directory. + +```bash +$ mpqcli extract -f "Documentation\Layout\Greeting.html" \ + "World of Warcraft_1.12.1.5875/Data/base.MPQ" +``` + +## Extract one specific file with locale + +Use the `--locale` argument to specify the locale of the file to extract. If there is no file with the requested name and locale, the default locale will be used instead. + +```bash +$ mpqcli extract -f "rez\gluBNRes.res" Patch_rt.mpq --locale deDE +``` diff --git a/docs/commands/info.md b/docs/commands/info.md new file mode 100644 index 0000000..5d726bd --- /dev/null +++ b/docs/commands/info.md @@ -0,0 +1,37 @@ +# info + +Print information about MPQ archive properties. + +## Print information about an MPQ archive + +The `info` subcommand prints a list of useful information (property keys and values) of an MPQ archive. + +```bash +$ mpqcli info wow-patch.mpq +Archive size: 1798918 +File count: 65 +Format version: 1 +Header offset: 0 +Header size: 32 +Max files: 128 +Signature type: Weak +``` + +## Print one specific MPQ archive property + +The `info` subcommand supports the following properties: + +- `archive-size` +- `file-count` +- `format-version` +- `header-offset` +- `header-size` +- `max-files` +- `signature-type` + +You can use the `-p` or `--property` argument with the `info` subcommand to print just the value of a specific property. This can be useful for automation, for example, to determine the signature type of a directory of MPQ archives. + +```bash +$ mpqcli info -p file-count wow-patch.mpq +65 +``` diff --git a/docs/commands/list.md b/docs/commands/list.md new file mode 100644 index 0000000..0048326 --- /dev/null +++ b/docs/commands/list.md @@ -0,0 +1,89 @@ +# list + +List files in a target MPQ archive. + +## List all files in an MPQ archive + +Pretty simple, list files in an MPQ archive. Useful to "pipe" to other tools, such as `grep` (see the [Advanced Examples](../advanced.md) page). + +```bash +$ mpqcli list wow-patch.mpq +BM_COKETENT01.BLP +Blizzard_CraftUI.xml +CreatureSoundData.dbc +... +Blizzard_CraftUI.lua +30ee7bd3959906e358eff01332cf045e.blp +realmlist.wtf +``` + +## List all files with detailed output + +Similar to the `ls` command with the `-l` and `-a` options, additional detailed information can be included with the `list` subcommand. The `-a` option includes printing "special" files used in MPQ archives including: `(listfile)`, `(attributes)` and `(signature)`. + +```bash +$ mpqcli list -d -a wow-patch.mpq + 88604 enUS 2006-03-29 02:02:37 BM_COKETENT01.BLP + 243 enUS 2006-04-04 21:28:14 Blizzard_CraftUI.xml + 388 enUS 2006-03-29 19:32:46 CreatureSoundData.dbc + ... + 184 enUS 2006-04-04 21:28:14 Blizzard_CraftUI.lua + 44900 enUS 2006-03-29 02:01:02 30ee7bd3959906e358eff01332cf045e.blp + 68 enUS 2006-04-07 00:58:44 realmlist.wtf +``` + +## List specific properties + +The `list` subcommand supports listing the following properties: + +- `hash-index` - Index in the hash table where the file entry is. +- `name-hash1` - The first hash of the file name. +- `name-hash2` - The second hash of the file name. +- `name-hash3` - 64-bit Jenkins hash of the file name, used for searching in the HET table. +- `locale` - Locale info of the file. +- `file-index` - Index in the file table of the file. +- `byte-offset` - Offset of the file in the MPQ, relative to the MPQ header. +- `file-time` - Timestamp of the file. +- `file-size` - Uncompressed file size of the file, in bytes. +- `compressed-size` - Compressed file size of the file, in bytes. +- `encryption-key` - Encryption key for the file. +- `encryption-key-raw` - Encryption key for the file. +- `flags` - File flags for the file within MPQ: + * `i`: File is Imploded (By PKWARE Data Compression Library). + * `c`: File is Compressed (By any of multiple methods). + * `e`: File is Encrypted. + * `2`: File is Encrypted with key v2. + * `p`: File is a Patch file. + * `u`: File is stored as a single Unit, rather than split into sectors. + * `d`: File is a Deletion marker. Used in MPQ patches, indicating that the file no longer exists. + * `r`: File has Sector CRC checksums for each sector. This is ignored if the file is not compressed or imploded. + * `s`: Present on STANDARD.SNP\(signature). + * `x`: File exists; this is reset if the file is deleted. + * `m`: Mask for a file being compressed. + * `n`: Use default flags for internal files. + * `f`: Fix key; This is obsolete. + +You can use the `-p` or `--property` argument with the `list` subcommand to print the given properties. Many properties can be given, and they will be printed in the order given. + +```bash +$ mpqcli list -d -a Patch_rt.mpq -p hash-index -p locale -p flags +... +10893 enUS ixmn glue\ScorePv\pMain.pcx +11173 enUS ixmn rez\gluAll.tbl +11174 deDE ixmn rez\gluAll.tbl +11175 esES ixmn rez\gluAll.tbl +11176 frFR ixmn rez\gluAll.tbl +11177 itIT ixmn rez\gluAll.tbl +11178 ptBR ixmn rez\gluAll.tbl +11726 enUS cxmn (attributes) +11884 enUS ixmn glue\ScoreTd\pMain.pcx +... +``` + +## List all files with an external listfile + +Older MPQ archives do not contain (complete) file paths of their content. By using the `-l` or `--listfile` argument, one can provide an external listfile that lists the content of the MPQ archive, so that the listed files will have the correct paths. Listfiles can be downloaded on [Ladislav Zezula's site](http://www.zezula.net/en/mpq/download.html). + +```bash +$ mpqcli list -l /path/to/listfile StarDat.mpq +``` diff --git a/docs/commands/read.md b/docs/commands/read.md new file mode 100644 index 0000000..0d6952e --- /dev/null +++ b/docs/commands/read.md @@ -0,0 +1,46 @@ +# read + +Read a specific file to stdout. + +## Read a specific file from an MPQ archive + +Read the `patch.cmd` file from an MPQ archive and print the file contents to stdout. Even though the subcommand always outputs bytes, plaintext files will be human-readable. + +```bash +$ mpqcli read patch.cmd wow-patch.mpq +* set the product patch name +PatchVersion This patch upgrades World of Warcraft from version 1.10.0.5195 to version 1.10.1.5230. +* make sure that we don't patch version 1.10.1.5230 or greater +FileVersionEqualTo "$(InstallPath)\WoW.exe" 1.10.0.5195 0xffff.0xffff.0xffff.0xffff +* ProductVersionLessThan "$(InstallPath)\WoW.exe" 0.0.0.256 0.0.0.0xffff +Language enGB +SetLauncher "$(InstallPath)\WoW.exe" +* SetUninstall $(WinDir)\WoWUnin.dat +Self "World of Warcraft" +WoWPatchIndex 2 +PatchSize 1992294400 +``` + +The tool will always print output in bytes and is designed to be "piped" or redirected. For example, redirect the binary `WoW.exe` file from a WoW patch to a local file: + +```bash +$ mpqcli read "WoW.exe" wow-patch.mpq > WoW.exe +``` + +Another example, piping the bytes to the `xxd` tool. + +```bash +$ mpqcli read "WoW.exe" wow-patch.mpq | xxd +00000000: 1800 0404 de2a a5da 3240 4500 3250 4500 .....*..2@E.2PE. +00000010: 69ac 2703 0859 c601 0a7a 4500 8942 5344 i.'..Y...zE..BSD +00000020: 4946 4634 30a0 2905 8203 f244 0482 3250 IFF40.)....D..2P +00000030: 4504 8080 0280 9002 8098 0281 d801 0585 E............... +``` + +## Read one specific file with locale + +Use the `--locale` argument to specify the locale of the file to read. If there is no file with the requested name and locale, the default locale will be used instead. + +```bash +$ mpqcli read "rez\stat_txt.tbl" Patch_rt.mpq --locale ptPT +``` diff --git a/docs/commands/remove.md b/docs/commands/remove.md new file mode 100644 index 0000000..73bb009 --- /dev/null +++ b/docs/commands/remove.md @@ -0,0 +1,21 @@ +# remove + +Remove a file from an existing MPQ archive. + +## Remove a file from an existing archive + +Remove a file from an existing MPQ archive. + +```bash +$ mpqcli remove fth.txt wow-patch.mpq +[-] Removing file: fth.txt +``` + +## Remove a file from an MPQ archive with a given locale + +Use the `--locale` argument to specify the locale of the file to be removed. + +```bash +$ mpqcli remove alianza.txt wow-patch.mpq --locale esES +[-] Removing file for locale esES: alianza.txt +``` diff --git a/docs/commands/verify.md b/docs/commands/verify.md new file mode 100644 index 0000000..c5120a0 --- /dev/null +++ b/docs/commands/verify.md @@ -0,0 +1,27 @@ +# verify + +Verify a target MPQ archive signature. + +## Verify an MPQ archive + +Check the digital signature of an MPQ archive by verifying the signature in the archive with a collection of known Blizzard public keys (bundled in StormLib library). The tool will print if verification succeeds or fails, as well as return `0` for success and any other value for failure. + +```bash +$ mpqcli verify wow-patch.mpq +[*] Verify success +``` + +If verification passes, a zero (`0`) exit status will be returned. This can be helpful to verify a large number of MPQ archives without the need to review the status message that is printed out. + +```bash +$ echo $? +0 +``` + +## Verify an MPQ archive and print the digital signature + +Check the digital signature of an MPQ archive, by verifying the signature in the archive and also printing the digital signature value in hexadecimal using the `-p` or `--print` argument. + +```bash +$ mpqcli verify -p wow-patch.mpq > signature +``` diff --git a/docs/commands/version.md b/docs/commands/version.md new file mode 100644 index 0000000..b16466f --- /dev/null +++ b/docs/commands/version.md @@ -0,0 +1,8 @@ +# version + +Print the tool version number. + +```bash +$ mpqcli version +0.9.8-041480a92e698514d7938426587e93582b336b7d +``` diff --git a/docs/commands_list.md b/docs/commands_list.md new file mode 100644 index 0000000..859881d --- /dev/null +++ b/docs/commands_list.md @@ -0,0 +1,19 @@ +# Commands + +Many of the command examples use the MPQ archive file, named `wow-patch.mpq`, from a original (Vanilla) World of Warcraft patch file, named `WoW-1.10.0-to-1.10.1-enGB-patch.zip`. If you want to replicate these examples, you can [download the `wow-patch.mpq` file](https://archive.org/download/World_of_Warcraft_Client_and_Installation_Archive/Patches/1.x/WoW-1.10.0-to-1.10.1-enGB-patch.zip/wow-patch.mpq) from the Internet Archive. + + +The `mpqcli` program has the following subcommands: + +| Command | Description | +|---|---| +| [`version`](./commands/version.md) | Print the tool version number | +| [`about`](./commands/about.md) | Print information about the tool | +| [`info`](./commands/info.md) | Print information about MPQ archive properties | +| [`create`](./commands/create.md) | Create an MPQ archive from a target directory or a single file | +| [`add`](./commands/add.md) | Add a file to an existing MPQ archive | +| [`remove`](./commands/remove.md) | Remove a file from an existing MPQ archive | +| [`list`](./commands/list.md) | List files in a target MPQ archive | +| [`extract`](./commands/extract.md) | Extract one or all files from a target MPQ archive | +| [`read`](./commands/read.md) | Read a specific file to stdout | +| [`verify`](./commands/verify.md) | Verify a target MPQ archive signature | diff --git a/docs/contributing.md b/docs/contributing.md new file mode 100644 index 0000000..72fa163 --- /dev/null +++ b/docs/contributing.md @@ -0,0 +1,123 @@ +# Contributing + +Contributions are welcome. Please read the guidelines below before opening a pull request. + +## Before You Start + +If you are unsure whether a feature fits the project, or whether an existing tool could already be combined with `mpqcli` to achieve the same result, open an issue first. This avoids wasted effort and keeps the project focused. + +**mpqcli follows the Unix philosophy.** The tool is designed to do one thing well and to compose with other tools via pipes and redirection. If you find yourself wanting to add functionality that could be handled by a separate tool - for example, sorting the output of `list` - the right answer is usually to pipe the output to that tool rather than adding it here. + +## Prerequisites and Setup + +Clone the repository and initialise submodules: + +``` +git clone https://github.com/TheGrayDot/mpqcli.git +cd mpqcli +git submodule update --init --recursive +``` + +Install the clang lint tools: + +``` +make setup +``` + +## Makefile Reference + +Run `make help` to list all available targets. Common ones: + +| Target | Description | +|---|---| +| `make setup` | Install clang-format and clang-tidy via apt | +| `make build_linux` | Build for Linux using cmake | +| `make build_windows` | Build for Windows using cmake | +| `make build_clean` | Remove the cmake build directory | +| `make test_create_venv` | Create Python venv and install test dependencies (first-time only) | +| `make test_mpqcli` | Run the pytest test suite | +| `make lint` | Run all C++ linters (clang-format + clang-tidy) | +| `make lint_format` | Check formatting only (dry run) | +| `make lint_format_fix` | Auto-fix formatting in-place | +| `make lint_cpp` | Run clang-tidy static analysis | +| `make clean` | Remove all build and test artifacts | + +## Requirements for a Pull Request + +### 1. Builds on your platform + +Make sure the project builds cleanly on your development machine before opening a PR: + +``` +make build_linux # Linux +make build_windows # Windows +``` + +A PR automatically triggers the CI build workflow, which compiles and tests across all supported Linux targets (AMD64 and ARM64, glibc and musl). You are not expected to reproduce all of those locally. + +### 2. Tests pass + +Run the test suite before submitting: + +``` +make test_create_venv # first-time setup only +make test_mpqcli +``` + +All tests must pass without errors. + +### 3. New features should include tests + +If your change adds or modifies user-facing functionality - such as a new subcommand flag or a change in output format - please include a corresponding test in the `test/` directory. The existing test files (`test_list.py`, `test_add.py`, etc.) are good references for the test style and fixtures used. + +### 4. Linting must pass + +All C++ code is formatted with clang-format and analysed with clang-tidy. Run the full suite before submitting: + +``` +make lint +``` + +If there are formatting violations, auto-fix them with: + +``` +make lint_format_fix +``` + +Then re-run `make lint` to confirm everything passes. + +### 5. Match the existing code style + +C++ formatting is enforced by `.clang-format` (Google style base). Static analysis is enforced by `.clang-tidy`. Both configs live in the repo root. Python tests should follow the style of the existing test files. + +#### Suppression policy + +Suppressions are occasionally necessary for third-party code or intentional patterns. When suppressing a clang-tidy warning: + +- Use `// NOLINT(check-name)` with the specific check name - bare `// NOLINT` is not acceptable +- Every suppression must have a comment explaining why it is justified + +```cpp +// NOLINT(bugprone-easily-swappable-parameters): parameters validated by CLI11 +``` + +#### Disabling clang-format locally + +Use `// clang-format off` / `// clang-format on` only when the default formatting genuinely hurts readability (e.g. column-aligned tables). Add a brief comment explaining the intent: + +```cpp +// clang-format off: preserve column-aligned flag-to-char mappings for readability +if (flags & MPQ_FILE_IMPLODE) result += 'i'; +if (flags & MPQ_FILE_COMPRESS) result += 'c'; +// clang-format on +``` + +## Workflow Summary + +1. Fork the repository and create a branch for your change +2. Run `git submodule update --init --recursive` after cloning +3. Run `make install_clang_tools` to install lint dependencies +4. Make your changes and verify they build: `make build_linux` +5. Run `make lint` and fix any issues +6. Run `make test_mpqcli` and confirm all tests pass +7. Open a pull request with a clear description of what was changed and why diff --git a/docs/installation.md b/docs/installation.md new file mode 100644 index 0000000..cc72796 --- /dev/null +++ b/docs/installation.md @@ -0,0 +1,41 @@ +# Installation + +## Precompiled Binaries + +Pre-built binaries are available for Linux and Windows. + +Linux/WSL: + +```bash +$ curl -fsSL https://raw.githubusercontent.com/thegraydot/mpqcli/main/scripts/install.sh | bash +``` + +Microsoft Windows: + +```powershell +PS> irm https://raw.githubusercontent.com/thegraydot/mpqcli/main/scripts/install.ps1 | iex +``` + +Check the [latest release with binaries](https://github.com/TheGrayDot/mpqcli/releases). + +## Docker Image + +The Docker image for `mpqcli` is hosted on [GitHub Container Registry (GHCR)](https://ghcr.io). It provides a lightweight and portable way to use `mpqcli` without needing to build or download a binary. + +To download the latest version of the `mpqcli` Docker image, run: + +```bash +$ docker pull ghcr.io/thegraydot/mpqcli:latest +``` + +You can run `mpqcli` commands directly using the Docker container. For example: + +```bash +$ docker run ghcr.io/thegraydot/mpqcli:latest version +``` + +To use local files in the container, mount a directory from your host system. In the following example, the `-v` argument is used to mount the present working directory to `/data` directory in the container. Then the `mpqcli` container runs the `list` subcommand with `/data/example.mpq` as the target MPQ archive. + +```bash +$ docker run -v $(pwd):/data ghcr.io/thegraydot/mpqcli:latest list /data/example.mpq +``` diff --git a/docs/introduction.md b/docs/introduction.md new file mode 100644 index 0000000..9bc1145 --- /dev/null +++ b/docs/introduction.md @@ -0,0 +1,16 @@ +# Introduction + +A command-line tool to create, add, remove, list, extract, read, and verify MPQ archives using the [StormLib library](https://github.com/ladislav-zezula/StormLib). + +> ⚠️ **Warning:** This project is under active development and will change functionality between released versions until version 1.0.0. + +## Overview + +**This is a command-line tool, designed for automation and built with the Unix philosophy in mind.** It is designed to work seamlessly with other command-line tools, supporting piping, redirection, and integration into shell scripts and workflows. For example: + +- Run one command to create an MPQ archive from a directory of files or a single file +- Run one command to list all files in an MPQ archive +- Pipe the output to `grep` or other tools to search, filter, or process files +- Redirect output to files or other commands for further automation + +If you require an MPQ tool with a graphical interface (GUI) and explicit support for more MPQ archive versions - I would recommend using [Ladik's MPQ Editor](http://www.zezula.net/en/mpq/download.html). diff --git a/docs/js/copy-command.js b/docs/js/copy-command.js new file mode 100644 index 0000000..ca1deb8 --- /dev/null +++ b/docs/js/copy-command.js @@ -0,0 +1,110 @@ +(function () { + "use strict"; + + // Run after the page is fully loaded + window.addEventListener("load", function () { + attachCopyHandlers(); + }); + + // Re-attach on mdBook page navigation (it uses a SPA-style renderer) + document.addEventListener("mdbook-loaded", function () { + attachCopyHandlers(); + }); + + function attachCopyHandlers() { + // mdBook renders copy buttons with the class .clip-button + // Find all code blocks that have a copy button + var codeBlocks = document.querySelectorAll("pre code"); + + codeBlocks.forEach(function (codeEl) { + var pre = codeEl.parentElement; + var copyBtn = pre.querySelector(".clip-button"); + if (!copyBtn) return; + + // Detect language from class e.g. "language-bash" or "language-powershell" + var lang = detectLanguage(codeEl); + if (lang !== "bash" && lang !== "powershell") return; + + // Replace the default click handler with our filtered one + var newBtn = copyBtn.cloneNode(true); + copyBtn.parentNode.replaceChild(newBtn, copyBtn); + + newBtn.addEventListener("click", function (e) { + e.stopPropagation(); + var text = filterCommands(codeEl.innerText, lang); + navigator.clipboard.writeText(text).then(function () { + // Brief visual feedback - reuse mdBook's own checkmark if present, + // otherwise flash the button title + newBtn.classList.add("clip-button--success"); + setTimeout(function () { + newBtn.classList.remove("clip-button--success"); + }, 1500); + }); + }); + }); + } + + function detectLanguage(codeEl) { + var classes = codeEl.className.split(" "); + for (var i = 0; i < classes.length; i++) { + if (classes[i].startsWith("language-")) { + return classes[i].replace("language-", "").toLowerCase(); + } + } + return null; + } + + function filterCommands(rawText, lang) { + var lines = rawText.split("\n"); + var result = []; + var continuation = false; + + for (var i = 0; i < lines.length; i++) { + var line = lines[i]; + + if (lang === "bash") { + if (continuation) { + // This line is a continuation of the previous command - include it + result.push(line); + // Check if this line itself continues further + continuation = line.trimEnd().endsWith("\\"); + + } else if (line.startsWith("$ ")) { + // Bash prompt line - strip the prompt and include + var stripped = line.slice(2); + result.push(stripped); + continuation = stripped.trimEnd().endsWith("\\"); + + } else if (line.startsWith("> ")) { + // Bash secondary prompt (e.g. after a heredoc or multiline) + result.push(line.slice(2)); + continuation = false; + + } else { + // Output line - skip + continuation = false; + } + + } else if (lang === "powershell") { + if (continuation) { + // Continuation line after backtick - include it + result.push(line); + continuation = line.trimEnd().endsWith("`"); + + } else if (line.startsWith("PS> ")) { + // PowerShell prompt line - strip the prompt and include + var stripped = line.slice(4); + result.push(stripped); + continuation = stripped.trimEnd().endsWith("`"); + + } else { + // Output line - skip + continuation = false; + } + } + } + + // Join and trim trailing newline + return result.join("\n").trimEnd(); + } +})(); From fb82295f4c960cd52201d09b0922e4d5a018aa8c Mon Sep 17 00:00:00 2001 From: thomaslaurenson Date: Tue, 28 Apr 2026 20:10:54 +1200 Subject: [PATCH 2/4] Removed examples from README that are in new documentation website --- CONTRIBUTING.md | 8 +- README.md | 525 ++---------------------------------------------- 2 files changed, 16 insertions(+), 517 deletions(-) diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 9d9ba29..89453f0 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -6,7 +6,7 @@ Contributions are welcome. Please read the guidelines below before opening a pul If you are unsure whether a feature fits the project, or whether an existing tool could already be combined with `mpqcli` to achieve the same result, open an issue first. This avoids wasted effort and keeps the project focused. -**mpqcli follows the Unix philosophy.** The tool is designed to do one thing well and to compose with other tools via pipes and redirection. If you find yourself wanting to add functionality that could be handled by a separate tool — for example, sorting the output of `list` — the right answer is usually to pipe the output to that tool rather than adding it here. +**mpqcli follows the Unix philosophy.** The tool is designed to do one thing well and to compose with other tools via pipes and redirection. If you find yourself wanting to add functionality that could be handled by a separate tool - for example, sorting the output of `list` - the right answer is usually to pipe the output to that tool rather than adding it here. ## Prerequisites and Setup @@ -68,7 +68,7 @@ All tests must pass without errors. ### 3. New features should include tests -If your change adds or modifies user-facing functionality — such as a new subcommand flag or a change in output format — please include a corresponding test in the `test/` directory. The existing test files (`test_list.py`, `test_add.py`, etc.) are good references for the test style and fixtures used. +If your change adds or modifies user-facing functionality - such as a new subcommand flag or a change in output format - please include a corresponding test in the `test/` directory. The existing test files (`test_list.py`, `test_add.py`, etc.) are good references for the test style and fixtures used. ### 4. Linting must pass @@ -94,7 +94,7 @@ C++ formatting is enforced by `.clang-format` (Google style base). Static analys Suppressions are occasionally necessary for third-party code or intentional patterns. When suppressing a clang-tidy warning: -- Use `// NOLINT(check-name)` with the specific check name — bare `// NOLINT` is not acceptable +- Use `// NOLINT(check-name)` with the specific check name - bare `// NOLINT` is not acceptable - Every suppression must have a comment explaining why it is justified ```cpp @@ -116,7 +116,7 @@ if (flags & MPQ_FILE_COMPRESS) result += 'c'; ### StormLib locale state is global and not thread-safe -`SFileSetLocale` sets a process-wide locale variable (`g_lcFileLocale`) inside StormLib. All locale-sensitive operations in `mpq.cpp` — file open, add, remove, read, extract, and list — call `SFileSetLocale` immediately before the relevant StormLib call. There is no locale-explicit alternative in StormLib's public API (`SFileOpenFileEx`, `SFileAddFileEx`, etc. all read `g_lcFileLocale` internally). +`SFileSetLocale` sets a process-wide locale variable (`g_lcFileLocale`) inside StormLib. All locale-sensitive operations in `mpq.cpp` - file open, add, remove, read, extract, and list - call `SFileSetLocale` immediately before the relevant StormLib call. There is no locale-explicit alternative in StormLib's public API (`SFileOpenFileEx`, `SFileAddFileEx`, etc. all read `g_lcFileLocale` internally). This means: diff --git a/README.md b/README.md index 778eee9..873c3cf 100644 --- a/README.md +++ b/README.md @@ -4,7 +4,7 @@ ![Release Version](https://img.shields.io/github/v/release/TheGrayDot/mpqcli?style=flat) -![Release downloads](https://img.shields.io/github/downloads/thegraydot/mpqcli/total?label=release_downloads) ![Package downloads](https://img.shields.io/badge/package_downloads-497-green) +![Release downloads](https://img.shields.io/github/downloads/thegraydot/mpqcli/total?label=release_downloads) ![Package downloads](https://img.shields.io/badge/package_downloads-647-green) A command-line tool to create, add, remove, list, extract, read, and verify MPQ archives using the [StormLib library](https://github.com/ladislav-zezula/StormLib). @@ -21,533 +21,32 @@ A command-line tool to create, add, remove, list, extract, read, and verify MPQ If you require an MPQ tool with a graphical interface (GUI) and explicit support for more MPQ archive versions — I would recommend using [Ladik's MPQ Editor](http://www.zezula.net/en/mpq/download.html). -## Download +## Releases -### Precompiled Binaries +Pre-built binaries are available for Linux and Windows on [GitHub releases](https://github.com/thegraydot/mpqcli/releases). -Pre-built binaries are available for Linux and Windows. +Docker images are available on [GitHub packages](https://github.com/thegraydot/mpqcli/pkgs/container/mpqcli). -Linux/WSL: +## Quickstart ```bash curl -fsSL https://raw.githubusercontent.com/thegraydot/mpqcli/main/scripts/install.sh | bash +... +mpqcli list ``` Microsoft Windows: ```powershell irm https://raw.githubusercontent.com/thegraydot/mpqcli/main/scripts/install.ps1 | iex -``` - -Check the [latest release with binaries](https://github.com/TheGrayDot/mpqcli/releases). - -### Docker Image - -The Docker image for `mpqcli` is hosted on [GitHub Container Registry (GHCR)](https://ghcr.io). It provides a lightweight and portable way to use `mpqcli` without needing to build or download a binary. - -To download the latest version of the `mpqcli` Docker image, run: - -```bash -docker pull ghcr.io/thegraydot/mpqcli:latest -``` - -You can run `mpqcli` commands directly using the Docker container. For example: - -```bash -docker run ghcr.io/thegraydot/mpqcli:latest version -``` - -To use local files in the container, mount a directory from your host system. In the following example, the `-v` argument is used to mount the present working directory to `/data` directory in the container. Then the `mpqcli` container runs the `list` subcommand with `/data/example.mpq` as the target MPQ archive. - -```bash -docker run -v $(pwd):/data ghcr.io/thegraydot/mpqcli:latest list /data/example.mpq -``` - -## Subcommands - -The `mpqcli` program has the following subcommands: - -- `version`: Print the tool version number -- `about`: Print information about the tool -- `info`: Print information about MPQ archive properties -- `create`: Create an MPQ archive from a target directory or a single file -- `add`: Add a file to an existing MPQ archive -- `remove`: Remove a file from an existing MPQ archive -- `list`: List files in a target MPQ archive -- `extract`: Extract one/all files from a target MPQ archive -- `read`: Read a specific file to stdout -- `verify`: Verify a target MPQ archive signature - -## Command Examples - -Many of the examples use the MPQ archive file, named `wow-patch.mpq`, from a original (Vanilla) World of Warcraft patch file, named `WoW-1.10.0-to-1.10.1-enGB-patch.zip`. If you want to replicate these examples, you can [download the `wow-patch.mpq` file](https://archive.org/download/World_of_Warcraft_Client_and_Installation_Archive/Patches/1.x/WoW-1.10.0-to-1.10.1-enGB-patch.zip/wow-patch.mpq) from the Internet Archive. - -### Print information about an MPQ archive - -The `info` subcommand prints a list of useful information (property keys and values) of an MPQ archive. - -```bash -$ mpqcli info wow-patch.mpq -Archive size: 1798918 -File count: 65 -Format version: 1 -Header offset: 0 -Header size: 32 -Max files: 128 -Signature type: Weak -``` - -### Print one specific MPQ archive property - -The `info` subcommand supports the following properties: - -- `archive-size` -- `file-count` -- `format-version` -- `header-offset` -- `header-size` -- `max-files` -- `signature-type` - -You can use the `-p` or `--property` argument with the `info` subcommand to print just the value of a specific property. This can be useful for automation, for example, to determine the signature type of a directory of MPQ archives. - -```bash -$ mpqcli info -p file-count wow-patch.mpq -65 -``` - -### Create an MPQ archive from a target directory - -Create an MPQ file from a target directory. Automatically adds `(listfile)` to the archive, and will skip this file if it exists in the target directory. - -```bash -$ mpqcli create -``` - -The default mode of operation for the `create` subcommand is to take everything from the "target" directory (and below) and recursively add it to the archive. The directory structure is retained. Windows-style backslash path separators are used (`\`), as per the observed behavior in most MPQ archives. - -### Create an MPQ archive for a specific game - -Target a specific game version by using the `-g` or `--game` argument. This will automatically set the correct archive format version and settings, although they can be overridden. - -```bash -$ mpqcli create -g starcraft -$ mpqcli create --game wow-wotlk --sector-size 16384 --version 3 # World of Warcraft - Wrath of the Lich King, but with non-standard sector size and MPQ version -``` - -### Create an MPQ archive from a single file - -Create an MPQ file from a single file. - -```bash -$ mpqcli create --game diablo2 -``` - -This will put the given file in the root of the MPQ archive. By optionally providing a path in the `--name-in-archive` parameter, the name that the file has in the MPQ archive can be changed, and it can be put in a directory. - -### Create and sign an MPQ archive - -Use the `-s` or `--sign` argument to cryptographically sign an MPQ archive with the Blizzard weak signature. - -```bash -$ mpqcli create --version 1 --sign -``` - -### Create an MPQ archive with a given locale - -Use the `--locale` argument to specify the locale that all added files will have in the archive. Note that subsequent added files will have the default locale unless the `--locale` argument is specified again. - -```bash -$ mpqcli create --locale koKR -``` - -### Add a file to an existing archive - -Add a local file to an already existing MPQ archive. - -```bash -$ echo "For The Horde" > fth.txt -$ mpqcli add fth.txt wow-patch.mpq -[+] Adding file: fth.txt -``` - -Alternatively, you can add a file under a specific file name using the `-f` or `--filename-in-archive` argument. - -```bash -$ echo "For The Alliance" > fta.txt -$ mpqcli add fta.txt wow-patch.mpq --filename-in-archive "alliance.txt" -[+] Adding file: alliance.txt -``` - -Alternatively, you can add a file to a specific subdirectory using the `-d` or `--directory-in-archive` argument. - -```bash -$ echo "For The Swarm" > fts.txt -$ mpqcli add fts.txt wow-patch.mpq --directory-in-archive texts -[+] Adding file: texts\fts.txt -``` - -Alternatively, you can add a file under a specific directory and filename using the `-p` or `--path` argument. - -```bash -$ echo "For The Swarm" > fts.txt -$ mpqcli add fts.txt wow-patch.mpq --path "texts\swarm.txt" -[+] Adding file: texts\swarm.txt -``` - -To overwrite a file in an MPQ archive, set the `w` or `--overwrite` flag: - -```bash -$ echo "For The Horde" > allegiance.txt -$ mpqcli add allegiance.txt wow-patch.mpq -[+] Adding file: allegiance.txt -$ echo "For The Alliance" > allegiance.txt -$ mpqcli add allegiance.txt wow-patch.mpq -[!] File already exists in MPQ archive: allegiance.txt - Skipping... -$ mpqcli add allegiance.txt wow-patch.mpq --overwrite -[+] File already exists in MPQ archive: allegiance.txt - Overwriting... -[+] Adding file: allegiance.txt -``` - - -### Add a file to an MPQ archive with a given locale - -Use the `--locale` argument to specify the locale that the added file will have in the archive. Note that subsequent added files will have the default locale unless the `--locale` argument is specified again. - -```bash -$ mpqcli add allianz.txt wow-patch.mpq --locale deDE -[+] Adding file for locale deDE: allianz.txt -``` - -### Add a file with game-specific properties - -Target a specific game version by using the `-g` or `--game` argument. This will automatically set the correct encryption rules and MPQ flags, although they can be overridden. - -```bash -$ mpqcli add khwhat1.wav archive.mpq --game wc2 # In StarCraft and Warcraft II MPQs, wav files are compressed in ADPCM form -[+] Adding file: khwhat1.wav -``` - - -### Remove a file from an existing archive - -Remove a file from an existing MPQ archive. - -```bash -$ mpqcli remove fth.txt wow-patch.mpq -[-] Removing file: fth.txt -``` - -### Remove a file from an MPQ archive with a given locale - -Use the `--locale` argument to specify the locale of the file to be removed. - -```bash -$ mpqcli remove alianza.txt wow-patch.mpq --locale esES -[-] Removing file for locale esES: alianza.txt -``` - -### List all files in an MPQ archive - -Pretty simple, list files in an MPQ archive. Useful to "pipe" to other tools, such as `grep` (see below for examples). - -```bash -$ mpqcli list wow-patch.mpq -BM_COKETENT01.BLP -Blizzard_CraftUI.xml -CreatureSoundData.dbc ... -Blizzard_CraftUI.lua -30ee7bd3959906e358eff01332cf045e.blp -realmlist.wtf +mpqcli.exe list ``` -### List all files with detailed output - -Similar to the `ls` command with the `-l` and `-a` options, additional detailed information can be included with the `list` subcommand. The `-a` option includes printing "special" files used in MPQ archives including: `(listfile)`, `(attributes)` and `(signature)`. - -```bash -$ mpqcli list -d -a wow-patch.mpq - 88604 enUS 2006-03-29 02:02:37 BM_COKETENT01.BLP - 243 enUS 2006-04-04 21:28:14 Blizzard_CraftUI.xml - 388 enUS 2006-03-29 19:32:46 CreatureSoundData.dbc - ... - 184 enUS 2006-04-04 21:28:14 Blizzard_CraftUI.lua - 44900 enUS 2006-03-29 02:01:02 30ee7bd3959906e358eff01332cf045e.blp - 68 enUS 2006-04-07 00:58:44 realmlist.wtf -``` - -### List specific properties - -The `list` subcommand supports listing the following properties: +## Documentation -- `hash-index` - Index in the hash table where the file entry is. -- `name-hash1` - The first hash of the file name. -- `name-hash2` - The second hash of the file name. -- `name-hash3` - 64-bit Jenkins hash of the file name, used for searching in the HET table. -- `locale` - Locale info of the file. -- `file-index` - Index in the file table of the file. -- `byte-offset` - Offset of the file in the MPQ, relative to the MPQ header. -- `file-time` - Timestamp of the file. -- `file-size` - Uncompressed file size of the file, in bytes. -- `compressed-size` - Compressed file size of the file, in bytes. -- `encryption-key` - Encryption key for the file. -- `encryption-key-raw` - Encryption key for the file. -- `flags` - File flags for the file within MPQ: - * `i`: File is Imploded (By PKWARE Data Compression Library). - * `c`: File is Compressed (By any of multiple methods). - * `e`: File is Encrypted. - * `2`: File is Encrypted with key v2. - * `p`: File is a Patch file. - * `u`: File is stored as a single Unit, rather than split into sectors. - * `d`: File is a Deletion marker. Used in MPQ patches, indicating that the file no longer exists. - * `r`: File has Sector CRC checksums for each sector. This is ignored if the file is not compressed or imploded. - * `s`: Present on STANDARD.SNP\(signature). - * `x`: File exists; this is reset if the file is deleted. - * `m`: Mask for a file being compressed. - * `n`: Use default flags for internal files. - * `f`: Fix key; This is obsolete. +Please see [mpqcli.thegraydot.com](https://mpqcli.thegraydot.com) for full documentation. -You can use the `-p` or `--property` argument with the `list` subcommand to print the given properties. Many properties can be given, and they will be printed in the order given. +## Contributing -```bash -$ mpqcli list -d -a Patch_rt.mpq -p hash-index -p locale -p flags -... -10893 enUS ixmn glue\ScorePv\pMain.pcx -11173 enUS ixmn rez\gluAll.tbl -11174 deDE ixmn rez\gluAll.tbl -11175 esES ixmn rez\gluAll.tbl -11176 frFR ixmn rez\gluAll.tbl -11177 itIT ixmn rez\gluAll.tbl -11178 ptBR ixmn rez\gluAll.tbl -11726 enUS cxmn (attributes) -11884 enUS ixmn glue\ScoreTd\pMain.pcx -... -``` - -### List all files with an external listfile - -Older MPQ archives do not contain (complete) file paths of their content. By using the `-l` or `--listfile` argument, one can provide an external listfile that lists the content of the MPQ archive, so that the listed files will have the correct paths. Listfiles can be downloaded on [Ladislav Zezula's site](http://www.zezula.net/en/mpq/download.html). - -```bash -$ mpqcli list -l /path/to/listfile StarDat.mpq -``` - -### Extract all files from an MPQ archive - -The output will be saved in a folder with the same name as the target MPQ file, without the extension. - -```bash -$ mpqcli extract wow-patch.mpq -[*] Extracted: BM_COKETENT01.BLP -[*] Extracted: Blizzard_CraftUI.xml -[*] Extracted: CreatureSoundData.dbc -... -[*] Extracted: Blizzard_CraftUI.lua -[*] Extracted: 30ee7bd3959906e358eff01332cf045e.blp -[*] Extracted: realmlist.wtf -``` - -### Extract all files to a target directory - -Extract files to a specific target directory, which will be created if it doesn't already exist. In this example, `patch-1.10` is the user-specified output directory. - -```bash -$ mpqcli extract -o patch-1.10 wow-patch.mpq -``` - -### Extract all files with an external listfile - -Older MPQ archives do not contain (complete) file paths of their content. By providing an external listfile that lists the content of the MPQ archive, the extracted files will have the correct names and paths. Listfiles can be downloaded on [Ladislav Zezula's site](http://www.zezula.net/en/mpq/download.html). - -```bash -$ mpqcli extract -l path/to/listfile War2Dat.mpq -``` - -### Extract one specific file - -Extract a single file using the `-f` option. If the target file in the MPQ archive is nested (in a directory) you need to include the full path. Similar to the examples above, you can use the `-o` argument to specify the output directory. - -```bash -$ mpqcli extract -f "Documentation\Layout\Greeting.html" "World of Warcraft_1.12.1.5875/Data/base.MPQ" -``` - -### Extract one specific file with locale - -Use the `--locale` argument to specify the locale of the file to extract. If there is no file with the requested name and locale, the default locale will be used instead. - -```bash -$ mpqcli extract -f "rez\gluBNRes.res" Patch_rt.mpq --locale deDE -``` - - -### Read a specific file from an MPQ archive - -Read the `patch.cmd` file from an MPQ archive and print the file contents to stdout. Even though the subcommand always outputs bytes, plaintext files will be human-readable. - -```bash -$ mpqcli read patch.cmd wow-patch.mpq -* set the product patch name -PatchVersion This patch upgrades World of Warcraft from version 1.10.0.5195 to version 1.10.1.5230. -* make sure that we don't patch version 1.10.1.5230 or greater -FileVersionEqualTo "$(InstallPath)\WoW.exe" 1.10.0.5195 0xffff.0xffff.0xffff.0xffff -* ProductVersionLessThan "$(InstallPath)\WoW.exe" 0.0.0.256 0.0.0.0xffff -Language enGB -SetLauncher "$(InstallPath)\WoW.exe" -* SetUninstall $(WinDir)\WoWUnin.dat -Self "World of Warcraft" -WoWPatchIndex 2 -PatchSize 1992294400 -``` - -The tool will always print output in bytes and is designed to be "piped" or redirected. For example, redirect the binary `WoW.exe` file from a WoW patch to a local file: - -```bash -$ mpqcli read "WoW.exe" wow-patch.mpq > WoW.exe -``` - -Another example, piping the bytes to the `xxd` tool. - -```bash -$ mpqcli read "WoW.exe" wow-patch.mpq | xxd -00000000: 1800 0404 de2a a5da 3240 4500 3250 4500 .....*..2@E.2PE. -00000010: 69ac 2703 0859 c601 0a7a 4500 8942 5344 i.'..Y...zE..BSD -00000020: 4946 4634 30a0 2905 8203 f244 0482 3250 IFF40.)....D..2P -00000030: 4504 8080 0280 9002 8098 0281 d801 0585 E............... -``` - -### Read one specific file with locale - -Use the `--locale` argument to specify the locale of the file to read. If there is no file with the requested name and locale, the default locale will be used instead. - -```bash -$ mpqcli read "rez\stat_txt.tbl" Patch_rt.mpq --locale ptPT -``` - - -### Verify an MPQ archive - -Check the digital signature of an MPQ archive by verifying the signature in the archive with a collection of known Blizzard public keys (bundled in StormLib library). The tool will print if verification suceeds or fails, as well as return `0` for success and any other value for failure. - -```bash -$ mpqcli verify wow-patch.mpq -[*] Verify success -``` - -If verification passes, a zero (`0`) exit status will be returned. This can be helpful to verify a large number of MPQ archives without the need to review the status message that is printed out. - -```bash -$ echo $? -0 -``` - -### Verify an MPQ archive and print the digital signature - -Check the digital signature of an MPQ archive, by verifying the signature in the archive and also printing the digital signature value in hexidecimal using the `-p` or `--print` argument. - -```bash -$ mpqcli verify -p wow-patch.mpq > signature -``` - -## Advanced Command Examples - -### Search and extract files on Linux - -The `mpqcli` tool has no native search feature — instead, it is designed to be integrated with other, external operating system tools. For example, `mpqcli list` can be "piped" to `grep` in Linux or `Select-String` in Windows PowerShell to perform searching. - -The following command lists all files in an MPQ archive, and each filename is filtered using `grep` — selecting files ending in `.exe` (note: `.` is a regex wildcard matching any character; use `\.exe` for a strictly literal match) — which is then passed back to `mpqcli extract`. The result: search and extract all `exe` files. - -```bash -$ mpqcli list wow-patch.mpq | grep -i \.exe | xargs -I@ mpqcli extract -f "@" wow-patch.mpq -[*] Extracted: Launcher.exe -[*] Extracted: BackgroundDownloader.exe -[*] Extracted: WoW.exe -[*] Extracted: BNUpdate.exe -[*] Extracted: Repair.exe -[*] Extracted: WowError.exe -``` - -Note that directories are specified with backslashes, which need to be escaped (due to how grep and xargs handles them). The following example extracts all `dat` files in the `arr` directory from the `StarDat.mpq` archive and handles escaping via `sed`. - -```bash -$ mpqcli list -l scbw.txt StarDat.mpq | grep -i "arr\\\.*dat$" | sort | sed 's|\\|\\\\|g' | xargs -I@ mpqcli extract -f "@" -k StarDat.mpq -[*] Extracted: arr/flingy.dat -[*] Extracted: arr/images.dat -[*] Extracted: arr/mapdata.dat -[*] Extracted: arr/orders.dat -[*] Extracted: arr/portdata.dat -[*] Extracted: arr/sfxdata.dat -[*] Extracted: arr/sprites.dat -[*] Extracted: arr/techdata.dat -[*] Extracted: arr/units.dat -[*] Extracted: arr/upgrades.dat -[*] Extracted: arr/weapons.dat -``` - -### Search and extract files on Windows - -The following command lists all files in an MPQ archive, and each filename is filtered using `Select-String` — selecting files ending in `.exe` (note: `.` is a regex wildcard matching any character; use `\.exe` for a strictly literal match) — which is then passed back to `mpqcli extract`. The result: search and extract all `exe` files. - -```powershell -mpqcli.exe list wow-patch.mpq | Select-String -Pattern \.exe | ForEach-Object { mpqcli.exe extract -f $_ wow-patch.mpq } -``` - -## Building - -### Requirements - -- cmake -- C++ 17 compiler -- StormLib (provided as Git Submodule) -- CLI11 (provided as Git Submodule) - -### Linux - -```bash -git clone --recursive https://github.com/TheGrayDot/mpqcli.git -cd mpqcli -cmake -B build -cmake --build build -``` - -The `mpqcli` binary will be available in: `./build/bin/mpqcli` - -### Windows - -```bash -git clone --recursive https://github.com/TheGrayDot/mpqcli.git -cd mpqcli -cmake -B build -cmake --build build --config Release -``` - -The `mpqcli.exe` binary will be available in: `.\build\bin\Release\mpqcli.exe` - -## Dependencies - -### StormLib - -This project requires the [StormLib](https://github.com/ladislav-zezula/StormLib) library. Many thanks to [Ladislav Zezula](https://github.com/ladislav-zezula) for authoring such a good library and releasing the code under an open-source license. The StormLib library has a number of requirements. However, the build method specifies using the libraries bundled with StormLib. - -### CLI11 - -This project also uses the [CLI11](https://github.com/CLIUtils/CLI11) command line parser for C++11 and beyond. It provides simple and easy-to-use CLI arguments. - -## Tests - -This project implements End-to-end (E2E) testing, sometimes referred to as system testing or integration testing. This methodology is used because it verifies application functionality by simulating actual usage by an end user. Testing includes creating a variety of MPQ archives, as well as dynamically downloading some small (~1-5MB) MPQ archives from the Internet Archive. The Python programming language coupled with the [pytest framework](https://github.com/pytest-dev/pytest) is used to implement testing, mainly due to ease of implementation. - -To configure the testing environment you will need Python installed, as well as the required `pytest` package. On a Debian-based Linux system, the following will configure the environment: - -```bash -sudo apt install python3-venv python3-pip -python3 -m venv test/.venv -source test/.venv/bin/activate -pip3 install -r test/requirements.txt -``` - -Then you can run the tests using: - -```bash -python3 -m pytest test -s -``` +Please see the [CONTRIBUTING.md](CONTRIBUTING.md) document. From c484cafaf75e984e1593cc7a351c8e77ecae9af5 Mon Sep 17 00:00:00 2001 From: thomaslaurenson Date: Wed, 29 Apr 2026 20:27:12 +1200 Subject: [PATCH 3/4] Only run main workflow for prerelease on code change --- .github/workflows/main.yml | 10 ++++++++++ 1 file changed, 10 insertions(+) diff --git a/.github/workflows/main.yml b/.github/workflows/main.yml index 369afa6..2ac712d 100644 --- a/.github/workflows/main.yml +++ b/.github/workflows/main.yml @@ -4,6 +4,16 @@ on: push: branches: - "main" + paths: + - ".github/workflows/**" + - ".clang-format" + - ".clang-tidy" + - "src/**" + - "extern/**" + - "test/**" + - "CMakeLists.txt" + - "Dockerfile.glibc" + - "Dockerfile.musl" concurrency: group: prerelease From f4d951cae4fc3264d6f8c1e625fd22bdb38ccc42 Mon Sep 17 00:00:00 2001 From: thomaslaurenson Date: Wed, 29 Apr 2026 20:30:06 +1200 Subject: [PATCH 4/4] Only run pr workflow for prerelease on code change --- .github/workflows/pr.yml | 12 +++++++++++- 1 file changed, 11 insertions(+), 1 deletion(-) diff --git a/.github/workflows/pr.yml b/.github/workflows/pr.yml index 820781c..678b3ce 100644 --- a/.github/workflows/pr.yml +++ b/.github/workflows/pr.yml @@ -1,7 +1,17 @@ name: Pull Request on: - pull_request + pull_request: + paths: + - ".github/workflows/**" + - ".clang-format" + - ".clang-tidy" + - "src/**" + - "extern/**" + - "test/**" + - "CMakeLists.txt" + - "Dockerfile.glibc" + - "Dockerfile.musl" permissions: contents: write