vpk

2012-07-13

VPK (“Valve Pak”) is the archive format used by the Source Engine, Valve’s game engine (see the Valve Developer Wiki article). This vpk is a Ruby library for reading and writing those VPK files — it can both extract a VPK file and pack a directory into one.

Published as a gem, with a command-line tool bundled in.

gem install vpk

require 'vpk'

# VPK ファイルを展開する
VPK::VPKFile.new("./path_to.vpk").extract_to("./")

# ディレクトリを VPK にまとめる
VPK::VPKFile.archive("./path_to_dir").write_to("./archive.vpk")

From the command line it works like this: -c creates a .vpk with the same name as the directory, and -x extracts into the current directory.

vpk -c some_dir       # some_dir を some_dir.vpk に圧縮
vpk -x some_file.vpk  # カレントディレクトリに展開

Internals

What it implements is version 1 of the VPK format. A file consists of three layers: a “12-byte header”, a “directory section holding the file listing”, and a “data section of concatenated file contents”.

• The directory section is a 3-level tree: “extension → path → filename”. Each element is a NUL-terminated string, with an empty string terminating that level. Files directly under the root use a single ASCII space as their path

ディレクトリ部の並び(例):
"vtf"                                ← 第 1 段: 拡張子
  "materials/vgui"                   ← 第 2 段: パス
    "logo" + エントリ情報(18 バイト)  ← 第 3 段: ファイル名
    ""                               ← ファイル名の終端
  ""                                 ← パスの終端
""                                   ← 拡張子の終端 = ディレクトリ部おわり

• The per-file entry information is a fixed 18 bytes, holding the CRC, offset, length, and so on. Positions inside the data section are referenced via entry_offset (relative to the end of the directory section). The data section is, as the name suggests, the payloads of the files concatenated as-is

The read and write flows look like this.

• VPKFile (lib/vpk.rb) — on read, it first validates the header signature (0x55AA1234), parses the directory section with a triple loop, then loads every entry’s payload into memory and verifies it against the CRC32 (Zlib). A signature mismatch raises VPKFileFormatError, a CRC mismatch raises VPKFileInvalidCRCError
• VPKDirectoryEntry — the class holding one file’s metadata (CRC, offset, length, etc.) and payload. Conversion to and from full paths and CRC verification also live here
• The write side (VPKFile.archive) walks the directory recursively with Find.find to build entries, then groups them by extension and path with to_file_struct_tree. It computes the directory section’s length up front (string lengths plus 18 bytes per entry) and then writes the header, directory section, and data section in order
• The VPK it writes is a single-file layout with no data split into separate files; archive_index is fixed at 0x7FFF and preload data is unused (always 0)
• VPKUtil — shortcuts for calling extract / archive as a single method
• Logging can be plugged in via VPKFile.logger= (no output by default)

github.com/kimoto/vpk