melee_c → from ISO to 2D: write our own extraction

Goal: render all characters, all stages (2D for now), all animations · Start: Fox + Falco, all action states

Extract from the ISO: fighters · fighter colours · animations · stages · stage animations

Direction: no borrowed Zig code — original C extraction, Zig/other code used only as behavioral reference · Plan: 2026-08-15

TL;DR. The end goal is unchanged: a 2D renderer in viewer.c that draws (eventually) every character, costume, and stage, driven by the replay's action_state + anim_frame. What changed is the front half: the first part of this plan is writing a from-scratch C extraction pipeline (tools/extract) that reads the ISO → FST → DAT container → HSD types, and produces cached assets for fighters, fighter colours, animations, stages, and stage animations. We do it in C, as original work: Zig (litewing) is a reference for behavior only — we study what it does and why, then write our own code. Understanding HSD deeply is a prerequisite, not a side quest.

0What exists & what we lean on

~/Dev/fixtures/game.iso 1.46 GB Melee ISO — the source of all assets. melee_c/src/parser.{c,h} Our C .slp parser: per-frame action_state, anim_frame, facing, x/y, character_id, FOD/Whispy/Stadium events. melee_c/src/viewer.c Our C HTTP + software-2D + PNG renderer (browser at melee.zamachnoi.com). litewing (Zig) reference only dat-extractor + melee-viewer prove the pipeline works and reveal the hard parts. We read it for behavior and never copy code; it is not a build dependency. Known-good numbers Validation target (Falco): bones=67, primitive_groups=26, textures=24, actions > 300. Our extractor must reproduce these independently. Toolchain cc 13.3 + existing Makefile. No Zig needed to build melee_c.

Reference sources (behavior, formats, reverse engineering)

Melee DAT format thread (Smashboards) — the DAT container layout and naming conventions; ground truth for section/offset formulas.
HSDLib (UncleDicken) — strongest public reference for HSD object types (JOBJ/DOBJ/POBJ/MOBJ/TOBJ), scene graph semantics, and HSD-specific math conventions.
project-slippi/SlippiRendering — reads DATs directly and renders replays; reference for node classification, geometry decode, materials.
Dolphin source — GX/TPL texture formats (I4/I8/RGB5A3/RGBA8/C4/C8 + TLUT) and TEV/lighting conventions.
litewing docs/DAT_FILE_LAYOUT.md + USEFUL_LINKS.md — useful index of the above (read for pointers, not code).

1Architecture

Concern	Where it lives	Language	Note
ISO → cached assets (extraction)	`tools/extract` in melee_c	C	Original implementation. Offline, run on dev box.
Asset loading	`src/asset.{c,h}`	C	Reads the cache produced by extract.
Pose + 2D raster	`src/viewer.c` renderer	C	Same server, PNG pipeline unchanged.
Action-state → animation lookup	`src/asset.c` (tables)	C	Common states + Fox/Falco specials, name → prefix → raw-index fallback.

House rule: never git subtree, vendor, or transpile litewing. We implement the same formats with our own structure. Shared facts (offsets, byte layouts from the community thread) are fine to reproduce verbatim — that's data, not code.

Data flow

game.iso ──[tools/extract: C]──► /data/cache   (versioned, ISO-fingerprinted)
  │                                  ├─ meta.json                fingerprint, schema ver
  │  FST ──► DAT bytes ──► HSD       ├─ char/<char>-<colour>.model  bones, prims, verts, tex
  │                                  ├─ char/<char>-<colour>.anims  action table + figatree tracks
  │                                  └─ stage/<stage>.stage        sections, scale, camera, stage anims
master.slp ──[parser.c]──► action_state, anim_frame, facing, x, y
     │
     └─[viewer.c render]────────► 4x4 RGBA ──PNG──► browser
         action_state ─lookup→ animation   anim_frame ─sample→ bones
         bones ─skin→ verts                x,y,facing ─project→ 2D ─paint

2Phase A — Understand HSD (the prerequisite)

Goal: a written, precise model of the formats before writing decoder code. This is the milestone that makes extraction possible — do it properly.

ISO/FST layer → docs/DAT.md: GameCube disc structure, FST entries, locating *.dat, reading file bytes.
DAT container → same doc: header, data block, relocation table, root/reference node tables, string table, offset rules (0x20 + dataOffset), what root names mean (ftData*, *figatree*, *_joint, coll_data).
HSD object model → docs/HSD.md: the scene graph — JOBJ (joints/bones + transforms), DOBJ (draw objects), POBJ (primitive objects: display lists), MOBJ (materials: color, alpha, TEV), TOBJ (textures: TPL image + wrap/scale), COBJ? (lights/cameras). Enumerate every HSD type we must handle for fighters and stages.
Geometry: POBJ display-list encoding for Melee's GX subset (primitive types, positions, normals, colors, UVs, skin weights).
Textures: TPL/CI image formats + TLUT lookup; which fonts/emblem/costume textures live in fighter DATs.
Animation → docs/ANIM.md: figatree — AOBJ-based joint/material/texture tracks, keyframes (constant/step/linear/hermite), track interpolation math; and how the fighter action table (in ftData*, 0x18-byte entries) links to byte slices inside Pl*AJ.dat.
Stages → docs/STAGES.md: stage DAT layout (GrNLa = FD …), sections, camera/scale/lights, and stage animations (platform movement, Whispy, Stadium transforms).

Exit: reading any of these docs + the community DAT thread should let us explain any byte in a Pl*.dat without peeking at Zig sources.

3Phase B — Build the C extractor (`tools/extract`)

Goal: original C code, module per format layer, each with a tiny self-check.

Module	Parses / produces	Self-check
`iso.c`	FST index; `dat_entry(path) -> bytes`	Known DAT paths resolve; sizes sane.
`dat.c`	DAT container → sections, nodes, names	Root names match the thread's naming.
`hsd_jobj.c`	JOBJ tree → bones, base/inv-bind transforms	Falco skeleton: 67 bones.
`hsd_dobj.c`	DOBJ/POBJ/MOBJ → primitive groups, indices, vertices	Falco: 26 prim groups.
`texture.c`	TOBJ/TLU + GX formats → RGBA	Falco: 24 textures, decodes to correct palette.
`colour.c`	costume → mesh DAT mapping (e.g. `PlFxOr/Bu/Gr/Nr`) + material swap	Each costume yields a distinct model.
`anim.c`	figatree → joint/material/texture keyframe tracks	Tracks interpolate identically at known frames.
`action.c`	`ftData` action table → linked `PlAJ.dat` slices	Falco: >300 actions, `Wait1` resolves.
`stage.c`	stage sections, scale, camera, lights, stage animations	FD: scale ~1.0, sections match.
`cache.c`	serialize all of the above; `meta.json`	Re-read round-trips byte-identical.

Exit: tools/extract reproduces every validation number above with zero Zig in the build. Cache never lands in git.

4Phase C — Renderer (the part our viewer already gestures at)

Asset loader (src/asset.{c,h}): mmap cache files into runtime structs; validate schema version + ISO fingerprint; load Fox/Falco on startup (ASSET_DIR=/data/cache).
Pose sampler: action_state → animation; sample at anim_frame (mind the +1 frame quirk); local → world transforms; mirror x when facing < 0.
2D rasterizer: project with the existing side-view camera; painter's-algorithm triangle fill; flat shading from MOBJ colors first, affine texture mapping next.
/api/pose?char=falco&action=Wait1&frame=30&facing=1 debug endpoint — iterate on one skeleton before touching playback.
Wire into /api/frame: followers/Nana, 4 ports, same character twice, Zelda/Sheik (key models by internal char id).
Stages: real stage meshes + stage animations driven by our existing slp_fod_at / slp_whispy_at / slp_stadium_at events.
Scale: all 26 characters + all costumes; 2.5D perspective + textures; later WebGL 3D is its own project.

5Decisions

#	Decision	Resolved
1	Who writes the extraction?	Us, in C, original code. Zig is reference-only. No litewing in the build.
2	What counts as "copying"?	Facts/offsets from the community thread = fine verbatim. Structure, idioms, and code = ours. No copy-paste from litewing.
3	Cache location	Dev box `/data/cache`, versioned + ISO-fingerprinted, never in git.
4	First projection	Side view (matches replay x/y; facing → mirror).
5	Textures in renderer	Flat-shaded first (silhouettes > accuracy); texture decode still built in Phase B for correctness.
6	Costume order	Default (0) for Fox/Falco first, then all colours.

6Risks

The reverse-engineering is the schedule — HSD node classification and GX display-list decoding are fiddly. Phase A is deliberately heavy to cut Phase B surprises.
ISO version sensitivity — confirm game.iso is the version the known-good numbers were measured on before trusting them.
Discipline to not copy — the threshold (data vs. code) is explicit above; review diffs against that line.
Perf — 960×540 software triangles + PNG could get slow; client-side pngCache already exists; optimize the triangle filler only if needed.
Actions without a figatree — some states have no animation; fall back to Idle/last pose.

7First concrete actions

Write docs/DAT.md + a C ISO/FST → DAT tool in tools/extract; dump a real PlFc.dat header, sections, roots, and names.
Decode the JOBJ tree of PlFc.dat; print the bone hierarchy + base transforms; hit 67 bones.
Decode one DOBJ/POBJ/MOBJ primitive group into vertices/indices; hit 26 prim groups.
Action table + figatree: resolve Falco's Wait1 / a blaster action; hit >300 actions.
First cached Falco asset + /api/pose render of Wait1 in 2D.