Nicholas ButzkeandClaude Opus 5
a97ccca13c
Three things the lab could not do. WRISTS. Each hand had one scalar, a twist about the barrel. That is the only axis a hand wrapping a cylinder is free in ONCE the arc onto the barrel is solved — which is true of the support hand, was never true of the trigger hand, and in neither case left a way to cock a wrist forward or break it inward. Both now take pitch, yaw and roll, applied in the GUN's frame so the three sliders mean the same thing whether the muzzle is down at low ready or level down the sights. Zero is exactly the old behaviour, since the roll term defaulted to zero too. HAND POINTS. `gun_stock` and `gun_fore` are the distances along the weapon at which each hand sits, and they were labelled by what they measure rather than by whose hand it is. They now say TRIGGER and SUPPORT, next to the off-barrel shifts for the same two hands, so the four controls that place a hand read as four controls that place a hand. POSES. The hold's knobs are now per pose, and the lab shows one pose's at a time. Half of them mean something different at low ready than down the sights; showing both sets at once meant every slider on screen was for one of two poses with nothing saying which. Selecting a pose rebuilds the panel. Two poses, not four, and deliberately: the runtime blends between exactly two holds on `ads`. Running and Crouched are locomotion states that still use the low-ready hold, so they edit the same numbers — and the heading says so, rather than letting someone tune "Running" and wonder why standing still changed. Offering four independent tunings would be inventing a capability the code does not have, and the fourth would silently do nothing. `pitch` is the case that forced the design: down the sights the muzzle follows the CAMERA, so there is nothing there to tune. It exists at low ready and nowhere else, and a spec table where a knob names the poses it applies to is what lets that be said instead of shipping a control that does nothing. hold_pose_check asserts both halves — that no pose shows another's knobs, that aiming offers no muzzle pitch, that the heading names the hold being edited, and that all twelve wrist axes turn the hand they name. Its first version reported every wrist axis as moving the hand by 0.0 degrees, which is precisely the answer it would have given if the wrists had never been implemented: it read `get_bone_pose_rotation` from a SceneTree script, and Godot restores every bone's local pose after the modifier pass. The repo has a reference section about exactly this and it still cost a cycle. Measured through a PoseProbe, every axis turns its hand ~20 degrees. Co-Authored-By: Claude Opus 5 <[email protected]>
319 lines
17 KiB
Markdown
319 lines
17 KiB
Markdown
---
|
|
name: character-pipeline
|
|
description: Import, rig, stylize and animate an anime-styled character into Papaya-Shooter as a selectable skin — keeping the model's own skeleton, artist weights and separate body/cloth/hair meshes, with cloth and hair driven by the spring solver. Use when adding a new playable character, re-importing an existing one, debugging skinning/cloth/hair/animation problems on a character, or changing the cel-shaded look. Triggers on "add a character", "import a skin", "new playable model", "skirt clipping", "hair flailing", "T-posing", "character looks squashed".
|
|
---
|
|
|
|
# Character pipeline
|
|
|
|
Turns a source model into a playable, cel-shaded, cloth-simulated character skin.
|
|
|
|
The whole design follows one principle, which is also what the Hoyoverse-class
|
|
anime pipelines (Genshin / Star Rail / Zenless Zone Zero) are built on:
|
|
|
|
> **The character is not one object. It is a body, a set of garments, and hair —
|
|
> authored separately, rigged separately, and moved by different systems.**
|
|
> The body is skinned and animated. The garments and hair are bone chains that
|
|
> the animation never touches; physics moves them. Keeping those separate is
|
|
> what makes the result read as an anime character instead of a mannequin in a
|
|
> painted-on costume.
|
|
|
|
Everything below exists to protect that separation.
|
|
|
|
## The one rule
|
|
|
|
**If a model arrives with a skeleton, that skeleton ships.** Its bones, its
|
|
artist-painted weights, its per-part meshes and its skirt/hair chains all
|
|
survive. Only the ANIMATION is moved onto it.
|
|
|
|
The old route (`strip_rig.py` → `autorig.py` → `merge_animations.py`) solved a
|
|
bone-*naming* problem by destroying the asset — 18 meshes became 1, 21 skirt
|
|
bones and ~50 hair bones became 0, and 16% of vertices ended up pulled by both
|
|
legs. `tools/rig_map.py` solves naming properly now. **Never reach for
|
|
`strip_rig.py` or `--rebind`** unless the model genuinely has no skeleton at all.
|
|
|
|
## Doing it
|
|
|
|
```bash
|
|
# From an already-rigged local model (the normal case)
|
|
python tools/pipeline.py --input assets/characters/incoming/<name>.glb --name <name> --rigged
|
|
|
|
# From a Sketchfab UID (needs SKETCHFAB_API_TOKEN)
|
|
python tools/pipeline.py --uid <uid> --name <name>
|
|
|
|
# From an unrigged mesh — auto-rigs, and accepts the quality loss
|
|
python tools/pipeline.py --input <mesh.glb> --name <name>
|
|
|
|
# Rigged (or auto-rigged) but with NO skirt/hair bones — grow them, or the
|
|
# costume is welded solid and the spring solver has nothing to simulate
|
|
python tools/pipeline.py --input <model.glb> --name <name> --rigged --grow-cloth
|
|
```
|
|
|
|
Then, once, so Godot sees the new files:
|
|
|
|
```bash
|
|
godot --headless --path . --import
|
|
```
|
|
|
|
The result is `assets/characters/skins/<name>.glb` + `<name>.rig.json`, and a
|
|
registry entry in `skins.json` that `SkinManager` picks up with no code change.
|
|
|
|
Blender is required (`BLENDER_PATH`, or auto-found under
|
|
`C:\Program Files\Blender Foundation`). Godot lives at
|
|
`C:\Program Files\Godot\Godot_v4.7-stable_win64_console.exe`.
|
|
|
|
## The stages, and what each one protects
|
|
|
|
| Stage | Where | Protects |
|
|
|---|---|---|
|
|
| Fix unlit/emissive materials | `tools/gltf_fix.py` | Textures surviving import at all |
|
|
| Resolve bone ROLES, not names | `tools/rig_map.py` | The model's own skeleton |
|
|
| Rebuild parenting | `retarget.py::rebuild_hierarchy` | Limbs/cloth following the hips |
|
|
| Grow cloth chains (opt-in) | `tools/cloth_bones.py` | A costume that has no bones being able to move at all |
|
|
| Subdivide cloth panels | `retarget.py::subdivide_cloth_panels` | A skirt being able to bend at all |
|
|
| Retarget clips as rest-relative deltas | `retarget.py::retarget_clip` | Limbs not being twisted by foreign bone roll |
|
|
| Leave cosmetic bones unkeyed | `export_optimize_animation_keep_anim_armature=False` | Physics owning the cloth |
|
|
| Classify every SURFACE | `tools/surface_map.py` | The runtime never re-guessing what a surface is |
|
|
| Write the rig sidecar | `retarget.py::describe_rig` | The runtime never re-guessing anatomy |
|
|
| Cel look, per surface class | `LevelMaterials.apply_character_look` | Hair not reading as a solid dark cap |
|
|
| Cloth + hair | `characters/spring_bones.gd` | Clothes reading as clothes |
|
|
| Per-character judgement calls | `characters/tuning_store.gd` | Art direction not becoming another constant |
|
|
|
|
## The surface table
|
|
|
|
The sidecar carries a `surfaces` list saying what each mesh surface IS — `body`,
|
|
`cloth`, `hair`, `accessory`, or `linework` (the model's own ink shell, which is
|
|
not a surface of the character at all). It is keyed on the MATERIAL name, because
|
|
every character in this game arrives with its meshes called `Object_7` through
|
|
`Object_32` while material names survive the glTF round trip intact.
|
|
|
|
`SkinSurfaces` reads it and `apply_character_look` acts on it: hair takes a much
|
|
thinner outline than the body, cloth a heavier one and a crisper terminator,
|
|
accessories the heaviest. `SkinnedPlayerModel.surfaces_of(cls)` answers the
|
|
question for anything else that needs it.
|
|
|
|
Backfill a character that predates it, without re-importing:
|
|
|
|
```bash
|
|
blender --background --python tools/surface_map.py -- \
|
|
assets/characters/skins/<name>.glb
|
|
```
|
|
|
|
New imports get it from `describe_rig`, built from the same chains the solver
|
|
uses, so the surface table and the cloth solver can never disagree about which
|
|
bones are a skirt.
|
|
|
|
## Per-character judgement
|
|
|
|
Anything derivable from the skeleton is derived. What is left is genuinely an
|
|
artist's call, and it lives in layered JSON rather than in a constant:
|
|
|
|
| File | Scope | Class |
|
|
|---|---|---|
|
|
| `assets/characters/weapon_holds.json` | character + weapon | `WeaponHoldTuning` |
|
|
| `assets/characters/rig_anchors.json` | character | `RigAnchors` |
|
|
|
|
Both layer `defaults` → `skins.<skin>._all` → `skins.<skin>.<subject>` through
|
|
`TuningStore`. An absent file means "use what the code derives", so nothing here
|
|
is required for the game to run. Adding a knob is adding a row to a `KNOBS` spec
|
|
table — the lab builds its whole UI from those.
|
|
|
|
`RigAnchors` is where "the grip sits here in the palm" lives. A hand bone's
|
|
origin is the WRIST; how far down the palm a grip belongs depends on the
|
|
character's hand and cannot be derived. It defaults to identity, and identity is
|
|
exactly the derived mount. Do **not** put a fixed rotation on the weapon mount
|
|
instead — a bone attachment is expressed in the BONE's axes, no two rigs agree on
|
|
those, and that constant is why the hand mount points were once wrong on every
|
|
character.
|
|
|
|
## The rig lab
|
|
|
|
```bash
|
|
godot --path . res://debug/rig_lab.tscn
|
|
```
|
|
|
|
Pick a character, a weapon, a pose or a single clip. Drag sliders for the HOLD
|
|
(character + weapon) and the ANCHORS (character), and save.
|
|
|
|
**The HOLD is per pose.** The runtime blends between exactly two holds, on
|
|
`ads`, so the lab offers two: low ready and aiming. Selecting a pose rebuilds
|
|
the hold sliders to that pose's — you never see a control belonging to the pose
|
|
you are not adjusting. Running and Crouched use the low-ready hold, and the
|
|
heading says so rather than letting someone tune "Running" and wonder why
|
|
standing still changed. `pitch` exists at low ready only: down the sights the
|
|
muzzle follows the camera, so there is nothing there to tune, and a slider that
|
|
does nothing is worse than a missing one.
|
|
|
|
Knobs that describe the WEAPON and the hands on it — where each hand sits along
|
|
it and off its barrel line, the finger curls, the weapon size — are shared,
|
|
because shouldering a gun does not move the hand along it. Both wrists take
|
|
pitch, yaw and roll in the gun's own frame, per pose. Click a surface class
|
|
to isolate it — that is how the classifier gets checked: click `hair` and
|
|
anything else still standing was misclassified.
|
|
|
|
**Drag the coloured markers.** They ARE the anchor points the hands are solved
|
|
onto — red the trigger grip, green the support hand, blue the buttstock — and
|
|
the one under the mouse swells and draws through the body so it can be grabbed
|
|
where the hands would otherwise hide it.
|
|
|
|
Dragging an anchor is not the same as any slider:
|
|
|
|
| | moves |
|
|
|---|---|
|
|
| `grip_offset` (ANCHORS) | the GUN, inside the fist |
|
|
| `gun_fore` / `gun_stock` (HOLD) | the hands ALONG the weapon's own axis |
|
|
| dragging a marker | the anchor itself, in three dimensions |
|
|
|
|
That distinction was the gap. `gun_fore` and `gun_stock` are distances along the
|
|
barrel, so the two hand anchors could slide up and down the gun and nowhere
|
|
else — no use for a handguard below the bore, an angled foregrip, or a pistol
|
|
whose grip is nowhere near its barrel line. A drag writes `grip_shift` or
|
|
`fore_shift` in the gun's own across/up/along frame, so a sideways nudge stays
|
|
sideways as the weapon pitches; the buttstock marker writes the shoulder pocket
|
|
for whichever pose is showing.
|
|
|
|
The anchors sit on the shoulder, and the arm chasing them moves the shoulder, so
|
|
a drag settles at 0.77x-1.13x of the mouse. Small enough to ignore — you stop
|
|
when it looks right — and measured, not assumed.
|
|
|
|
## Read before you touch anything
|
|
|
|
Load the reference that matches what you are doing. They are short and each one
|
|
is a list of things that cost a debugging cycle to learn.
|
|
|
|
- **`references/failure-modes.md`** — **read this first.** Seven characters
|
|
shipped "All checks passed" and four were visibly broken. What each failure
|
|
was, why the suite missed it, and the rule that generalises it to any model.
|
|
- **`references/separation.md`** — body vs. garments vs. hair: what must stay
|
|
separate, how cloth chains are detected and classed, why cloth is never
|
|
skinned to a leg, and the ZZZ-convention mapping.
|
|
- **`references/growing-cloth-bones.md`** — what to do when the source has no
|
|
skirt or hair bones: how the chains are fitted to the geometry and re-weighted.
|
|
- **`references/rigging.md`** — role resolution, hierarchy rebuild, cloth panel
|
|
subdivision, twist bones, joint helpers, the retarget maths.
|
|
- **`references/cloth-and-hair.md`** — the position-based spring solver, its
|
|
colliders, per-class tuning, collision hulls, LOD and cost.
|
|
- **`references/stylization.md`** — cel shading, outlines, the imported
|
|
line-work trap, eyes, materials.
|
|
- **`references/verification.md`** — every measuring tool, what each one
|
|
actually measures, and the pose-reading trap that invalidated all of them
|
|
once. **Read this before trusting any measurement.**
|
|
|
|
## Non-negotiables
|
|
|
|
1. **Never join meshes.** Per-part meshes are how body, cloth and hair stay
|
|
separable — for materials, for the outline pass, and for the cloth solver's
|
|
hull extraction.
|
|
2. **Never key cosmetic bones.** If a clip has tracks on skirt/hair bones, the
|
|
AnimationPlayer overwrites the solver every frame and the cloth goes rigid.
|
|
3. **Never skin cloth to a leg.** A vertex weighted 0.9 to a thigh cannot be
|
|
moved by its own cloth bone, so the solver loses the authority to push it out
|
|
of that thigh — and the leg still overtakes it. There was a
|
|
`bind_cloth_to_legs()`; it is deleted, and the note above its grave in
|
|
`retarget.py` says why.
|
|
4. **Never run `SkinLegRepair` on authored weights.** It snaps weights and
|
|
deletes triangles. It exists only to undo auto-rigging. It is gated on
|
|
`weights_authored`, which is MEASURED, not assumed.
|
|
5. **Measure from inside the modifier pass.** See `references/verification.md`.
|
|
6. **Never assume an axis.** Up, forward and scale are all measurable from the
|
|
skeleton. Assuming +Z is up scaled three characters 7x and left them on their
|
|
backs — and the height check passed on every one of them, because the number
|
|
being normalised always comes out right whether or not it was the right
|
|
number. See `references/failure-modes.md`.
|
|
7. **Never look a bone up by name.** `tools/rig_map.py` resolves roles and writes
|
|
them to the sidecar so nothing downstream has to guess. Any hardcoded spelling
|
|
— `_find_bone(["RightHand", ...])`, a `thigh`/`shin` substring test — is a rig
|
|
this project has not met yet. Four characters could not hold a gun because of
|
|
exactly one such lookup.
|
|
|
|
## After importing
|
|
|
|
```bash
|
|
godot --headless --path . -s res://debug/spawn_smoke_test.gd # 29 checks
|
|
godot --headless --path . -s res://debug/surface_class_check.gd # every surface classified
|
|
godot --headless --path . -s res://debug/character_picker_check.gd # the escape-menu roster
|
|
godot --headless --path . -s res://debug/rig_anchor_check.gd # anchors move the weapon
|
|
godot --headless --path . -s res://debug/cloth_clip_check.gd # leg-through-cloth
|
|
godot --headless --path . -s res://debug/cloth_settle_check.gd # idle stability
|
|
godot --headless --path . -s res://debug/cloth_perf_check.gd # ms per character
|
|
godot --headless --path . -s res://debug/travel_dir_check.gd # legs face travel
|
|
```
|
|
|
|
And LOOK at it, which is where four of the last five real defects were found:
|
|
|
|
```bash
|
|
godot --path . -s res://debug/roster_capture.gd -- <dir> # every character, one shot each
|
|
godot --path . -s res://debug/ui_capture.gd -- <dir> # every menu screen
|
|
godot --path . res://debug/rig_lab.tscn -- shot <png> <skin>
|
|
```
|
|
|
|
`surface_class_check` fails on any surface that falls through to the heuristic
|
|
rather than resolving from the table. That is deliberate: a model whose names
|
|
stopped matching still RENDERS, because the fallback catches it — it just
|
|
quietly loses its per-class art direction, which is exactly the kind of
|
|
regression nothing else would report.
|
|
|
|
What "good" looks like on Taila, for calibration:
|
|
|
|
| Measure | Good | Bad |
|
|
|---|---|---|
|
|
| Idle skirt movement | < 0.1 deg/frame | 0.5+, or never decaying |
|
|
| Leg inside cloth, idle/walk | < 25 mm | 100 mm |
|
|
| Leg inside cloth, run/slide/dash | ~95 mm *(current, unsolved)* | — |
|
|
| Solver cost | ~2.6 ms/character | 10 ms |
|
|
| Stride vs. travel direction | < 10° (except a capped sidestep) | 90° |
|
|
| Bind-pose AABB | tall on the hips→head axis, others < 2.5 m | tallest axis is depth |
|
|
|
|
## Characters currently shipping
|
|
|
|
The six in `skins.json`, with what `surface_class_check` reports:
|
|
|
|
| Skin | Source | Cloth chains | Surfaces |
|
|
|---|---|---|---|
|
|
| taila | rigged, Sketchfab CC-BY | 35 | 18 — body 6, cloth 7, hair 1, linework 4 |
|
|
| kiyoko | VRoid, CC-BY | 20 | 13 — body 8, cloth 3, hair 2 |
|
|
| aria | VRoid, CC-BY | 15 | 15 — body 9, cloth 4, hair 2 |
|
|
| momo | VRoid, CC-BY | 9 | 5 — body 2, cloth 1, hair 2 |
|
|
| miku | unrigged source, auto-rigged | 0 | 4 — body 3, hair 1 *(one mesh, four surfaces)* |
|
|
| mannequin | Quaternius CC0, from the animation library | 0 | 2 — body 2 |
|
|
|
|
## Known-unsolved
|
|
|
|
- **Peak cloth clipping** in a run, slide and dash sits at ~95 mm of thigh
|
|
inside the skirt. Idle, walk and fall are clean. The solver sees the contact
|
|
and pushes on it every iteration; the remaining gap is a standing fight
|
|
between the collision and the garment's own shape constraints.
|
|
- **No foot IK.** Feet do not plant on ground height, so stairs and uneven
|
|
ground read as sliding.
|
|
- **No strafe or backpedal clips.** Direction is conveyed by yawing the hips
|
|
(`SkinnedPlayerModel._update_travel`), which is capped, so a pure sidestep
|
|
still runs its legs ~40° off the direction of travel.
|
|
- **momo's idle pose is wrong** — arms overhead and a pinched waist. Every
|
|
assertion passes on her: she loads, animates, classifies and mounts a weapon.
|
|
It shows up only in `roster_capture`.
|
|
|
|
**Diagnosed, not fixed.** Her `driven_bones` contains `Root_001` through
|
|
`Root_007` — and those are her HAIR roots. The Godot surface dump shows
|
|
`Hair_A` dominated by `Root_001_001:3203`, `Root_007:3203`, `Root_005:2642`.
|
|
So the animation is keying bones that physics is supposed to own, which is
|
|
non-negotiable #2 being violated by the role resolver rather than by a clip.
|
|
The corroboration is in the surface table: her hair surfaces report only 1.8%
|
|
and 9.4% chain share, because most of their vertices belong to `Root_00N`,
|
|
which is in no chain at all.
|
|
|
|
`Root_00N` matches no COSMETIC stem, so `is_cosmetic` does not catch it and
|
|
nothing keeps it out of the driven set. Fixing it by adding "root" to the
|
|
stems would be wrong — a rig whose actual root is called `Root` would lose its
|
|
hips. The fix is structural: a bone whose geometry is dominated by a mesh
|
|
classified `hair` is a hair bone, whatever it is called. The surface table
|
|
now makes that answerable at build time, which it was not when this rig was
|
|
imported. Not attempted here — it needs a Blender re-run and re-verification
|
|
of all six characters.
|
|
|
|
Her `head` role is also wrong (`Unused_Noname_010`, when a real `Head` bone
|
|
exists and is in her spine chain), and her spine chain runs two junk bones
|
|
PAST the head. Probably the same import; worth fixing in the same pass.
|
|
- **A stray `Icosphere` ships inside every skin GLB** — 42 vertices, no parent,
|
|
no vertex groups. It rides in from the animation library. Harmless, and now
|
|
skipped by construction rather than by name in `surface_map`, but the export
|
|
should not be producing it.
|