Papay-Shooter/docs/3D_ASSET_PIPELINE.md

Character Pipeline: source model → rigged → animated → in-game

One command turns a model into a playable character skin:

python tools/pipeline.py --uid <sketchfab-uid> --name space_marine

That downloads the model, moves the shared animation library onto it, writes assets/characters/skins/space_marine.glb plus a .rig.json sidecar, verifies the result, and registers it in skins.json. Restart the game — the skin is in the main menu dropdown, fully animated in first AND third person, synced in multiplayer.

The one rule: keep the model's own rig

If a model arrives with a skeleton, that skeleton is what ships. Its bones, its artist-painted weights, its separate per-part meshes, and its dedicated skirt/hair bone chains all survive; only the ANIMATION is moved onto it.

This is the whole point, and the pipeline used to do the opposite. The old route (strip_rig.py → autorig.py → merge_animations.py) discarded any foreign skeleton, joined every mesh into one, and rebound the result by weighting each vertex to its nearest four bone segments. It did that to work around a naming problem — the retarget matched bones by exact name — and the cost was the entire asset. Measured on the shipped taila.glb against the source it was built from:

	source model	old pipeline output
vertices pulled by BOTH legs	17 (0.1%)	2817 (16%), worst a dead 50/50
vertices at the full 4 influences	26%	86%
meshes (body / cloth / hair separable)	18	1
skirt bone chains	21 bones	0
hair bone chains	~50 bones	0
limb twist bones	8	0

A vertex pulled equally by both legs sits between them and stays there while they separate, stretching every triangle around it — that is the squashing and the "elongated boot". characters/skin_leg_repair.gd exists solely to undo this at load time, by snapping weights and deleting triangles on a mesh it has to rebuild every spawn.

The naming problem is now solved properly, in tools/rig_map.py, so nothing has to be thrown away.

The pipeline, step by step

Step	Tool	What it does
1. Find	python tools/sketchfab_import.py search "anime robot" --rigged	Search downloadable models (license shown per result)
2. Download	python tools/sketchfab_import.py download <uid>	GLB + license JSON into assets/characters/incoming/
3. Retarget	blender --background --python tools/retarget.py -- in.glb assets/characters/animations out.glb	Keeps the rig; moves the clip library onto it
3b. (unrigged only)	blender --background --python tools/autorig.py -- in.glb rigged.glb	Fits the library skeleton and solves weights — lossy, see below
4. Verify	blender --background --python tools/verify_character.py -- out.glb	Gates the build on the defects listed below
5. Register	(automatic in pipeline.py)	Copies to skins/, adds an entry to skins.json

pipeline.py chains all of it and picks the path automatically — it reads the glTF container to see whether a skins array is present. Useful flags:

• --input file.glb instead of --uid for local files.
• --rigged — force the keep-the-rig path (needed for FBX, which cannot be probed).
• --rebind — force the lossy path. Last resort.
• --height 1.6 — target character height in metres.

How the retarget works

tools/rig_map.py resolves both skeletons to ROLES and pairs them up, so bone names never have to match. It works structurally wherever a name would lie:

• Hips is found as the base of the longest non-limb, non-cosmetic chain — not by looking for "hips". Rigify calls it DEF-spine.
• The head is wherever that chain ends. A stock Rigify rig has no bone with "head" in its name at all; the head is DEF-spine.006.
• Chains of different lengths are matched by normalised position, so a 6-bone spine is driven by a 5-bone one.

tools/retarget.py then bakes each clip as a rest-relative delta — R_world = src_pose · src_rest⁻¹, applied as R_world · tgt_rest — rather than copying absolute world orientation, which would force the library's bone roll onto a mesh bound with a different one and twist every limb by a constant offset.

It also rebuilds parenting. A Rigify DEF-rig exports its chain roots parented straight to the armature root (Rigify drives them by constraint, not hierarchy), so on import the thighs, skirt and hair all hang off the root and would float in place while the body moves. Orphans are re-attached by anatomy where it is known and by rest geometry otherwise. Cloth may only ever attach to the trunk, never to a limb — anchor a skirt panel to the nearest bone and 16 of Taila's 21 land on a thigh, where the panel rides one leg like a trouser leg.

What is deliberately NOT driven

Skirt, hair, twist and face bones get no animation tracks at all. They rest relative to their parents and belong to the runtime instead. That split — clips animate the body, physics animates the cloth — is what makes clothes read as clothes, and it is why the exported clips only carry the ~53 bones they need (taila.glb went from 3.5 MB to 2.9 MB even after regaining its textures).

Secondary motion (characters/spring_bones.gd)

Skin weights can only ever make a garment a rigid shell of whatever it is weighted to: weight a skirt to the thighs and it becomes trousers, weight it to the hips and it becomes a bell that never moves. Neither is cloth. A skirt is cloth because it LAGS. That is inertia, and it has to be integrated, not skinned.

Each cloth bone is a damped spring holding its tip toward where rigidly following its parent would have put it, then pinned to the bone's length and pushed out of the leg capsules so a skirt swings AROUND a thigh rather than through it. Bone tip directions and capsule radii are measured from the model's own geometry at build time and stored in the sidecar — a glTF skeleton carries no bone tails, and Taila's skirt bones have no children either, so nothing in the skeleton says which way a panel hangs or how thick a thigh is.

Tuning per class (hair stiffer and lighter, skirt slacker and heavier) lives in SpringBones.TUNING.

Collision, and why cloth "never settles"

Getting the capsules wrong does not look like a collision bug — it looks like cloth that jitters forever. Four things had to be right:

1. A limb tapers. Taila's thigh is ~0.10 m across at the hip and ~0.055 m above the knee. Head and tail radii are stored separately and interpolated.
2. A leg's own vertices are not the leg. Most of the thigh is weighted to the TWIST bone; what is left dominated by DEF-thigh.L is the hip flare, which measured a 0.154 m radius — a 30 cm thigh. Twist children are folded in, and only vertices that clearly belong to the limb (dominant weight > 0.6) are counted, or a hip/thigh/glute blend puts the flare straight back. 2b. Measure the taper, do not pick a percentile of a pooled bucket. Every single-number answer is wrong at one end: the median leaves half the limb's surface OUTSIDE its own collider, so cloth pushed out to it is clear of the capsule while the thigh is visibly through it; a high percentile over- measures the shaft by 30% because the top bucket is still the hip. Take the 90th percentile in each of ten bands along the bone, drop the two contaminated end bands, and fit a line. Taila's thigh: 0.116 m at the hip to 0.063 m above the knee, against a real surface of 0.106 → 0.058 plus cloth thickness.
3. The rest pose must be a valid state. The artist modelled the skirt over these legs, so a capsule is capped per bone to just inside its own rest clearance. Without that, bones resting against the thigh were shoved out and pulled straight back in every frame — measured 13 of 70 cloth bones in permanent contact in a dead-still idle, a limit cycle that never decayed. That was the "hair and skirt never settle", and it also left the collider saturated and useless against real clipping.
4. Resolve as a rotation, sampled along the bone. A skirt panel is a sheet and its bone tip is at the far edge; pushing only the tip out leaves the middle of the panel inside the thigh — the thigh visibly clipping through the front of the skirt at a stride.
5. Carry the whole rotation back, not a tip. Most of what lifts a wide sheet off a thigh is rotation about the bone's OWN axis, and a twist moves the tip not at all. Reading the correction off the corrected tip therefore threw away the part that mattered: the solver measured 50–70 mm of thigh inside a panel, corrected it every frame, and the render never changed.
6. Apply the drape to the bone, not to the spring's target. The final rotation is measured FROM the drape target and applied TO the undraped basis, so rotating only the target cancelled out exactly at equilibrium and the drape was a silent no-op. This is why raising the drape weight — even to 0.99 — never moved a panel off a thigh, and it sent three rounds hunting the collision solver for a fault that was never there. In a chain, apply each link's share as the DIFFERENCE from what its parent already carries, or three segments at 0.45 compound to 1.35 of the thigh's swing at the hem.
7. Fix the deepest contact, not the one wanting the largest angle. The angle is depth ÷ leverage, so a graze on a hull point sitting almost on the bone's head outbids a 60 mm impalement further down and spends the pass rotating about a point that barely moves. Cap the turn so it never throws the rest of the panel further than the overlap being fixed, or a 5 mm contact swings a panel 34°.
8. Take the leg out of the REST TARGET too. While the spring's target sits inside a thigh, the collision pushes out and the spring hauls straight back forever. Resolving the target first makes resting on a leg an equilibrium.

The drape has to RIDE the limb, not copy its angle

This was the one that mattered, and it hid behind every other theory for a long time. The drape used to take the thigh's ROTATION and apply it to the panel's basis — which turns the panel about ITS OWN head, up at the waist. The thigh turns about the HIP, some 20 cm lower. The two arcs are nothing alike, so the panel never tracked the leg however high the drape weight went, and the thigh walked straight out through the front of the skirt.

Carry the bone's rest tip through the limb's FULL transform instead — pivot included — and aim the bone at the result:

carried = L_now * L_rest⁻¹ * tip_rest        # where the limb would take it
aim     = slerp(current_dir, carried_dir, w) # w = how much of it rests there

That is what "the cloth rides the leg" actually means, and it is the difference between a panel that gets overtaken and one that lifts over the thigh.

Two supporting pieces, both needed:

• Which limb, decided per frame. A static both-thigh split by rest distance is right for a panel hanging between the legs and wrong the moment they scissor — the halves cancel, the panel does not move, and the advancing thigh walks into it. DRAPE_BITE keeps a limb's share climbing once its gap goes negative, so the leg actually inside the cloth wins outright.
• The correction walks UP the chain (_lift_chains). A bone rotates about its own head, so it can move a point by at most twice that point's distance from the head — and the front panels' contacts sit 10-48 mm from their pivot against a thigh ~100 mm inside them. Their ancestors have 5-15x the lever, so a few degrees up the chain does what no local rotation could. Keep the per-bone cap small (4°): at 9° over 3 passes the whole panel hiked up.

Tuning that matters: DRAPE_MAX 0.55. At 0.9 the panel over-swings and hikes; at 0.0 the thigh passes straight through. Panels subdivide into 4 segments — 3 leaves the pivots too far from the contacts, 6 lets the chain curl up.

What a production cloth setup actually does

Everything above was arrived at by measurement, and it converged on a solver that was still missing the single most important piece. From Magica Cloth 2's BoneCloth skirt guide (the Unity asset most anime-style games use for exactly this garment):

Normally, the skirt bones are linked vertically due to the parent-child structure, but not horizontally … If horizontal bones are not connected, the accuracy of collision detection will be significantly reduced. This is the most important work when expressing a skirt with BoneCloth.

That was the gap. Every chain here solved alone, so each panel individually satisfied its constraints while the garment came apart. Four things were taken from that guide and its penetration page:

1. Horizontal links between chains (LINK_PASSES, _build_links). Distance constraints between the TIPS of bones that share mesh vertices, relaxed Gauss-Seidel after the springs and collision — Magica's "Near Point" connection. Adjacency comes from the sidecar's neighbours, which is shared vertex weight: the artist's answer to what is sewn to what, not a guess from names or rest distance. 215 links on Taila.
2. A lid across the waist. "Put one big sphere collider on your waist… it acts as a lid that prevents particles in the skirt from slipping into the body." There was no torso collider at all — leg capsules stop cloth going through a thigh, but nothing stopped a panel swinging INWARD into the pelvis, which is where several of the worst contacts sat. Marked lid in the sidecar so the drape does not treat it as a limb to be carried by, and sized from non-cloth geometry (including the garment measured the skirt itself, 0.24 m).
3. A graded bend clamp. "The start point can be bent up to 20 degrees and the tip up to 50." A flat limit either lets the waist collapse or stops the hem moving; MAX_SWING_ROOT/MAX_SWING_TIP interpolate along each chain. Collision still overrides it, which is Magica's precedence too.
4. Collision is not enough on its own — "if you make a vigorous movement, you will inevitably penetrate" — which is why the drape and the bodily shift exist alongside it rather than instead of it. Magica's two extra modes are Surface Penetration (needs the cloth weighted to the main bones) and Collider Penetration (for cloth that does not follow the leg animation). This skirt is deliberately the second kind, since any leg weighting on it tears.

The skirt "breaking" instead of stretching

A separate failure from clipping, and it looks like clipping: the front of the skirt is pushed aside and then splits, and the thigh shows through the split. Measure it with debug/cloth_stretch_check.gd, which skins every cloth triangle and compares each edge against its own rest length — a bone-level or capsule- level number cannot see it, because every individual bone is behaving.

It was the WEIGHTS. A band of skirt vertices carried DEF-thigh.L at 0.24 while the vertices next to them carried none, so when the thigh swung one followed it and its neighbour did not. Measured during a slide: 80 mm apart, 3.3x rest length. 0.24 of the thigh's ~0.35 m of travel is 84 mm, so that discontinuity was the whole of it.

Cloth is not skinned to the legs at all — tools/retarget.py:: unbind_cloth_from_legs. Smoothing the field instead of deleting it does not work, and the arithmetic says why: differential motion is the weight GRADIENT times how far the limb travels, so holding an edge under 10 mm against a thigh that moves 350 mm needs under 0.03 of weight difference across it — and this skirt's edges are ~48 mm long, so the falloff would have to span most of the character. Tried: a 1.7-radius falloff put 0.24 on one vertex and 0.00 on its neighbour and tore by 95 mm. After stripping, the worst cloth edge in the whole sweep grows 0.0 mm.

There was also a bind_cloth_to_legs() that did this deliberately, on the (wrong) conclusion that clipping was a weights problem. It is gone for the same reason, plus two of its own: a vertex weighted 0.9 to a thigh cannot be moved by its own cloth bone, so the solver loses the authority to push it out; and it poisoned the collider measurement, fitting a 0.28 m thigh out of 2258 skirt vertices.

Keeping the leg out of the weights costs the one thing the weights were doing, so the solver has to make it up:

• The capsule starts 30% down the thigh. The top of a thigh is hip, not limb. Cloth points there sit 20-30 mm from their own bone's head, and a rotation moves a point near its pivot by at most twice the lever — 48 mm against 86 mm of overlap. The solver spent all six passes saturated at its cap and still left 60-90 mm.
• The drape picks its limb per frame, by current proximity. Splitting a panel between both thighs by REST distance is right for a panel hanging between them and wrong the moment they scissor: the two opposite swings cancel, the panel does not move, and the advancing thigh walks into it.
• Chains are shifted bodily as well as turned (PUSH_MAX). Translation is the degree of freedom rotation lacks, and it is safe here precisely because the whole chain moves as one piece — unlike a weight gradient it cannot stretch the mesh, and neighbouring panels see the same contact and shift the same way, so the seams between them stay shut.
• Neighbouring cloth bones are known to the runtime. neighbours in the sidecar means SHARED VERTICES — the artist's own answer to which pieces are sewn together — and the drape is relaxed across them so sewn panels cannot be handed wildly different shares.

On the shipped materials the skirt now reads as one continuous sheet with an unbroken hem through the whole sweep, and the thighs emerge from below it.

Two measurement traps found while confirming that, both of which produced "remaining clipping" that was not there:

• A silhouette test cannot see clipping. When the legs are apart you see the FAR side of the skirt through the gap between them, and the thighs are legitimately in front of that. tools/measure_clipview.py counts those as hits. It is a good continuity check — a tear is a hole in the cloth mask — and a bad clipping check. Judge clipping from debug/skirt_probe.gd and the plain render.
• The torso is in front of the waistband, and that is correct. Counting all body pixels reported 5% of the cloth covered on a pose that is clean, so the measurement is cut at the hip joint. Colouring the legs separately does not work either: the bare thigh is part of the body mesh, and a per-surface colour caught only the boots.

Together those took idle penetration from 65 mm to ~1 mm and contacts from ~10 per frame to ~2. Damping then matters: 0.3 rang for about three visible oscillations after every step, so it sits near 0.6. The lag that makes a skirt read as cloth comes from inertia, not from low damping, so this costs nothing in the run cycle. Gravity is small for the same reason — a constant force offsets the resting tip by g/w², and the 5.0 first used pulled the hem 41 mm below where it was modelled, into the thigh it then had to be pushed out of.

Locomotion transitions

Two separate things change when a character starts running, and they have to arrive together:

• The clip, crossfaded by AnimationNodeTransition over BLEND_TIMES (~0.4 s for locomotion). Tier selection has hysteresis and a minimum dwell: without them, hard acceleration crossed Idle → Walk → Run in under a second and each crossfade cut off the one before it, giving Walk 0.19 s of a 0.40 s blend. With a dwell, a hard start now goes Idle → Run in one step.
• The procedural lean (ShooterPoseModifier), which is NOT part of the blend graph. The controller passes a normalised input direction, so it stepped 0 → 1 the instant a key went down and planted a full run posture in about a tenth of a second — the body snapping forward ahead of the run cycle. It is now scaled by actual speed and smoothed on its own slower rate (LEAN_SMOOTH), so it grows as the character accelerates.

Measure with debug/transition_check.gd: it reports clip changes, when the lean reaches 10% and 90%, and the worst single-frame change.

<model>.rig.json

Written next to every built GLB, so the runtime never re-guesses anatomy:

• roles — resolved bone per role. ShooterPoseModifier reads this; without it its hips/neck/head lookups silently missed on any rig that names them differently, and every lean and aim-pitch did nothing.
• chains — cloth/hair chains, with a measured tip vector per bone.
• colliders — leg capsules with measured radii.
• twist — limb twist bones.
• weights_authored — measured, not inferred from which branch ran. This is what decides whether SkinLegRepair runs at load time. A model that arrives unrigged still gets a sidecar, and its solved weights still need the repair.

Raw .glb and .json must be in the export include filter, or the sidecar is missing from a build and every character loses its cloth.

When a model has no skeleton

Then there is no authored weighting to keep and autorig.py fits the library skeleton with nearest-bone weights. This is genuinely lossy and the verifier reports it as warnings rather than failures, because no better result is available:

[WARN] cross-leg blending is limited to draping cloth — 590 verts (21.2%)
[WARN] influences look authored, not solved — 78% of verts carry 4 influences
[WARN] model keeps its per-part meshes — 1 meshes

miku is such a model. SkinLegRepair stays on for it at load time.

Prefer, in order: a model that ships rigged → Mixamo web (upload FBX/OBJ, place 7 markers, download rigged "without animations") → Reallusion AccuRig → UniRig / Tripo / Meshy. All of them produce a rig this pipeline will keep.

Materials

Anime models are very often exported "unlit": KHR_materials_unlit, a black baseColorFactor, and the real texture wired to emissiveTexture. Renderers honouring the unlit extension use base colour and ignore emission — so Blender reads black, never references the images, and imports with bpy.data.images empty. The character comes out a silhouette, and there is no node graph left to patch afterwards.

tools/gltf_fix.py rewrites the container before import: emissive becomes base colour, the unlit flag is dropped. The game shades characters with its own toon material off ALBEDO anyway.

Swapping the animation library

assets/characters/animations/_library.glb is the CC0 Quaternius Universal Animation Library. Clips map through LIBRARY_CLIP_MAP in tools/retarget.py (18 mapped: Idle, Walk, Run, Sprint, Jump, Fall, Land, CrouchIdle, CrouchWalk, Dash, Death, Hit, Dance, Grapple, PistolIdle/Shoot/Reload, Throw).

Characters no longer have to be rigged to the library's skeleton, so replacing _library.glb does not require re-rigging anything — just rebuild the skins. Missing clips are fine: the game falls back along sensible chains (Slide → CrouchIdle → Idle, WallRun → Run — see CLIP_FALLBACKS in characters/skinned_player_model.gd).

Measuring, not eyeballing

• blender --background --python tools/verify_character.py -- <glb> — the build gate. Every check corresponds to a defect this project actually shipped.
• godot --headless --path . -s res://debug/limb_deform_check.gd -- <glb> — skins the mesh itself and reports lengthwise stretch and cross-section loss against the skeleton's REST pose, so an unposed model reads exactly 1.00 and a wrong metric is visible immediately. Read its header before trusting a number you add: three earlier versions of this measurement were themselves wrong and nearly caused bad "fixes".
• godot --path . --windowed --resolution 1280x720 -s res://debug/anim_capture.gd -- <out_dir> <skin_id> — renders every movement state front and side.
• godot --headless --path . -s res://debug/cloth_settle_check.gd -- <glb> — does the cloth actually come to rest? Reports deviation, per-frame motion and, most usefully, which bones are penetrating a leg capsule and by how much. A steady non-zero contact count is a limit cycle that no amount of damping will fix.
• godot --headless --path . -s res://debug/transition_check.gd -- <glb> — clip changes and lean ramp when accelerating from a standstill.
• godot --path . --windowed --resolution 900x900 -s res://debug/skirt_clip_view.gd -- <out_dir> — the tool that settles "is the leg through the skirt?". Paints every cloth surface flat magenta and the body flat grey across a movement sweep, so grey inside the magenta is the leg in front of the cloth and grey outside it is just the leg past the hem. Those two look identical on the shipped materials and were guessed at, in both directions, for several rounds. It also saves a cloth-only frame, which separates clipping from a gap opening between panels.
• godot --headless --path . -s res://debug/skirt_probe.gd — per cloth bone, how deep the leg is inside it and whether the solver can SEE that depth (the rest-clearance allowance can hide it). Also reports the lever the solver has on the point, and its height above the hip joint — anything positive is inside the fictional sphere the capsule puts at the top of the thigh, not inside the leg.
• godot --headless --path . -s res://debug/cloth_stretch_check.gd — the tool for "the skirt breaks instead of stretching". Skins every cloth triangle over a movement sweep and compares each edge against its own rest length, reporting seams between panels separately from edges inside one. Nothing at bone or capsule level can see a tear, because each bone individually is fine. Restrict it to genuinely cloth-owned vertices — body surfaces carry stray cloth influence (one arm vertex measured 0.54 forearm, 0.35 skirt) and counting those made the skirt look like it was tearing by half a metre when the arm moved.
• godot --headless --path . -s res://debug/leg_radius_check.gd — the real per-band radius of each limb next to the capsule actually shipped.
• godot --path . --windowed --resolution 900x900 -s res://debug/idle_jitter_check.gd -- <out_dir> — consecutive frames of a still idle. Counting changed PIXELS between them is the only trustworthy settling measure: cloth_settle_check reports LOCAL bone rotation, and a parent's correction shows up as an equal and opposite delta on each of its segments, so a hem that has not moved on screen can read 18 deg/frame. Measured here: 24866 changed px/frame with collision on against 38594 with it off — the collision was damping the idle, not driving it.

Current Taila, worst over a run/walk/jump/fall/slide/dash sweep: knee cross-section 0.85–0.86, everything else 0.89–1.00, worst stretch 1.16.

How it works in-game

• SkinManager (autoload) reads skins.json at boot; the selection persists per user and syncs via synced_skin_id.
• SkinnedPlayerModel loads the GLB, reads the sidecar, installs ShooterPoseModifier (lean / ADS / slide / wall-run / weapon hold) then SpringBones — in that order, so the springs react to the FINAL body pose.
• SkinJointHelper runs for every model regardless of rig. It is not a weight repair: linear-blend skinning collapses any joint by cos(θ/2) however good the weights are. It subdivides the knee through helper bones. It resolves the joint's parent from the SKELETON — hardcoding DEF-thigh.L meant it silently did nothing on a rig with twist bones, and the knee measured 0.76 instead of 0.85.
• First person (owner): the model renders shadows-only (the camera sits inside the head). Press V to swap to the over-the-shoulder camera.
• Licensing: every Sketchfab download writes <name>.license.json. CC-BY models require crediting the author — surface these in a credits screen.

Troubleshooting

• Character is a black silhouette — unlit materials; see Materials. Check the built GLB actually has images.
• Model T-poses — the GLB has no animations, or the retarget produced a frozen rest pose. verify_character.py catches both.
• Skirt rides one leg — a cloth bone got parented to a limb. Cloth must anchor to the trunk only.
• Cloth is rigid — the sidecar is missing (check the export filter) or the clips are keying cloth bones (verify_character.py checks this).
• Cloth flies off the model — a spring instability. SpringBones substeps and clamps for exactly this; do not remove those guards.
• Cloth jitters and never settles — almost certainly a capsule the rest pose is already inside, not the damping. Run cloth_settle_check.gd and look at the contact count before touching TUNING.
• A limb clips through cloth — check the measured capsule in the sidecar is a believable size for that limb, and remember the collider only knows about the legs.
• Limbs squash at a stride — measure with limb_deform_check.gd before changing anything. Renders are repeatedly misleading; a slim anime leg at full stride genuinely looks stretched.