Dotts/Papay-Shooter
1
0
Fork 0
Code Issues 2 Pull Requests Actions Packages Projects Releases Wiki Activity

feat(characters): say what every surface IS, and light it accordingly

Browse Source 
A character has arrived as separate body, garment and hair meshes since the
pipeline stopped joining them — but nothing recorded which was which, so every
system downstream re-guessed from the material. That guess ("untextured and
nearly black means ink") had already rendered the mannequin's flat yellow body
as a black silhouette once.

The question is answerable once, at build time, where the mesh, the weights and
the skeleton are all in hand. tools/surface_map.py answers it three ways, in
order of how much it trusts them: the material name, which on VRoid exports is
formal and on hand-authored models is still explicit; the weights, which are
decisive when the name says nothing — a surface pulled by the skirt chain is a
skirt whatever it is called; and the material flags, which catch the model's own
line-work. The answer goes in the rig sidecar next to the roles and the chains,
and SkinSurfaces reads it.

All eighteen of Taila's surfaces, and every surface of the other five skins,
now resolve from the table with nothing falling through to the heuristic
(debug/surface_class_check.gd). The heuristic stays as the fallback, which is
the one job it was ever right for.

What that buys immediately is per-class art direction, which was impossible
while every surface had to take numbers calibrated on skin. Hair takes a much
thinner line — at the body's 5 mm each strand's hull swallows its neighbour and
the head reads as a solid dark cap. Cloth takes a heavier line and a crisper
terminator, because a garment's silhouette is most of what separates a character
from the background at range. Accessories take the heaviest. `body` is unchanged
on purpose, so the look this was all calibrated against does not move.

That required moving the outline from the instance to the surface: Miku's body,
face and hair are three surfaces of ONE mesh, so an instance-wide overlay could
only ever give all three the same weight.

Two things found on the way, fixed here because they are one line each: the
surface classifier skips meshes with no vertex groups, which drops the stray
42-vertex Icosphere that rides inside every shipped skin — two older tools
already skipped it by spelling its name — and load_model now clears _rig_info,
which a model with no skeleton used to inherit from the last character loaded.

Co-Authored-By: Claude Opus 5 <[email protected]>
This commit is contained in:
Nicholas Butzke
2026-07-27 14:35:06 -04:00
co-authored by Claude Opus 5
parent 33d07b3717
commit 53f175ed6d
16 changed files with 17393 additions and 15884 deletions
Download Patch File Download Diff File Expand all files Collapse all files
tools/surface_map.py
+448
View File
@@ -0,0 +1,448 @@
#!/usr/bin/env python3
"""
Say what every surface of a character IS: body, cloth, hair or accessory.
The pipeline already refuses to join meshes, so a character arrives at runtime
as the eighteen separate pieces the artist authored. But nothing recorded what
those pieces WERE, so every system downstream had to re-guess from scratch:
`LevelMaterials.apply_character_look` decides "is this the model's own ink
line-work?" by asking whether the material is untextured and nearly black, and
that heuristic has already cost one debugging cycle — it rendered Quaternius'
flat-coloured mannequin as a black silhouette, because a yellow body and a
lilac joint are untextured too.
Guessing per-frame from a material is the wrong place to ask the question. The
answer is knowable ONCE, at build time, where the mesh, the weights and the
skeleton are all in hand:
* the material NAME is usually explicit, and on VRoid exports it is formal —
`N00_000_00_Body_00_SKIN_Instance` carries its own class infix, and every
VRoid character in the game uses SKIN / FACE / EYE / HAIR / CLOTH;
* the WEIGHTS are decisive when the name says nothing: a surface whose
vertices are pulled by the skirt chain is a skirt, whatever it is called;
* the MATERIAL FLAGS catch the model's own line-work, which is the one class
that is not really a surface of the character at all.
So the classification is written into `<name>.rig.json` beside the roles and the
cloth chains, and the runtime reads it instead of re-deriving it. That is what
lets per-class art direction exist at all — a different outline width on hair
than on the body, cloth that takes the garment shading, accessories that keep
their crisp banding — and it gives the rig lab something to show and to let an
artist correct.
Two ways in:
# As part of an import — retarget.py calls describe_surfaces() directly.
# Backfilling a character that is already exported, without re-importing it
blender --background --python tools/surface_map.py -- \
assets/characters/skins/taila.glb
The backfill route loads the SHIPPED glb, so the names it records are by
construction the names Godot will see. Nothing else is touched: the surface
table is merged into the existing sidecar and every other key is left alone.
"""
import json
import os
import re
import sys
sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
import rig_map
# ── What a surface can be ────────────────────────────────────────────────────
#
# Four classes, because that is what the rest of the game wants to reason about,
# plus one that is not a surface of the character at all.
#
# body skin, face, eyes — anything that is the character themselves
# cloth garments: tops, skirts, shoes, sleeves, capes
# hair hair, including the parts of it that are welded to the head
# accessory rigid worn props: hats, glasses, bags, jewellery, horns
# linework the model's own outline shell — drawn, not worn (see below)
#
# `detail` narrows it without forcing a new class on anyone who does not care.
BODY, CLOTH, HAIR, ACCESSORY, LINEWORK = (
"body", "cloth", "hair", "accessory", "linework")
# Material-name tokens, most specific class first. Order is the whole design
# here: `N00_000_00_EyeIris_00_EYE_Instance` carries both "eye" and "iris" and
# must not be caught by a later, broader rule, while `FaceEyeline` tokenises to
# "face" + "eyeline" and stays a face.
NAME_RULES = [
({"eye", "eyes", "iris", "pupil", "sclera", "cornea"}, BODY, "eyes"),
({"hair", "bang", "bangs", "fringe", "ponytail", "braid", "sidelock"},
HAIR, "hair"),
({"brow", "eyebrow", "eyeline", "eyelash", "lash", "mouth", "lip", "teeth",
"tooth", "tongue", "nose", "face", "facial"}, BODY, "face"),
({"cloth", "clothes", "clothing", "garment", "outfit", "costume", "uniform",
"top", "tops", "bottom", "bottoms", "skirt", "dress", "onepiece", "shirt",
"blouse", "jacket", "coat", "hoodie", "pants", "trousers", "shorts",
"socks", "stocking", "stockings", "shoe", "shoes", "boot", "boots",
"sleeve", "apron", "cape", "cloak", "scarf", "tie", "necktie", "collar",
"ribbon", "frill", "underwear", "bra", "panty", "swimsuit"},
CLOTH, "garment"),
({"accessory", "acc", "hat", "cap", "helmet", "glasses", "goggle",
"goggles", "mask", "bag", "backpack", "belt", "buckle", "earring",
"necklace", "choker", "bracelet", "pin", "clip", "badge", "horn", "wing",
"weapon", "prop", "item", "jewel", "gem", "strap", "headphone",
"headphones", "antenna", "halo"}, ACCESSORY, "accessory"),
({"body", "skin", "torso", "chest", "arm", "arms", "leg", "legs", "hand",
"hands", "foot", "feet", "head"}, BODY, "skin"),
]
# Spring-chain classes, grouped by what the SURFACE they pull should be called.
# `spring_class` already tells the solver how a chain should behave; this says
# what the geometry hanging off it is. Bust bones move like cloth and belong to
# the body, which is exactly why the two questions are asked separately.
CHAIN_TO_SURFACE = {
"hair": HAIR,
"skirt": CLOTH, "cloth": CLOTH, "coat": CLOTH, "cape": CLOTH,
"scarf": CLOTH, "sleeve": CLOTH, "ribbon": CLOTH,
"breast": BODY, "bust": BODY,
"tail": ACCESSORY, "feather": ACCESSORY, "antenna": ACCESSORY,
"wing": ACCESSORY, "strap": ACCESSORY,
}
# How much of a surface has to hang off one kind of chain before that decides
# it. Deliberately low: VRoid welds the whole cap of the hair to the head bone
# and springs only the strands, so kiyoko's hair mesh is 85% head — a majority
# rule would call it skin. A surface with a sixth of itself on skirt bones is a
# skirt; nothing else on a character has any weight on those bones at all.
CHAIN_SHARE = 0.05
# Darker than this, with no texture, and it is ink rather than a colour. Kept
# identical to LevelMaterials._is_line_work, which this replaces — the two must
# agree, or a model with no surface table would render differently from one
# with it.
INK_LEVEL = 0.18
def datablock_name(name):
"""A Blender datablock name as the glTF it came from spelled it.
Blender uniquifies a name it has already seen by appending `.001`, so a
second import of the same character in one session gives `ClothA.001`.
That, and only that, is stripped.
Explicitly NOT `rig_map.strip_gltf_suffix`, which removes a trailing
`_<digits>` — right for bones, whose names the glTF importer really does
suffix with a node index, and catastrophic here: every mesh in this game is
called something like `Object_15`, and stripping would collapse all
eighteen of Taila's to `Object`.
"""
return re.sub(r"\.\d{3}$", "", name)
def _tokens(name):
"""Material name -> lowercase word tokens, splitting camelCase too.
`N00_001_03_Bottoms_01_CLOTH_Instance` -> bottoms, cloth, instance (plus
the numbers), and `EyesFullBlack` -> eyes, full, black. Without the
camelCase split the VRoid names still work and the hand-authored ones do
not, and Taila's are all hand-authored.
"""
spaced = re.sub(r"(?<=[a-z0-9])(?=[A-Z])", " ", name)
return [t for t in re.split(r"[^A-Za-z0-9]+", spaced.lower()) if t]
def classify_by_name(material_name):
"""(class, detail) from the material name alone, or (None, None)."""
toks = set(_tokens(material_name))
for stems, cls, detail in NAME_RULES:
hit = toks & stems
if hit:
return cls, detail
return None, None
def classify_linework(material_name, textured, albedo, front_culled):
"""Is this untextured surface part of the model's own DRAWING?
Lifted wholesale from the runtime rule it replaces, including its history:
"no albedo texture" alone is not the question, and answering it that way
hid the mannequin's yellow body and lilac joints as though they were an
outline shell. What actually distinguishes line-work is being BLACK, being
drawn inside-out, or saying outright that it is an eye card.
Returns a detail string, or None if this is an ordinary flat-coloured
surface that should shade normally.
"""
if textured:
return None
lower = material_name.lower()
if lower.startswith("eyes"):
# A lash or an iris card — a real facial feature, kept. "HL" is the
# glint in the pupil, and that one really is meant to be white.
return "eyes_highlight" if "hl" in lower else "eyes_ink"
if front_culled:
return "outline_hull"
if max(albedo[0], albedo[1], albedo[2]) < INK_LEVEL:
return "outline_hull"
return None
def classify(material_name, textured, albedo, front_culled, chain_shares,
cosmetic_share):
"""Everything known about one surface -> (class, detail, evidence).
Name first where it is explicit, because it is the artist's own statement of
intent and it survives every transform the pipeline applies. Weights second,
because they are the ground truth for anything the name does not say. The
fallback is `body`, which is the safe answer: it gets ordinary character
shading, which is what an unrecognised surface should get.
"""
ink = classify_linework(material_name, textured, albedo, front_culled)
if ink == "outline_hull":
return LINEWORK, ink, "material is untextured ink or inside-out"
if ink is not None:
return BODY, ink, "material names itself an eye card"
cls, detail = classify_by_name(material_name)
if cls is not None:
return cls, detail, "material name says '%s'" % detail
# Nothing in the name. Ask the skeleton what is pulling this geometry.
best = max(chain_shares.items(), key=lambda kv: kv[1], default=(None, 0.0))
if best[0] is not None and best[1] >= CHAIN_SHARE:
surface_cls = CHAIN_TO_SURFACE.get(best[0], CLOTH)
return (surface_cls, best[0],
"%.0f%% of it hangs off %s bones" % (best[1] * 100.0, best[0]))
if cosmetic_share >= 0.5:
return ACCESSORY, "accessory", (
"%.0f%% of it is on bones no clip drives" % (cosmetic_share * 100.0))
return BODY, "skin", "no name or weight evidence — treated as body"
# ── Measuring one Blender scene ──────────────────────────────────────────────
def _dominant_bone_per_vertex(mesh_obj):
"""vertex index -> the vertex group holding its largest weight."""
gname = {g.index: g.name for g in mesh_obj.vertex_groups}
out = {}
for v in mesh_obj.data.vertices:
best = None
for g in v.groups:
if best is None or g.weight > best.weight:
best = g
if best is not None and best.weight > 0.25:
out[v.index] = gname.get(best.group, "")
return out
def _vertices_per_slot(mesh_obj):
"""material slot index -> the set of vertices drawn with it.
A Blender object with four material slots exports as one glTF mesh with
four primitives, which Godot loads as one MeshInstance3D with four
surfaces — Miku is exactly this, one object carrying body, body_parts, hair
and face. Partitioning by polygon is the only way to tell those apart;
reading the object's weights as a whole says all four are the same thing.
"""
per = {}
for poly in mesh_obj.data.polygons:
per.setdefault(poly.material_index, set()).update(poly.vertices)
return per
def describe_surfaces(meshes, chain_class, driven, role_bones):
"""The surface table for one character.
`chain_class` bone name -> spring class ("hair", "skirt", ...)
`driven` bones the animation drives — the body proper
`role_bones` resolved anatomy bone names, also body
Keyed on the MATERIAL name rather than the mesh node name. Node names do not
survive as anything meaningful — every character in the game arrives with
its meshes called `Object_7` through `Object_32` — while material names come
through the glTF round trip intact and are what the artist actually named.
The mesh name and surface index are recorded alongside as hints, so a model
that reuses one material for two different things can still be told apart.
"""
body_bones = set(driven) | set(role_bones)
out = []
for mesh_obj in meshes:
slots = _vertices_per_slot(mesh_obj)
dominant = _dominant_bone_per_vertex(mesh_obj)
mesh_name = datablock_name(mesh_obj.name)
# Surface index, not slot index. The exporter emits one primitive per
# material slot that any polygon actually uses, in slot order, so an
# empty slot in the middle shifts every surface after it down by one.
for surface_index, slot_index in enumerate(sorted(slots)):
verts = slots[slot_index]
mat = (mesh_obj.material_slots[slot_index].material
if slot_index < len(mesh_obj.material_slots) else None)
mat_name = mat.name if mat else ""
# Weight evidence, as a share of the vertices that belong clearly
# to any one bone — not of all vertices, so a surface that is half
# unweighted is not written off as having no evidence.
counted = 0
shares = {}
cosmetic = 0
for vi in verts:
bone = dominant.get(vi)
if not bone:
continue
counted += 1
cls = chain_class.get(bone)
if cls:
shares[cls] = shares.get(cls, 0) + 1
elif bone not in body_bones and rig_map.is_cosmetic(bone):
cosmetic += 1
denom = float(max(1, counted))
shares = {k: v / denom for k, v in shares.items()}
textured, albedo, front_culled = _material_look(mat)
cls, detail, why = classify(
mat_name, textured, albedo, front_culled, shares,
cosmetic / denom)
out.append({
"mesh": mesh_name,
"surface": surface_index,
"material": datablock_name(mat_name),
"class": cls,
"detail": detail,
"why": why,
"verts": len(verts),
"textured": textured,
# Top chain classes pulling this surface, for the rig lab to
# show when someone disagrees with the answer above.
"chain_share": {k: round(v, 3) for k, v in
sorted(shares.items(), key=lambda kv: -kv[1])},
})
return out
def _material_look(mat):
"""(textured, (r, g, b), front_culled) for a Blender material.
Both routes to the same two facts: a glTF imported by Blender's own
importer keeps a Principled BSDF, and one written by gltf_fix may be a
plain emission or have no node tree at all.
`front_culled` is always False here, and honestly so: glTF has no way to say
"draw only the backfaces" — it has a single `doubleSided` flag — so an
inverted-hull outline cannot survive the round trip as a cull mode, and
Blender has no property that would carry it. Every character in the game
imports as CULL_BACK or CULL_DISABLED, never CULL_FRONT. The runtime keeps
the cull-mode test as a backstop for a model that does arrive that way, so
nothing is lost by not being able to see it from here.
"""
if mat is None:
return False, (1.0, 1.0, 1.0), False
front_culled = False
albedo = tuple(mat.diffuse_color[:3])
textured = False
if mat.use_nodes and mat.node_tree:
for node in mat.node_tree.nodes:
if node.type == "TEX_IMAGE" and node.image is not None:
textured = True
if node.type == "BSDF_PRINCIPLED":
base = node.inputs.get("Base Color")
if base is not None and not base.is_linked:
albedo = tuple(base.default_value[:3])
elif node.type == "EMISSION":
col = node.inputs.get("Color")
if col is not None and not col.is_linked:
albedo = tuple(col.default_value[:3])
return textured, albedo, front_culled
# ── Backfilling an already-exported character ────────────────────────────────
def _chain_class_from_sidecar(sidecar):
"""bone -> spring class, taken from the chains the pipeline already found.
Backfilling deliberately reuses the sidecar's own answer rather than
re-deriving it from the bone names: if the two ever disagreed, the surface
table would describe a rig that the solver is not simulating.
"""
out = {}
for chain in sidecar.get("chains", []):
cls = chain.get("class", "cloth")
for bone in chain.get("bones", []):
out[bone] = cls
return out
def _role_bone_names(sidecar):
names = []
for value in sidecar.get("roles", {}).values():
if isinstance(value, list):
names.extend(value)
elif value:
names.append(value)
for bones in sidecar.get("fingers", {}).values():
names.extend(bones)
return names
def _is_character_mesh(obj):
"""Is this mesh part of the character, or is it riding along in the file?
Every skin shipped so far carries a stray 42-vertex `Icosphere`: no parent,
no vertex groups, no material slot. It comes in from the animation library,
and `add_animations.py` and `rig_and_animate.py` both already skip it — by
checking `o.name != 'Icosphere'`, which is precisely the kind of hardcoded
spelling this project has a rule against.
Asked by construction instead: a surface of the character is skinned to the
character. Anything with no vertex groups is not being deformed by the
skeleton, so whatever it is, it is not a body, a garment, hair or a worn
accessory — and it has no business in a table that says which of those a
surface is.
"""
return len(obj.vertex_groups) > 0
def backfill(glb_path):
"""Load a shipped skin, classify it, merge the table into its sidecar."""
import bpy
sidecar_path = os.path.splitext(glb_path)[0] + ".rig.json"
if not os.path.exists(sidecar_path):
raise SystemExit("no sidecar beside %s — run the pipeline first"
% glb_path)
with open(sidecar_path, "r", encoding="utf-8") as f:
sidecar = json.load(f)
bpy.ops.wm.read_factory_settings(use_empty=True)
bpy.ops.import_scene.gltf(filepath=glb_path)
meshes = [o for o in bpy.context.scene.objects
if o.type == "MESH" and _is_character_mesh(o)]
if not meshes:
raise SystemExit("no skinned meshes in %s" % glb_path)
surfaces = describe_surfaces(
meshes,
_chain_class_from_sidecar(sidecar),
sidecar.get("driven_bones", []),
_role_bone_names(sidecar))
sidecar["surfaces"] = surfaces
with open(sidecar_path, "w", encoding="utf-8") as f:
json.dump(sidecar, f, indent=1)
tally = {}
for s in surfaces:
tally[s["class"]] = tally.get(s["class"], 0) + 1
print("%s: %d surfaces — %s" % (
os.path.basename(glb_path), len(surfaces),
", ".join("%s x%d" % kv for kv in sorted(tally.items()))))
for s in surfaces:
print(" %-14s s%d %-30s %-10s %-14s %s" % (
s["mesh"][:14], s["surface"], s["material"][:30], s["class"],
s["detail"], s["why"]))
if __name__ == "__main__":
argv = sys.argv
argv = argv[argv.index("--") + 1:] if "--" in argv else []
if not argv:
print(__doc__)
sys.exit(1)
for path in argv:
backfill(path)