Upload folder using huggingface_hub

README.md

@@ -0,0 +1,65 @@
+---
+license: mit
+tags:
+  - motion-capture
+  - pose-estimation
+  - inverse-kinematics
+  - animation
+  - arbitrary-skeleton
+library_name: pytorch
+---
+
+# pose2rot — Joint Positions → 6D Rotations for Arbitrary Skeletons
+
+Pretrained checkpoints for the **pose2rot** model (`Pose2RotMemoryRestModel`): given a sequence of
+3D joint **positions**, predict per-joint **6D rotations** (forward kinematics then recovers the full
+skeletal animation). One model handles **arbitrary skeletons** across 72 animal species — quadrupeds,
+bipeds, birds, reptiles, dinosaurs, arthropods, limbless snakes — via T5 joint-name embeddings,
+skeleton graph attention, and rest-pose FiLM conditioning.
+
+**Code, training recipe, eval & QA scripts:** https://github.com/CHDTevior/pose2rot
+
+This is a derivative work of [MocapAnything](https://github.com/phongdaot/MocapAnything)
+(MIT, © 2026 Dao Thien Phong; arXiv:2604.28130 MoCapAnything V2). ~29.7M params.
+
+## Checkpoints
+
+| file | training data | use case |
+|---|---|---|
+| `pose2rot_v9_alldata_epoch60.pt` | all 72 species | **best for demos / inference** (the species is seen) |
+| `pose2rot_v10_heldout_epoch60.pt` | seen/rare/unseen held-out split (test motions excluded) | the **decisive paper model** for honest cross-topology eval |
+| `pose2rot_v8b_best_epoch40.pt` | all species (earlier converged best) | reference |
+
+Each `.pt` holds `{model_state, optimizer_state, epoch}`. Configs: `config_v9_alldata.yaml`,
+`config_v10_split_heldout.yaml` (model section instantiates `Pose2RotMemoryRestModel`).
+
+## Results (geodesic angle error, degrees; MoCapAnything V2 reports 6.54° unseen / V1 ~17°)
+
+| model | seen | rare | unseen | overall |
+|---|---|---|---|---|
+| v9 all-data (oracle) | 7.2° | 5.9° | 6.4° | **6.53°** ≈ MoCapAnything 6.54° |
+| v10 true held-out | 9.8° | 12.7° | 40.9° | 28.0° |
+
+When the species is **seen**, the model matches SOTA (6.5°). On a **true held-out** test, cross-topology
+generalization is a ceiling: unseen species with close training relatives generalize partially (Goat 17°,
+Coyote 19°), topologically distinctive ones do not (Pigeon ~67°, Spider ~73°).
+
+## Usage
+
+```python
+import torch
+from utils.config_utils import load_yaml_config, instantiate_from_config  # from the GitHub repo
+
+cfg = load_yaml_config("config_v9_alldata.yaml")
+model = instantiate_from_config(cfg["model"]).eval().cuda()
+model.load_state_dict(torch.load("pose2rot_v9_alldata_epoch60.pt", map_location="cpu")["model_state"])
+# batch dict: position[B,T,J,3] + rest pose + T5 joint embeddings + skeleton graph + reference (see GitHub data/loader_v2.py)
+pred_rot6d = model(batch)["pred_rot6d"]  # [B,T,J,6]
+```
+
+See https://github.com/CHDTevior/pose2rot for the full data pipeline, training, and evaluation.
+
+## License & Citation
+
+MIT. Built on [MocapAnything](https://github.com/phongdaot/MocapAnything) (Dao Thien Phong, MIT).
+Please also cite MoCapAnything (arXiv:2604.28130).

config_v10_split_heldout.yaml

@@ -0,0 +1,78 @@
+### train_pose2rot_v7_ddp_b4.yaml — v6 recipe (memabl+tvar) at batch4 DDP-2gpu for 3.6x throughput ###
+name: Pose2Rot training
+
+runtime:
+  device: cuda
+  seed: 42
+  debug: false
+
+output:
+  checkpoint_root: ./checkpoints/pose2rot
+
+
+experiment:
+  exp: exp_pose2rot_v10_split_heldout
+
+model:
+  target: models.v2.pose2rot.model.Pose2RotMemoryRestModel
+  params:
+    q_dim: 256
+    rest_layers: 4
+    pose_layers: 4
+    memory_layers: 4
+    decoder_layers: 10
+    num_heads: 8
+    joint_embed_dim: 768
+    temporal_window: 2
+    temporal_dropout: 0.1
+    decoder_cond_mode: add # add | concat
+    pose_rest_film: true
+    memory_rest_film: true
+    decoder_rest_film: true
+    pose_use_graph: true
+    use_grad_checkpoint: false
+    decoder_use_cross_layers: 0   # MEMORY ABLATION: no decoder cross-attn into memory bank (kill species-constant leakage)
+
+train:
+  batch_size: 4   # DDP global batch = 4/gpu x 2 gpu = 8
+  epochs: 60
+  grad_accum_steps: 1
+  lr: 0.0002   # DDP 2-gpu global batch 8 at the proven-safe lr2e-4 (test if batch-doubling breaks anti-collapse)
+  warmup_steps: 500   # linear LR warmup 0->8e-4 (codex: tame Adam startup at large scaled LR)
+  max_ckpt: 100
+  num_workers_train: 6   # 6/proc x 2 proc = 12 of 16 cores
+  test_every: 1
+  pretrain_ckpt: null
+
+  loss:
+    rot_loss_type: smooth_l1
+    vel_loss_type: smooth_l1
+    acc_loss_type: smooth_l1
+
+  weight:
+    root_wt: 0.1
+    fk_wt: 10.0   # FK ramp END=10(用户按MoCapAnything; 比v8b的30温和, fk梯度~0.33<<grad_clip1.0 不会再亚稳发散)
+    fk_wt_start: 0.0   # 从0起(纯抗塌缩早期, 让tvar正常破塌缩)
+    fk_ramp_start_epoch: 5
+    fk_ramp_end_epoch: 15    # 线性 0->10 over epoch5-15, 之后恒10
+    vel_wt: 1.0
+    acc_wt: 1.0   # 用户要求加 acc(2阶时序平滑); 随机初始化量级~0.0095
+    rot_wt: 1.0
+    tvar_wt: 2.0   # demeaned-temporal supervision (force motion-tracking, anti-collapse)
+
+  vis_every: 5
+  weight_decay: 0.0
+
+eval:
+  batch_size: 1
+  num_workers: 2
+
+data:
+  seq_len: 48
+  bvh_dir: datasets/zoo1030/bvh
+  cache_scale: true
+  limit_species_debug: []
+  mmap: true
+  split_json: datasets/zoo1030/test_split_seen_rare_unseen.json
+  train_memory_pkl_path: datasets/zoo1030/cache/species_fps_memory_yAll/fps_select_by_rot_32.pkl
+  test_memory_pkl_path: datasets/zoo1030/cache/species_fps_memory_yAll/fps_select_by_rot_32.pkl

config_v9_alldata.yaml

@@ -0,0 +1,78 @@
+### train_pose2rot_v7_ddp_b4.yaml — v6 recipe (memabl+tvar) at batch4 DDP-2gpu for 3.6x throughput ###
+name: Pose2Rot training
+
+runtime:
+  device: cuda
+  seed: 42
+  debug: false
+
+output:
+  checkpoint_root: ./checkpoints/pose2rot
+
+
+experiment:
+  exp: exp_pose2rot_v9_fk10ramp_60ep
+
+model:
+  target: models.v2.pose2rot.model.Pose2RotMemoryRestModel
+  params:
+    q_dim: 256
+    rest_layers: 4
+    pose_layers: 4
+    memory_layers: 4
+    decoder_layers: 10
+    num_heads: 8
+    joint_embed_dim: 768
+    temporal_window: 2
+    temporal_dropout: 0.1
+    decoder_cond_mode: add # add | concat
+    pose_rest_film: true
+    memory_rest_film: true
+    decoder_rest_film: true
+    pose_use_graph: true
+    use_grad_checkpoint: false
+    decoder_use_cross_layers: 0   # MEMORY ABLATION: no decoder cross-attn into memory bank (kill species-constant leakage)
+
+train:
+  batch_size: 4   # DDP global batch = 4/gpu x 2 gpu = 8
+  epochs: 60
+  grad_accum_steps: 1
+  lr: 0.0002   # DDP 2-gpu global batch 8 at the proven-safe lr2e-4 (test if batch-doubling breaks anti-collapse)
+  warmup_steps: 500   # linear LR warmup 0->8e-4 (codex: tame Adam startup at large scaled LR)
+  max_ckpt: 100
+  num_workers_train: 6   # 6/proc x 2 proc = 12 of 16 cores
+  test_every: 1
+  pretrain_ckpt: null
+
+  loss:
+    rot_loss_type: smooth_l1
+    vel_loss_type: smooth_l1
+    acc_loss_type: smooth_l1
+
+  weight:
+    root_wt: 0.1
+    fk_wt: 10.0   # FK ramp END=10(用户按MoCapAnything; 比v8b的30温和, fk梯度~0.33<<grad_clip1.0 不会再亚稳发散)
+    fk_wt_start: 0.0   # 从0起(纯抗塌缩早期, 让tvar正常破塌缩)
+    fk_ramp_start_epoch: 5
+    fk_ramp_end_epoch: 15    # 线性 0->10 over epoch5-15, 之后恒10
+    vel_wt: 1.0
+    acc_wt: 1.0   # 用户要求加 acc(2阶时序平滑); 随机初始化量级~0.0095
+    rot_wt: 1.0
+    tvar_wt: 2.0   # demeaned-temporal supervision (force motion-tracking, anti-collapse)
+
+  vis_every: 5
+  weight_decay: 0.0
+
+eval:
+  batch_size: 1
+  num_workers: 2
+
+data:
+  seq_len: 48
+  bvh_dir: datasets/zoo1030/bvh
+  cache_scale: true
+  limit_species_debug: []
+  mmap: true
+  split_json: datasets/zoo1030/selected_test_split1010.json
+  train_memory_pkl_path: datasets/zoo1030/cache/species_fps_memory_yAll/fps_select_by_rot_32.pkl
+  test_memory_pkl_path: datasets/zoo1030/cache/species_fps_memory_yAll/fps_select_by_rot_32.pkl

pose2rot_v10_heldout_epoch60.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:7738568a8087d4f2d9f9fb84ea6ac1dac51b7eca2ab2046df07399a305617f80
+size 325972891

pose2rot_v8b_best_epoch40.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:36398c06b7cfade441ca74527b994388f7b1e6da7ee0b81a3c1b8db1a4c76691
+size 325972891

pose2rot_v9_alldata_epoch60.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:3a8a075999f7ffbc02bb085f161157f9d9d8369be1799a5779ca6b89131e5d9f
+size 325972891