Argus-3D: discovered class-agnostic 3D detection head on EUPE-ViT-B

README.md

@@ -23,13 +23,13 @@ The detector pairs a non-linear (or linear) per-patch foreground head with featu
 
 ```
 Per-frame:
-  Image (768x768)
+  Image (1024x1024)
     -> EUPE-ViT-B (frozen, reused from phanerozoic/argus)
-    -> patch tokens (2304, 768) on a 48x48 grid
+    -> patch tokens (4096, 768) on a 64x64 grid
     -> instance head: 2-layer MLP (default) or linear ridge -> per-patch foreground score
        depth head:    ridge over 768 dims                   -> per-patch metric depth (m)
        k-means modes: 8 cluster centers (20-scene)          -> per-patch object-type assignment
-    -> threshold instance score, upsample mask to 768x768, connected components
+    -> threshold instance score, upsample mask to 1024x1024, connected components
     -> for each component:
          unproject to 3D using depth + K
          DBSCAN-split for instance separation
@@ -48,8 +48,8 @@ Multi-view (per scene):
 
 | Component | Parameters | Discovery / training |
 |---|---|---|
-| EUPE-ViT-B backbone (frozen, reused) | not part of this head | reused from phanerozoic/argus |
-| Instance head — MLP (default) | ~200 K | 2-layer MLP (768 → 256 → 1, ReLU + dropout 0.5), 30 epochs AdamW BCE-with-logits on 4-scene mixed split |
+| EUPE-ViT-B backbone (frozen, reused) | not part of this head | reused from phanerozoic/argus, run at 1024x1024 input (64x64 patch grid) |
+| Instance head — MLP (default) | ~200 K | 2-layer MLP (768 → 256 → 1, ReLU + dropout 0.5), 30 epochs AdamW BCE-with-logits on 20-scene patch split at 1024 input |
 | Instance head — linear ridge (fallback) | 769 floats + threshold | random K=20 subset search + hard-neg mining, AUC selection |
 | Depth head (ridge over 768 dims) | 769 floats | random K=20 subset search, RMSE selection |
 | K-means cluster centers | 8 × 768 floats | MiniBatchKMeans on foreground patches across 20 CA-1M val scenes |
@@ -94,17 +94,17 @@ CA-1M val sequence `ca1m-val-45662921`. Class-agnostic per-scene 3D IoU after mu
 
 ### Headline result
 
-3D-IoU multi-view fusion + 4-scene MLP head + 20-scene size priors:
+1024-input MLP head + 20-scene size priors + 3D-IoU multi-view fusion + iterative refinement:
 
-| Metric | Linear ridge (v0) + position fusion | MLP + position fusion | MLP + IoU fusion + refine (default) |
+| Metric | Linear ridge (v0) + 768 + position fusion | MLP + 768 + IoU fusion | MLP + 1024 + IoU fusion (default) |
 |---|---|---|---|
-| Mean 3D IoU | 0.063 | 0.076 | **0.177** |
-| Median 3D IoU | — | 0.019 | **0.146** |
-| Fraction > 0.1 IoU | 27.8 % | 25.6 % | **60.9 %** |
-| Fraction > 0.25 IoU | 6.9 % | 11.5 % | **34.8 %** |
-| Fraction > 0.5 IoU | 0.0 % | 1.3 % | **4.3 %** |
-| Recall (matched / GT) | 19.8 % | 24.4 % | 18.0 % |
-| Fused boxes per scene | 72 | 78 | 46 |
+| Mean 3D IoU | 0.063 | 0.177 | **0.216** |
+| Median 3D IoU | — | 0.146 | **0.175** |
+| Fraction > 0.1 IoU | 27.8 % | 60.9 % | **68.2 %** |
+| Fraction > 0.25 IoU | 6.9 % | 34.8 % | **36.4 %** |
+| Fraction > 0.5 IoU | 0.0 % | 4.3 % | **9.1 %** |
+| Recall (matched / GT) | 19.8 % | 18.0 % | 17.5 % |
+| Fused boxes per scene | 72 | 46 | 44 |
 
 Default config: 3D-IoU fusion at threshold 0.4, min 4 observations per cluster, weight floor at the 80th percentile of nonzero cluster weights, plus iterative refinement (drop observations with IoU < 0.4 to the cluster consensus, re-fuse, up to 3 iterations). Looser fusion thresholds trade per-box quality for recall:
 

config.json

@@ -1,6 +1,6 @@
 {
-  "input_res": 768,
-  "patch_grid": 48,
+  "input_res": 1024,
+  "patch_grid": 64,
   "patch_size": 16,
   "feature_dim": 768,
   "prior_weight": 80.0,
@@ -9,5 +9,10 @@
     0.1,
     10.0
   ],
-  "fg_threshold": 0.31326109170913696
+  "fg_threshold": 0.569353461265564,
+  "fusion_iou_thresh": 0.4,
+  "fusion_min_obs": 4,
+  "fusion_weight_pct": 80,
+  "fusion_refine_iou": 0.4,
+  "fusion_refine_iters": 3
 }

instance_head_mlp.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:25da02eb73105206bc26d313481fad4b6fe9686615b0696dcbc020e0fff10c10
+oid sha256:f8531239d67617aebd377c0cbdbb2dfd6ed59d690bfcb25cbf1ae11c2996a2c5
 size 788780

instance_head_mlp_meta.json

@@ -1,10 +1,35 @@
 {
-  "method": "2-layer MLP (in-distribution)",
+  "method": "2-layer MLP",
   "in_dim": 768,
   "hidden": 256,
   "dropout": 0.5,
   "epochs": 30,
-  "auc": 0.9795503411078317,
-  "best_f1": 0.815474970975893,
-  "best_threshold": 1.1412171125411987
+  "train_scenes": [
+    "42446540",
+    "42897501",
+    "42897521",
+    "42897538",
+    "42897545",
+    "42897552",
+    "42897561",
+    "42898486",
+    "45260903",
+    "45261133",
+    "45261143",
+    "45261179",
+    "45261587",
+    "45261615",
+    "45662921",
+    "45663113",
+    "47115543",
+    "47204559",
+    "47331068",
+    "47331262"
+  ],
+  "val_scenes": [
+    "45662921"
+  ],
+  "auc": 0.9806332802845262,
+  "best_f1": 0.8387126179329744,
+  "best_threshold": 0.569353461265564
 }

size_priors.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:ba90168daf451ea21bf5009df47bd5353bb455b160f6c2b718ba33ce1f97fd61
+oid sha256:047ccb36af9d535b252ad66d0370cd37bc24ec7f0a78aadaab4f9cd325b94b24
 size 24832