Raiff1982 commited on
Commit
e3a4780
·
verified ·
1 Parent(s): 8f60f0e

Update model card: benchmark results, v2 adapters, public access

Browse files
Files changed (1) hide show
  1. README.md +9 -2
README.md CHANGED
@@ -39,8 +39,15 @@ Llama 3.1 8B Instruct with the **Codette Orchestrator LoRA** permanently merged
39
  - Turing naturalness: 0.245 → 0.820 (+235%) — depth–naturalness tradeoff resolved
40
  - Coherence: 0.477 → 0.700
41
 
42
- **GPQA (graduate-level science, 0-shot):** ~32% on 198-question diamond set
43
- *(vs. random 25%, GPT-4 0-shot 39%, human expert 65%)*
 
 
 
 
 
 
 
44
 
45
  ---
46
 
 
39
  - Turing naturalness: 0.245 → 0.820 (+235%) — depth–naturalness tradeoff resolved
40
  - Coherence: 0.477 → 0.700
41
 
42
+ **GPQA (graduate-level science, 0-shot, 198-question diamond set):**
43
+
44
+ | Run | Accuracy | Environment |
45
+ |-----|----------|-------------|
46
+ | Base model + adapters (Kaggle cloud, June 17 2026) | **27.8%** (55/198) | Direct transformers+PEFT, no orchestration |
47
+ | Full Codette system (local server, June 6 2026) | **30.8%** (61/198) | Multi-agent debate + coherence tracking + cocoon memory |
48
+
49
+ Baselines: random 25%, GPT-4 0-shot 39%, human expert 65%.
50
+ The ~3pp gap between runs quantifies the system layer's contribution on GPQA specifically.
51
 
52
  ---
53