Update model card: benchmark results, v2 adapters, public access
Browse files
README.md
CHANGED
|
@@ -39,8 +39,15 @@ Llama 3.1 8B Instruct with the **Codette Orchestrator LoRA** permanently merged
|
|
| 39 |
- Turing naturalness: 0.245 → 0.820 (+235%) — depth–naturalness tradeoff resolved
|
| 40 |
- Coherence: 0.477 → 0.700
|
| 41 |
|
| 42 |
-
**GPQA (graduate-level science, 0-shot
|
| 43 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 44 |
|
| 45 |
---
|
| 46 |
|
|
|
|
| 39 |
- Turing naturalness: 0.245 → 0.820 (+235%) — depth–naturalness tradeoff resolved
|
| 40 |
- Coherence: 0.477 → 0.700
|
| 41 |
|
| 42 |
+
**GPQA (graduate-level science, 0-shot, 198-question diamond set):**
|
| 43 |
+
|
| 44 |
+
| Run | Accuracy | Environment |
|
| 45 |
+
|-----|----------|-------------|
|
| 46 |
+
| Base model + adapters (Kaggle cloud, June 17 2026) | **27.8%** (55/198) | Direct transformers+PEFT, no orchestration |
|
| 47 |
+
| Full Codette system (local server, June 6 2026) | **30.8%** (61/198) | Multi-agent debate + coherence tracking + cocoon memory |
|
| 48 |
+
|
| 49 |
+
Baselines: random 25%, GPT-4 0-shot 39%, human expert 65%.
|
| 50 |
+
The ~3pp gap between runs quantifies the system layer's contribution on GPQA specifically.
|
| 51 |
|
| 52 |
---
|
| 53 |
|