Methodology
DAB v1
215-point regression gate. Runs on every Quaid release.
Scoring Breakdown
Each section is deterministic and release-blocking.
| Name | Points | What it tests |
|---|---|---|
| Infrastructure | 25 | Install, version check, DB init, ingestion |
| FTS Search | 25 | Full-text search across 10 queries |
| Semantic Search | 25 | Vector similarity, paraphrase recall, 5 queries |
| MCP Server | 25 | JSON-RPC compliance, 20 tool calls |
| Performance | 25 | FTS <100ms, semantic <500ms latency targets |
| Data Integrity | 25 | Write atomicity, no loss under load |
| Retrieval Quality | 30 | P@5 and R@5 on FiQA 500-query corpus |
| Regression | 35 | Version consistency, schema migration |
How To Run
MEMORY_CMD=quaid bash benchmarks/dab/run.shThresholds
- 215 = perfect
- 193+ = passing
- Below 193 = regression blocked
Current Score
213/215 (99%)