Loading...
Loading...
OmegaEngine · Calibration
OmegaEngine uses Bayesian calibration to ensure that a 0.7 risk score really does correspond to ~70% probability of an adverse outcome. As you grade decisions, the system updates its priors and adjusts thresholds — automatically.
Calibration is the process of aligning predicted probabilities with observed frequencies. OmegaEngine continuously recalibrates using operator feedback:
Measures overall prediction accuracy. Lower is better. Target: < 0.15 for production policies.
Gap between predicted probability and observed frequency. Target: < 0.05.
Visual plot of predicted vs. actual outcomes across probability bins.
How concentrated predictions are. Higher sharpness = more decisive risk scores.
These thresholds determine how risk scores map to policy actions. Adjust per-policy or let calibration optimize them:
| Risk Score | Action | Description |
|---|---|---|
| 0.00 – 0.39 | ALLOW | Low risk — automatically approved |
| 0.40 – 0.74 | REVIEW | Medium risk — routed to human review |
| 0.75 – 1.00 | BLOCK | High risk — automatically blocked |
// Get current calibration state for a policy
const state = await omega.calibration.get({
policyId: "financial-advice-v2",
});
// state.brierScore → 0.12
// state.ece → 0.04
// state.sampleCount → 2847
// state.lastUpdated → "2026-02-20T18:30:00Z"
// Trigger manual recalibration
await omega.calibration.recalibrate({
policyId: "financial-advice-v2",
windowDays: 30,
minSamples: 100,
});
// Set custom thresholds (overrides auto-calibration)
await omega.calibration.setThresholds({
policyId: "financial-advice-v2",
escalateAbove: 0.65,
blockAbove: 0.85,
});