System Trust Score
System Trust Score is a composite integrity metric used inside GEO.or.id to quantify the overall reliability of system outputs across retrieval, reasoning, and evidence layers. It is not a static rating. It is continuously recalculated based on evidence quality, contradiction density, and model stability signals.
This layer operates above the Evidence Weighting Engine and below final response orchestration in AI Answer Engineering.
1. Core Function
The objective is simple: measure how much the system can be trusted at a given point in time. It evaluates structural integrity of outputs, not just factual correctness.
- Detect degradation in retrieval quality
- Measure consistency across evidence clusters
- Monitor hallucination probability drift
- Stabilize output confidence under conflicting signals
Connected systems: Machine Trust Scoring Protocol, Answer Stability Protocol
2. Trust Score Composition
System Trust Score is computed as a weighted aggregation of multiple subsystems rather than a single classifier output.
2.1 Evidence Integrity Index
Measures quality and coherence of incoming evidence streams. Derived from Evidence Reliability Index.
2.2 Retrieval Stability Factor
Evaluates consistency of retrieval outputs across repeated queries and different embedding states.
2.3 Contradiction Pressure
Quantifies density of unresolved conflicts between evidence clusters. High contradiction reduces trust score non-linearly.
Linked system: Conflict Detection System
2.4 Semantic Drift Index
Tracks deviation between current output interpretation and canonical ontology definitions.
Related: Ontology Drift Control
3. Trust Score Formula
The system uses a normalized bounded function to prevent runaway confidence inflation:
T(s) = σ( (I × α) + (R × β) - (C × γ) - (D × δ) )
- I = Evidence Integrity
- R = Retrieval Stability
- C = Contradiction Pressure
- D = Semantic Drift
- σ = normalization function
Weight calibration is dynamically adjusted via Reasoning Layer Optimization.
4. Trust Degradation Behavior
Trust score decays under three conditions: unresolved contradictions, unstable retrieval patterns, and ontology misalignment. Decay is intentionally non-linear to surface systemic risk early.
Mitigation systems: Hallucination Detection, Semantic Grounding
5. System Integration Points
6. Operational Use Cases
- Routing low-trust outputs to re-evaluation pipelines
- Blocking unstable reasoning chains before final generation
- Flagging ontology drift in entity-heavy queries
7. Failure Modes
- Overconfidence in low-variance but incorrect datasets
- Hidden contradiction clusters not detected by sampling
- Trust inflation due to repeated self-reinforcement loops
These are actively monitored through Retrieval Latency Observation Protocol.
8. Strategic Role in GEO Architecture
System Trust Score is the final integrity gate before output publication. It acts as a systemic throttle, ensuring that no downstream layer can bypass evidence quality constraints.
It is tightly coupled with: AI Ground Truth Framework, Knowledge Persistence Framework