Confidence Scoring
How to evaluate the reliability of accessibility measurements.
Overview
Every surface measurement receives a confidence score from 0–100%, indicating its reliability for OSM contributions. The current tooling calculates this score for single rides; rides over the same segment can additionally be combined into a multi-ride consensus.
Single Ride Scoring
Confidence for a single ride multiplies three factors, each capturing a different threat to data reliability. The product expresses how trustworthy one ride is for mapping surface accessibility – a weakness in any single factor pulls the whole score down.
| Factor | Description |
|---|---|
| Classification Clarity | Scores how clearly the Z-axis standard deviation falls within the calibrated smoothness thresholds |
| Speed Appropriateness | Assigns the full speed factor within the 2–6 km/h assessment range and reduces it outside that range |
| Movement Continuity | Reduces confidence when recent samples contain stops |
Confidence Over Time
The following chart tracks confidence scores for a single ride, showing how speed, classification clarity, and movement continuity affect the final reliability before aggregation at the street (“way”) level. High-confidence periods indicate strong alignment with wheelchair-relevant conditions, while low-confidence intervals signal stops, abnormal speed, or potential sensor noise.
Multi-Ride Scoring
Multi-ride consensus can combine rides over the same segment using spatial clustering and temporal distribution, so that agreement between independent riders, rather than one strong ride, carries the weight of a proposal.
| Factor | Description |
|---|---|
| Tolerance radius | 10 meters (spatial buffer for GPS error) |
| Minimum rides | 3 rides for consensus scoring |
| Temporal spread | Data from different days/times |
Clustering alone does not make rides trustworthy. A second set of indicators grades each cluster on internal consistency, timing spread, and rider variety before any consensus value can be proposed.
| Metric | Description |
|---|---|
| Classification consensus | Proposed minimum: ≥70% of rides agree |
| Spatial coverage | At least 30% of segment length sampled |
| Ride diversity | Different users and e-scooter types |
Confidence Thresholds
Confidence thresholds determine how OSM proposals are handled. Scores above 80% are generally considered safe for contribution, while everything below that bar earns progressively more scrutiny. The confidence in that bar comes from testing against OpenStreetMap itself: where a tag was already present, the algorithm's classification agreed with it in more than 8 out of 10 cases. While end-to-end automation is the goal, until the method is validated, pilot proposals are subject to especially conservative rules.
| Confidence Level | Score Range | Recommendation |
|---|---|---|
| High | >80% | Direct OSM contribution |
| Medium | 60–80% | Manual review recommended |
| Low | 40–60% | Additional validation needed |
| Insufficient | <40% | Exclude from proposals |
Practical Application
Single rides are enough to surface local accessibility gaps. The multi-ride workflow can analyze larger datasets systematically for city-wide coverage. The thresholds are the same in both cases; what changes is how much data stands behind each proposal.
Wheelchair confidence score tiers above are guidance for review priority. The current single-ride export and apply workflow defaults to coverage ≥70% and point agreement ≥70% (apply_smoothness_batch.py). High (>80%) remains the recommended bar for direct contribution without extra review until the method is validated.
Until the method is validated, more conservative rules were applied for the pilot.