Case study · Individual research project
SLA Sentinel
Spot service risk while there is still time to act
Overview
The project at a glance
An individual machine-learning study of SLA breach risk using ticket-opening information, leakage checks, and time-based robustness analysis.
- Incidents in the modelling cohort
- 23,099
- ROC-AUC on the stratified held-out test
- 0.862
- ROC-AUC in the chronological robustness check
- 0.750
Scope & contribution
SLA Sentinel investigates whether a service desk could identify incident breach risk when a ticket first arrives. I built a reproducible Python modelling workflow using the UCI Incident Management Process Enriched Event Log. The work covers incident-level data preparation, prediction-time feature checks, model comparison, and threshold analysis. It is an offline study of early-warning decisions, not a deployed service-desk integration.
Technology stack
Data preparation
- Python
- pandas
- NumPy
Modelling
- scikit-learn
- Gradient Boosting
Analysis
- Jupyter
- Matplotlib
- Seaborn
Evaluation
- Cross-validation
- Leakage checks
- Temporal analysis
The story
Decisions behind the delivery
01Predict with what the desk actually knows
The source contains 141,712 event updates across 24,918 incidents. After excluding incidents already resolved at their first recorded event, I modelled 23,099 incidents using their opening snapshot. Resolution times, reopens, reassignment counts, and other hindsight fields were excluded from the predictors. This keeps the question focused on early warning rather than reconstructing an outcome from a completed record.
02Build the evaluation into the pipeline
I compared six initial classifier families against majority-class and priority-rule baselines, then added gradient boosting. Imputation, encoding, and scaling run within training folds, with cross-fitted target encoding where used. Workload and caller-history features helped test whether queue context adds useful information beyond priority and category. Ablation, learning curves, and regularisation checks separated feature gains from overfitting.
03A useful ranking, with a clear boundary
Tuned gradient boosting achieved 0.862 ROC-AUC and 0.818 PR-AUC on the stratified held-out test. A chronological robustness check reduced ROC-AUC to 0.750 as breach rates and workloads changed. The temporal check reused settings selected during the earlier experiments, so it is not an independent deployment validation. Reporting both results makes the gap between an offline benchmark and future operational performance visible.
04An alert has to earn its place in the queue
Lowering the decision threshold caught more breaches in the held-out evaluation, but also flagged more tickets for review. I examined that precision, recall, and review-volume trade-off rather than selecting a threshold from accuracy alone. A next-stage evaluation would train only on outcomes known before each time cutoff, use rolling validation, and agree alert capacity with service managers. The current evidence supports further triage research, not automatic escalation.
Overview
SLA Sentinel investigates whether a service desk could identify incident breach risk when a ticket first arrives. I built a reproducible Python modelling workflow using the UCI Incident Management Process Enriched Event Log. The work covers incident-level data preparation, prediction-time feature checks, model comparison, and threshold analysis. It is an offline study of early-warning decisions, not a deployed service-desk integration.
- Incidents in the modelling cohort
- 23,099
- ROC-AUC on the stratified held-out test
- 0.862
- ROC-AUC in the chronological robustness check
- 0.750
01 · Decision framing
Predict with what the desk actually knows
The source contains 141,712 event updates across 24,918 incidents. After excluding incidents already resolved at their first recorded event, I modelled 23,099 incidents using their opening snapshot. Resolution times, reopens, reassignment counts, and other hindsight fields were excluded from the predictors. This keeps the question focused on early warning rather than reconstructing an outcome from a completed record.
02 · Engineering
Build the evaluation into the pipeline
I compared six initial classifier families against majority-class and priority-rule baselines, then added gradient boosting. Imputation, encoding, and scaling run within training folds, with cross-fitted target encoding where used. Workload and caller-history features helped test whether queue context adds useful information beyond priority and category. Ablation, learning curves, and regularisation checks separated feature gains from overfitting.
03 · Results
A useful ranking, with a clear boundary
Tuned gradient boosting achieved 0.862 ROC-AUC and 0.818 PR-AUC on the stratified held-out test. A chronological robustness check reduced ROC-AUC to 0.750 as breach rates and workloads changed. The temporal check reused settings selected during the earlier experiments, so it is not an independent deployment validation. Reporting both results makes the gap between an offline benchmark and future operational performance visible.
04 · Operational thinking
An alert has to earn its place in the queue
Lowering the decision threshold caught more breaches in the held-out evaluation, but also flagged more tickets for review. I examined that precision, recall, and review-volume trade-off rather than selecting a threshold from accuracy alone. A next-stage evaluation would train only on outcomes known before each time cutoff, use rolling validation, and agree alert capacity with service managers. The current evidence supports further triage research, not automatic escalation.
Next case study
ScamSpot