← WorkMLSToolbox — CodeAssessment
From a score to thefile that needs fixing.
I redesigned CodeAssessment and built its new assessment flow inside MLSToolbox, so ML engineers can review a pipeline and act on prioritized, file-level issues instead of a single cohesion score.
- Role
- Product design + software engineering
- Timeline
- 12 weeksSep — Dec 2025
- Team
- Principal researcher,an engineer and me
- Shipped
- Assessment flow in MLSToolbox · 12 analyzers
10:524:18
Median time to first actionable issue
2/65/6
Task success
3/65/6
Correct interpretation without help
Two moderated rounds · six participants each · directional findings, not production KPIs
01 — Context
A research toolbox, and the moment it had to make sense to engineers.
MLSToolbox is GESSI’s toolbox for building and assessing ML systems at UPC Barcelona. CodeAssessment runs 12 analyzers over a Python ML repository and reports on its structure and quality.
My part
The assessment experience end to end, and the ML cohesion metrics the principal researcher defined — implemented in Python, Flask and Angular. I didn’t create MLSToolbox or CodeAssessment from zero.
Visual 03 — Engineering — architecture
Pendiente
02 — The problem
“¿Qué archivo está
causando esto?”
“Which file is causing this?” — ML engineer, repository walkthrough
The score could not point to a file.
In one walkthrough, an engineer spent 11 minutes locating mixed responsibilities after seeing a cohesion score of 5.1. One session, not an average — but it matched what 7 interviews and 5 repository walkthroughs kept showing: the useful unit was a localized issue, not another metric.
Technical corpus: 12 Python ML repositories. The research definitions came from the principal researcher; I operationalized them.
03 — The flow
Six steps, from repository to export.
Connect a repository, confirm the pipeline, run the analyzers, start with one finding, inspect its evidence, export what matters.
Visual 04 — Flow — repository to export
Pendiente
04 — Decisions
Four decisions that changed the experience.
The interface had to make its assumptions visible and leave engineers with somewhere concrete to look.
Visual 06 — Decision — issues before scores
Pendiente
Decision 01
Open an issue, not a score.
Findings attach to files; the score stays as context. A folder heatmap we tried still left people searching through layers.
Visual 05 — Decision — confirm the pipeline
Pendiente
Decision 02
Let people correct the assumption.
Engineers confirm, move or exclude files per stage before dependent analysis. It added 20–40 seconds in observed sessions — I kept it.
Visual 07 — Decision — evidence for every action
Pendiente
Decision 03
Make every recommendation inspectable.
Each finding shows the functions and mixed responsibilities behind it, plus a clear next step. Metric details sit one level down.
Visual 08 — Decision — partial results stay valid
Pendiente
Decision 04
One failure shouldn’t erase the run.
If an analyzer fails, completed findings stay visible; the missing check is named and can be retried on its own.
A decision I dropped
One 0–100 health score.
It hid the difference between maintainability and separation of responsibilities — the exact thing engineers needed to see.
Visual 02 — System — one contract for 12 analyzers
Pendiente
05 — System
One contract for 12 analyzers.
Every analyzer reports through the same contract, so a failing check or a new analyzer doesn’t break the workspace.
~1 day2 h 20 min
to add an analyzer with the shared contract
Two integrations compared — not a productivity benchmark.
06 — Validation
A shorter path to an actionable issue.
Two moderated rounds, six participants each — four new and two returning in round 2. Several things changed between rounds, so read these as directional.
Measure
Round 1
Round 2
Median time to first actionable issue
10 min 52 s
4 min 18 s
Task success
2 / 6
5 / 6
Correct interpretation without help
3 / 6
5 / 6
Confidence
2.7 / 5
4.2 / 5
Median time to first actionable issue
Round 1
10 min 52 s
Round 2
4 min 18 s
Task success
Round 1
2 / 6
Round 2
5 / 6
Correct interpretation without help
Round 1
3 / 6
Round 2
5 / 6
Confidence
Round 1
2.7 / 5
Round 2
4.2 / 5
Pipeline classification
F1 0.84 → 0.92
Macro F1 before overrides, then after assisted correction.
The second number includes human review.
Diagnostic precision
49 / 60
Sampled issues confirmed in manual review (81.7%).
Sample precision, not overall accuracy.
Recommendations
26 / 30
Rated actionable by three senior reviewers · usefulness 4.3 / 5.
Reviewers rated 30 recommendations.
07 — Behind the scenes
Iterations, scope and the details.
08 — Reflection
Control was the harder decision.
I kept the pipeline review step because people could inspect and correct the assumption. Static analysis supports judgment; it doesn’t establish runtime behavior, data quality or model performance — and the interface says so.
Next time
- 01
Recruit external ML engineers earlier.
- 02
Record step-level behavior from the first version.
- 03
Define ground truth before each analyzer.
Next case · 02
Ominio
A gamified, AI-driven soft-skills app for students and early-career professionals.