A multi-sensor defect detection system combining a USB webcam and thermal infrared camera, deployed as a web-based quality management platform.
Every industry runs on steel. Every car frame, ship hull, bridge member, and aircraft component begins as a sheet on a rolling line — and every one of them fails when a surface defect goes uncaught.
We built a system that catches them — multi-sensor, ISO-aligned, and running on consumer-grade hardware for under ₱31,000.
Automotive frames, ship hulls, bridge members, aircraft components — defective steel propagates downstream into systems where failure has real consequences. Surface defects compromise corrosion resistance, introduce stress concentrations, and weaken load-bearing parts.
Linear abrasions from foreign objects or interlayer friction during coiling — disrupt protective oxide layers and accelerate corrosion in service.
Localized depressions from impact pressure — introduce stress concentrations that weaken load-bearing parts under fatigue loading.
Studies have confirmed that manual visual examination is subject to subjective judgment, generally lacks precision for small or subtle defects, and cannot match the pace of modern continuous-rolling lines (Wang et al., 2021; Zheng et al., 2021).
Even well-trained inspectors miss significant fractions of real defects after sustained visual work; degradation begins within minutes of continuous inspection.
Manual classification depends on training, experience, and individual judgment — producing inconsistent results across shifts and operators.
Human perception has hard physical limits; high-speed rolling lines produce sheets faster than any operator can verify reliably.
Tang et al. (2023) categorize automated steel-surface inspection methods into four broad approaches. The most notable trend across recent literature is the rapid rise of learning-based methods, particularly from 2021 onward.
Thresholding, histogram-based filters. Fragile to lighting variation and surface noise.
Fourier and wavelet-based filtering. Sensitive to scale and orientation.
GLCM, gray-level run-length. Works in controlled conditions, struggles with overlapping defects.
CNNs, YOLO variants, transformers. Significant performance gains since 2021 — the trajectory this thesis builds on.
A working multi-sensor defect detection system on consumer-grade hardware has measurable impact at the production line, downstream in end-user industries, in the research community, and for the broader workforce and environment.
Replaces subjective manual inspection with a data-driven pipeline. Reduces inspection downtime, material waste, and misclassification costs — improving competitiveness.
Construction, transportation, and infrastructure benefit from fewer defective sheets reaching their lines — mitigating structural-failure risk that carries public-safety consequences.
Contributes a replicable multi-sensor fusion architecture, an open YOLO benchmark on a custom steel dataset, and an ISO-aligned web platform pattern for future systems.
Reduces worker exposure to hazardous inspection environments. Earlier defect catch means less downstream rework and lower material waste — a more sustainable production cycle.
Every claim in this thesis is bounded by the scope established at the outset. The system was designed and evaluated within these boundaries — not beyond them.
How the system was designed, built, and evaluated.
A design-and-build study that integrates computer vision and thermal sensing into a working prototype, then evaluates it against industry benchmarks. The work is comparative (three YOLO variants), empirical (held-out test set), and standards-grounded (ISO 9001 + ISO 10012).
USB webcam + MLX90640 thermal IR combined through a soft-voting ensemble. Each modality calibrated against per-sensor validation accuracy.
YOLOv8, YOLOv11, and YOLO26 trained under identical conditions on the same dataset. Best-performing variant selected for deployment.
Precision, Recall, F1, mAP@0.5, mAP@0.5:0.95, FDR, MDR — all measured on a strictly held-out test set.
Twelve clauses across ISO 9001:2015 and ISO 10012:2003 reviewed with explicit Compliant / Partial / Not-compliant findings.
We considered three candidate system architectures and scored each against five weighted criteria: classification accuracy (30%), robustness to sensor noise (25%), processing speed (20%), implementation complexity (15%), and dataset suitability (10%). Design 3 took the lead on the two heaviest-weighted dimensions.
Single linear pipeline — sensors feed directly into the detection pipeline and out to a verdict. Simplest to implement but has no error correction or refinement.
Hierarchical inspection — webcam first, thermal second. Early rejection saves cycles but no probabilistic resolution of borderline cases.
Two sensors feed a soft-voting ensemble. Highest scores on accuracy and noise robustness — the two criteria that together carry 55% of the weight.
Classification accuracy and robustness to sensor noise together account for 55% of the total weight — the two dimensions where missing the call costs the most in production. Design 3 scored highest on both, even at the cost of higher implementation complexity and slower per-cycle throughput. These tradeoffs are manageable: a standard PC platform handles the simultaneous sensor evaluation, and the controlled laboratory environment supports the precise sensor alignment the ensemble requires.
USB webcam runs YOLOv11. MLX90640 catches subsurface anomalies. Soft-voting ensemble fuses them. Steel IRIS records every verdict with an immutable evidence trail.
Per-class F1: Dent 0.877 / Scratch 0.902. Inference 9.3 ms per image on the mini-PC.
A subsurface dent may not show a visible boundary under overhead LED, but it shifts local heat-conduction. Per-pixel EMA baseline + 2.5σ + 1.0°C + 3-frame persistence flags real anomalies, not sensor noise.
Built on the prototype rig under controlled LED illumination. Supplemented with a remapped NEU-DET subset to expand defect-morphology coverage. Roboflow handled bounding-box annotation, preprocessing, and augmentation — horizontal/vertical flips, rotational variation, contrast adjustment, and synthetic noise.
Evaluation runs in two stages: model-level (each YOLO variant alone), then system-level (the integrated multi-sensor pipeline). The test partition is maintained strictly separate from training and validation throughout development.
42 No-Good · 8 Good. Strictly separated from training and validation throughout all development activities.
Defect dimensions measured in millimeters against the ground truth established by physical inspection of each sample.
Accuracy ≥ 90% · MDR ≤ 10% · FDR ≤ 10% · Per-image processing within real-time conveyor budget.
The quantitative evaluation is supported by the confusion matrix and validation loss analysis. Seven derived performance metrics characterize the model and system; three more define the thermal anomaly rule, the soft-voting ensemble, and the composite severity score that produces every verdict.
Fraction of predicted No-Good classifications that were truly defective. Minimizes unnecessary rejection of conforming sheets.
Fraction of actual defective sheets correctly detected. Critical for safety — missed defects propagate downstream.
Harmonic mean of precision and recall. Particularly informative under class imbalance (8 Good vs. 42 No-Good).
Overall classification correctness. Primary acceptance criterion — minimum 90% threshold.
Mean Average Precision at IoU 0.50 (lenient) and averaged across 0.50–0.95 (strict). Primary YOLO benchmark metric.
Proportion of Good sheets incorrectly flagged as defective. Drives unnecessary rejection cost. Target ≤ 10%.
Proportion of defective sheets the system missed. Safety-critical — these defects reach downstream. Target ≤ 10%.
Three conditions hold simultaneously across 3 consecutive frames. Two filters reject sensor noise; persistence rejects transients.
Weighted average of per-modality confidence. Weights calibrated to per-sensor accuracy (90% webcam / 80% thermal). τ = 0.50 decision threshold.
0–100 score blending visual detections, thermal anomaly, hotspots, and fusion overlaps. Each component capped so no single sensor dominates.
How we solved each objective — with the evidence on the test set.
Each result that follows answers one of these five objectives directly — with the specific numbers, artifacts, and evidence that solve it.
Custom-collected dataset plus NEU-DET supplement trained YOLOv11 to 0.904 mAP@0.5 on the held-out test set — clearing the 90% literature threshold for prototype-level surface defect detection.
Three YOLO variants trained under identical conditions. YOLOv11 won on per-class F1 and validation loss. Convergent mounting geometry adopted for both sensors at comparable working distances.
mAP 0.791 · F1 0.776
mAP 0.904 · F1 0.889
mAP 0.847 · F1 0.835
Three-tier architecture: Vite/React frontend on Vercel, Supabase PostgreSQL backend with row-level security, Python hardware acquisition layer streaming from the Mini-PC over HTTPS.
Average pipeline latency per inspection cycle — ~9.4 fps continuous operation, well within the 200 ms per-sheet budget. Validated at 12 cm/s conveyor speed under controlled LED illumination.
Overall classification accuracy exceeds the 90% threshold by 6 percentage points. Missed-detection rate is well below the 10% safety-critical ceiling. False-detection rate marginally exceeds the 10% target — traced to one surface-oxidation case with a defined mitigation path.
How we met each of the five objectives — with the specific evidence.
Every claim in this thesis is bounded by the conditions we actually tested. Every gap has an explicit next step.
Only 8 Good samples — a single FP moves FDR by 12.5 pts. Fix: expand to balanced n ≥ 200.
Trained at 12 cm/s, stable LED. Real factories have variable lighting, vibration, dust. Fix: site-specific re-calibration before deployment.
Webcam underperforms at sheet corners due to barrel distortion and uneven edge illumination. Fix: lens correction + directional side lighting.
Only two defect classes trained. Pitting, scale, laminar cracks excluded. Fix: dataset expansion + retraining.
No formal R&R budget. Fix: per-modality repeatability/reproducibility study to close ISO 10012 §7.3.
One borderline surface-oxidation case (LIVE-003). Fix: fusion veto rule — reject NG when thermal Δ<1°C and YOLO conf<0.55.
It's deployed. You can open it now.
React frontend on Vercel, Supabase PostgreSQL backend, Python hardware acquisition layer streaming from the Mini-PC. The same nine modules you saw earlier — running live.
We've prepared backing materials in case the panel wants to dig deeper.