Object Detection That Misses Occluded or Tiny Parts
A part moves down the line, a robotic arm blocks it for an instant, and the AI model never logs it. Months later the customer returns the product, and nobody can prove what was assembled.
Illustrative scenario based on common operational problems. The company remains anonymous.
The Case
A contract electronics manufacturer runs automated assembly lines. Every unit has to be documented: which parts it contains and which ones were mounted. Today a person reviews and signs a record, or only a sample of the units is checked. To document every unit at line speed, the plant installed a camera and an object detection model that watches each part go by and logs it automatically. It is a single-pass model (a one-stage detector, YOLO type), chosen for its speed.
The model works well when the scene is clear. It fails when a robotic arm, a tool, or a box covers part of the component (occlusion), when the part is so small it is barely visible, or when the background is so cluttered it confuses the model. An unlogged part keeps moving as an invisible defect and comes back as rework, a customer return, or an audit penalty, because without a record of every part there is no traceability.
The architecture decision is what to do before reacting. Buying licenses or replacing the model entirely, without knowing why the current one fails, can cost a lot and leave the problem unchanged. The right question is in exactly which cases it fails and what the cheapest fix is for each.
The Solution Architecture
Figure 1 places the AI component inside a typical electronics assembly plant. The gray boxes (MES, traceability database, QMS, RMA system) are illustrative examples that show where the solution fits, and the blue boxes are the AI architecture. A PLC triggers the camera for each unit, the line edge server detects and tracks every part, and the part events reach the MES. Offline, errors are diagnosed before any model change and the signed model is released back to the edge.
Figure 1. Solution architecture: part-level traceability on an electronics assembly line (illustrative).
The table summarizes the layers of Figure 1.
Layer | Components | What it does |
Sources | Line cameras (1920 px, 20 fps), PLCs and line controllers, robotic arm controllers | Trigger and capture every unit |
Integration | Frame grabber with trigger, line event bus (MQTT), device identity, observability | Sync the camera with the PLC and publish part events |
AI on the line edge server (GPU) | Zone router, YOLOv8n INT8, SAHI tiles, NMS merge, part tracker, part events. RT-DETR and a comparator in shadow mode | Detect, track, and log every part |
AI offline | Error diagnosis, targeted data, training, optimization, model registry | Diagnose, retrain, and release the signed model |
Company systems (illustrative) | MES, traceability database, QMS, returns (RMA) system, HMI | Store each unit's genealogy, audit evidence, and alerts |
Today (as-is): high-resolution image → downscaling to 640 px → one-stage detector (YOLOv8n) on the full frame → non-maximum suppression (NMS) → parts record. It isn't one cause that makes it fail. It is six mechanisms that combine depending on the scene:
Input downsampling. A 1920 px capture becomes 640 px, and a 24 px part ends up at 8 px.
Grid assignment. Small, nearby parts compete for the same cell and only one is learned.
NMS. Two parts that overlap produce very similar boxes and NMS removes one even though it is real (Bodla et al., 2017).
Limited local context. The visible portion of an occluded part isn't enough and confidence falls below the threshold.
Object-background imbalance. Almost everything evaluated is background, which creates false positives on textured backgrounds (Lin et al., 2017).
Invariance that is only learned. The network is not invariant to scale or rotation by design, only if the data covers those variations.
Proposal (to-be): image → error analysis by segment (size and occlusion) → targeted actions → tiled inference (SAHI) in the small-parts zones → RT-DETR evaluation for occlusion → INT8 quantization → tracker.
The as-is architecture is a typical starting point, built from the problem statement and the baselines of the proof of concept (PoC).
Figure 2 details the detection design. The top strip shows how today's one-stage detector fails, with the steps that cause it in red, and they correspond to several of the six mechanisms listed above. The bottom shows the target design: a zone router that sends each frame whole or as SAHI tiles to the INT8 detector, a tracker that keeps each part's identity under brief occlusion, an RT-DETR challenger that runs in shadow mode without deciding, and the offline loop that diagnoses errors and releases the signed model.
Figure 2. AI architecture: robust detection with selective tiling and a shadow challenger.
Architecture Decisions
Decision | Why | Alternative discarded | What is given up |
Diagnose by segment (size, occlusion, background) before changing the model | Each of the six causes has a different fix (Trigka & Dritsas, 2025) | Buying or changing the model blindly | Analysis time before acting |
SAHI only in the small-parts zones, with a zone router (Akyon et al., 2022) | SAHI multiplies inferences and its cost is only paid where small parts are | SAHI over the whole frame | A router by zone to maintain (the PoC applied SAHI to the whole scene) |
RT-DETR as a challenger in shadow mode until it meets 50 ms at p95 (Zhao et al., 2024) | It uses global attention and needs no NMS. It raised the recall of occluded objects from 16% to 35%, but takes about 400 ms on CPU | Replacing the current detector right away. A pure ViT, because its 16 × 16 px patches lose detail on small objects | Double compute while the comparison lasts |
INT8 quantization, with the decoding head kept in FP32 | p95 latency from 48 to 28 ms and a model from 12.8 to 3.6 MB | Leaving the model in FP32 | About 4 points of AP50 (from 0.59 to 0.55) |
A tracker next to the detector | On a conveyor belt, a part that is covered now was visible a few frames earlier (Cohen et al., 2025) | Detecting every frame in isolation | One more component. It was not measured in this PoC |
Targeted data augmentation (erasing, mosaic, copy-paste) | Attacks causes 4 to 6 with synthetic occlusions | Waiting for more real data | Requires a GPU, and was not run in the PoC |
Validation with a Proof of Concept (PoC)
16 scenes were built from labeled COCO128 images (1,066 objects, 502 of them small), plus a synthetic occlusion test with 64 images and 110 objects covered by 40% to 60%. Latencies were measured on a workstation CPU.
Action | Metric | As-is | To-be |
SAHI on small parts | AP50 | 0.25 | 0.42 |
SAHI on small parts | Recall of small objects | 1.8% | 18.5% |
RT-DETR under occlusion | Recall of occluded objects | 16% | 35% |
RT-DETR under occlusion | Recall of unoccluded objects | 73% | 97% |
RT-DETR under occlusion | AP50 on images with occlusion | 0.27 | 0.51 |
INT8 quantization (ONNX) | p95 latency on CPU | 48 ms (FP32) | 28 ms |
INT8 quantization (ONNX) | Model size | 12.8 MB | 3.6 MB |
How to read it. For a line that needs 20 frames per second, the budget is 50 ms per frame (a working assumption). On this CPU, ONNX FP32 was slower than PyTorch (48 vs. 35 ms), so against PyTorch the INT8 improvement is 18%, not 41%. RT-DETR took about 400 ms per image (p95 of 408 ms), eight times the budget.
Risks and Pilot Acceptance Criteria
Proposed criterion (working assumption) | PoC result | Status |
Recall under occlusion ≥ 90% | 35% with RT-DETR | [-] |
p95 latency ≤ 50 ms | 28 ms with YOLOv8n INT8. 408 ms with RT-DETR | [+] and [-] |
[+] met, [-] not met. The thresholds are working assumptions that must be validated with operations.
Risks and what to validate in the pilot:
✓ The occlusion sample is small: 110 occluded objects (RT-DETR found 38 and YOLOv8n found 18). It is a directional result. The pilot needs real occlusion by a robotic arm or an operator.
✓ RT-DETR also improved on unoccluded objects (from 73% to 97%). Part of its advantage comes from being a stronger detector on these scenes, not only from handling occlusion better.
✓ Even with SAHI, recall of small parts is 18.5% (93 of 502). It doubled AP50, but most small parts still go undetected.
✓ Latencies depend on the CPU used. They must be measured on plant hardware, at p95.
✓ Retraining with targeted augmentation was not run, because it requires a GPU, and the tracker was not measured in this PoC.
How to Adapt It to Your Company
✓ If your MES already triggers the start and end of each unit, use it as the camera trigger and as the destination for part events. You don't need a new database for traceability.
✓ If you have no GPU on the line, YOLOv8n INT8 ran on CPU within the budget. RT-DETR does need a GPU or optimization.
✓ If your small parts concentrate in a few zones of the frame, limit SAHI to those zones so you don't pay the latency across the whole image.
✓ Connect the returns system and audit findings to the error diagnosis. It closes the improvement loop with real failure data.
Scope and Out of Scope
In scope | Out of scope |
Scenes with small parts built from COCO128 | Real data from an assembly line |
Synthetic occlusion of 40% to 60% of the object | Real occlusion by a robotic arm or operator |
YOLOv8n on the full frame vs. YOLOv8n + SAHI | Distillation and pruning (require a GPU) |
YOLOv8n vs. RT-DETR-l under occlusion | Latency on the plant's actual hardware |
CPU latency: PyTorch, ONNX FP32, and INT8 | Results of retraining with targeted augmentation |
The PoC used public images (COCO128) with synthetic occlusion, not real parts from an assembly line. The business case is the framing.
Code
The code of the proof of concept (PoC) is published on GitHub:
https://github.com/rodolfo-casanova/lab-02-object-detection-occlusion
References
Akyon, F. C., Altinuc, S. O., & Temizel, A. (2022). Slicing aided hyper inference and fine-tuning for small object detection. In 2022 IEEE International Conference on Image Processing (ICIP) (pp. 966–970). IEEE. https://doi.org/10.1109/ICIP46576.2022.9897990
Bodla, N., Singh, B., Chellappa, R., & Davis, L. S. (2017). Soft-NMS: Improving object detection with one line of code. In Proceedings of the IEEE International Conference on Computer Vision (pp. 5561–5569). https://doi.org/10.1109/ICCV.2017.593
Cohen, Y., Biton, A., & Shoval, S. (2025). Fusion of computer vision and AI in collaborative robotics: A review and future prospects. Applied Sciences, 15(14), 7905. https://doi.org/10.3390/app15147905
Lin, T.-Y., Goyal, P., Girshick, R., He, K., & Dollár, P. (2017). Focal loss for dense object detection. In Proceedings of the IEEE International Conference on Computer Vision (pp. 2980–2988). https://doi.org/10.1109/ICCV.2017.324
Trigka, M., & Dritsas, E. (2025). A comprehensive survey of machine learning techniques and models for object detection. Sensors, 25(1), 214. https://doi.org/10.3390/s25010214
Zhao, Y., Lv, W., Xu, S., Wei, J., Wang, G., Dang, Q., Liu, Y., & Chen, J. (2024). DETRs beat YOLOs on real-time object detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (pp. 16965–16974). https://doi.org/10.1109/CVPR52733.2024.01605
#ComputerVision #ObjectDetection #YOLO #Manufacturing #MLOps
