02. Hardware-Aware Model Compression and Shift-Robust Adaptive Offloading for Edge–Cloud Collaborative Visual Inspection

Executive Summary Working Title: Shift-Calibrated Split Computing and Adaptive Offloading for Industrial Visual Inspection

In high-speed industrial manufacturing—such as the textile looms I studied in our YOLOv8 + CNN ensemble paper (ICCA 2024) or semiconductor wafer lines—cameras capture dozens of high-resolution frames per second. Running a heavy multi-model ensemble on an embedded edge device violates frame-rate deadlines, while streaming raw $2048 \times 2048$ video from $50$ cameras to a central server saturates factory Wi-Fi/5G uplinks. Edge–Cloud Collaborative Inference offers a middle ground: execute a compressed lightweight detector on the edge and offload only uncertain frames or compressed intermediate features to the cloud. However, existing offloading schemes break down when factory lighting changes, fabric dye lots change, or camera optics accumulate dust. This research topic investigates hardware-aware compression coupled with shift-robust adaptive offloading, ensuring that neither INT8 quantization nor bottleneck entropy coding destroys the uncertainty signal needed to catch rare defects under domain drift.

Tags: #Edge-Cloud Offloading #Model Compression #Visual Inspection #Split Computing #Industrial AI

Research Motivation & Feasibility

Strongest industry-systems option for edge AI and smart manufacturing, directly building on my published YOLOv8+CNN textile defect inspection work (ICCA 2024). Especially attractive when paired with an industry-linked supervisor or edge hardware testbed.

Core Identified Research Gap

Existing edge–cloud offloading and split-computing systems (Matsubara et al., 2022) compress intermediate activations or offload frames based on static confidence thresholds measured on clean factory images. In real production lines, camera lens contamination, motion blur, or dye-lot illumination shifts interact non-linearly with quantization/bottleneck compression—causing the edge device to either drop subtle micro-defects or flood the wireless link.

Basic Information

Field	Details
Core Focus	Top Applied Systems & Edge AI Option
Final Research Title	Hardware-Aware Model Compression and Shift-Robust Adaptive Offloading for Edge–Cloud Collaborative Visual Inspection
Working Title	Shift-Calibrated Split Computing and ROI Offloading for Real-Time Defect Inspection
Research Area	Edge AI, Embedded Computer Vision & Distributed Systems
Subfield	Split Computing, Quantization-Aware Calibration & Collaborative Inference
Target Academic Level	MSc Thesis / Applied PhD Track
Current Status	Ready for Industry-Linked or Systems Lab Supervision

Idea Summary

2.1 One-Sentence Core Thesis

Co-design a quantized edge detector and an entropy-coded feature/ROI offloading gate that remains calibrated under illumination and sensor drift, minimizing false-negative defect escapes within strict frame-rate and network-bandwidth budgets.

2.2 Research Motivation & Strategic Fit

Direct Proven Track Record: Having already published on real-time textile fabric defect detection using YOLOv8 and CNNs (ICCA 2024), I know the exact failure modes of industrial inspection pipelines (false negatives on low-contrast thread breaks, latency spikes when running ensembles).
Practical Scope: Its strongest impact is realized when paired with an edge hardware testbed (e.g., Jetson Orin) or an industrial manufacturing lab, whereas #1 is purely self-contained on any single GPU. Even without physical factory hardware, however, hardware-in-the-loop LUT profiling and network trace simulation make this 100% executable.

Problem Exploration & Identified Research Gap

3.1 Where Existing Edge–Cloud Pipelines Fail

1
Quantization Corrupts Offloading Gates: When an edge backbone is quantized to INT8/FP8 for real-time throughput, asymmetric activation clipping alters logit margins and inflates Expected Calibration Error ($\text{ECE}$). An offloading gate calibrated on the FP32 model makes systematically wrong offloading decisions once quantized.
2
Full-Frame Offloading Wastes Bandwidth: In high-resolution surface inspection, defects occupy $< 2\%$ of pixels. Offloading the entire frame (or full dense feature tensor) when only two $128 \times 128$ patches have high epistemic uncertainty wastes $>90\%$ of wireless bandwidth.
3
Unmodeled Covariate Drift: Under gradual lighting or contrast shift, existing split-computing bottlenecks (Matsubara et al., 2022) suffer from quantization mismatch because activation ranges drift outside static calibration scales.

3.2 Explicit Research Gap

Prior work optimizes model compression (PTQ/QAT), entropic bottleneck coding (Matsubara et al., 2022), and uncertainty-based cloud offloading as three isolated stages. *No existing industrial inspection framework jointly optimizes quantization scales and ROI-sparse bottleneck compression to preserve calibrated spatial uncertainty under covariate shift, dynamically adapting offloading granularity (local accept vs. sparse ROI patch offload vs. full feature offload) to instantaneous network bandwidth $BW_t$.*

Proposed Research Direction

4.1 Core Hypothesis

Hypothesis: Replacing full-frame confidence offloading with shift-calibrated spatial uncertainty ROI offloading—where INT8 quantization scales are regularized to preserve predictive entropy ranking and only high-uncertainty spatial tiles are entropy-coded to the cloud—will reduce uplink bandwidth by $>75\%$ and cut defect escape rates by $>35\%$ under illumination/blur shift at $30\text{ FPS}$.

4.2 Testable Research Questions

RQ1: How severely does standard INT8/INT4 post-training quantization degrade spatial uncertainty calibration and offloading recall under industrial lighting/blur corruptions?
RQ2: Can a lightweight Tile-Level Epistemic Gate offload only the top-$K$ uncertain spatial patches/ROIs (with context padding) rather than the full image while matching full-cloud ensemble recall?
RQ3: How should the offloading controller adapt its threshold $\tau(BW_t)$ when wireless uplink bandwidth fluctuates dynamically?

4.3 System Architecture & Mathematical Objective

Given input frame $X_t \in \mathbb{R}^{3 \times H \times W}$ partitioned into a grid of spatial tiles $\{P_{i,j}\}$, the quantized edge model $f_{\text{edge}}^{(q)}$ produces local predictions $\hat{y}_{i,j}$ and tile uncertainty scores $u_{i,j}$ (combining spatial entropy and local feature norm drift). Given available uplink bit-budget $B_{\text{link}}(t)$, the offloading mask $m_{i,j} \in \{0, 1\}$ solves a knapsack-like selection:

$$ \max_{\{m_{i,j} \in \{0,1\}\}} \sum_{i,j} m_{i,j}\, u_{i,j} \qquad \text{subject to} \qquad \sum_{i,j} m_{i,j}\, \text{Bits}\big(z_q(P_{i,j})\big) \le B_{\text{link}}(t) $$

Interactive Diagram SpecificationMermaid Flow / Schedule

flowchart LR
    CAM["High-FPS Industrial Camera<br/>Frame X_t"] --> EDGE["Quantized Edge Detector f_edge^(q)<br/>+ Tile-Wise AdaBN"]
    EDGE --> MAP["Spatial Uncertainty Map u_{i,j}<br/>(Entropy + Feature Drift)"]
    MAP --> DEC{"Bandwidth-Aware Gate<br/>u_{i,j} > τ(BW_t) ?"}
    DEC -- "m_{i,j} = 0 (Clear Normal / Confident)" --> LOCAL["Commit Edge Result Locally<br/>(< 4 ms Latency)"]
    DEC -- "m_{i,j} = 1 (Ambiguous Defect ROI)" --> CODEC["Sparse ROI Bottleneck Encoder<br/>Transmit Top-K Patches"]
    CODEC --> CLOUD["Cloud Heavy Verifier Ensemble f_cloud<br/>Update Defect Decision"]

Preliminary Technical Plan

Datasets: MVTec-AD (15 categories), VisA (12 categories), and TILDA/NEU surface defect datasets augmented with realistic factory corruptions (LED flicker, Gaussian/motion blur, lint/dust occlusion, contrast shift).
Edge & Cloud Models:
Edge: INT8-quantized YOLOv8-Nano / MobileNetV4-Conv-Small with spatial uncertainty head.
Cloud: High-resolution Swin-Base / PatchCore + YOLOv8-X ensemble verifier.
Evaluation Metrics:
Defect Escape Rate (False Negative Rate at $99\%$ Recall Target).
End-to-End Frame Latency ($95\text{th}$ & $99\text{th}$ percentile ms) under simulated 5G/Wi-Fi traces (mahimahi / tc-netem).
Average Uplink Bandwidth (KB/frame) vs. AUROC / Pixel-AUPRO.

Expected Contributions

1
Calibration-Preserving Quantization Loss: A KL-and-entropy-ranking regularizer during QAT/PTQ that prevents INT8 clipping from flattening offloading uncertainty scores.
2
Sparse ROI Bottleneck Offloader: An adaptive tile-offloading protocol that transmits $<10\%$ of frame bytes by isolating uncertain candidate defect regions.
3
Edge-to-Cloud Simulation Benchmark: A reproducible PyTorch + ONNX latency/bandwidth simulator for collaborative industrial inspection.

Feasibility, Scope, and Honest Risks

Feasibility: Training and evaluation run on $1\times$ GPU; edge latency can be measured accurately via TensorRT/ONNX Runtime profiling or a single Jetson board.
Risk & Fallback: If learned neural compression (CompressAI) adds too much encoding latency on the edge CPU/NPU, fallback to adaptive ROI crop + hardware JPEG/WebP quality scaling guided by the exact same spatial uncertainty gate $u_{i,j}$.

Initial Literature Notes

[1]
Y. Matsubara, M. Levorato, and F. Restuccia, "Split Computing and Early Exiting for Deep Learning Applications: Survey and Research Challenges," ACM Computing Surveys, 55(5), 2022.
[2]
R. Islam et al., "Ensemble Deep Learning for Real-Time Textile Fabric Defect Detection and Classification Using YOLOv8 and CNNs," ICCA, 2024 — Our baseline multi-model inspection pipeline.
[3]
M. Nagel et al., "A White Paper on Neural Network Quantization," arXiv:2106.08295, 2021.

Idea Validation Checklist

Grounded in real manufacturing bottleneck from our ICCA 2024 textile defect studyVerified
Clear synergy between hardware compression, shift robustness, and bandwidth controlVerified
Hardware fallback defined (ONNX/TensorRT profiling + network trace simulation)Verified
Run preliminary INT8 vs. FP32 ECE comparison on NEU/MVTec-AD under illumination shiftPending

Current Assessment

Criterion	Score	My Assessment Notes
Problem clarity	9 / 10	Concrete latency-bandwidth-recall trade-off in visual inspection.
Novelty potential	8 / 10	Coupling quantization calibration with sparse ROI offloading under shift is under-explored.
Technical feasibility	9 / 10	Directly extends my existing YOLOv8 + CNN codebase.
Resource feasibility	8 / 10	Best with an edge device (Jetson), though fully simulatable on 1 GPU.
Dataset availability	9 / 10	MVTec-AD, VisA, and NEU are publicly available.
Publication potential	9 / 10	Strong fit for IEEE TII, IEEE IoT Journal, WACV, or CVPR Edge workshops.
Personal interest	9 / 10	High practical engineering satisfaction.

Next Actions

1
Quantize our trained YOLOv8n/ResNet defect detector to INT8 and measure how much $\text{ECE}$ degrades under synthetic brightness/blur shifts.
2
Prototype the tile-level knapsack selector under a $50\text{ KB/frame}$ bandwidth cap.

Open Questions and Working Notes

Can the cloud server send back lightweight affine normalization updates $(\Delta\gamma, \Delta\beta)$ to the edge device asynchronously whenever it detects systematic illumination drift on offloaded ROIs?