My working catalog of 15 curated research directions—grounded in theoretical defensibility,
single-GPU experimental feasibility, novelty differentiation against 2024–2026 literature, and alignment with my
background in reliable ML, computer vision, GNNs, and LLM systems.
Showing 15 of 15 topics
Topic 01
Adaptive Inference & Reliable ML
·RT-2026-01
Best overall balance of reliable ML, adaptive inference, single-GPU feasibility, and industry relevance. Novelty comes from our specific shift-aware primal-dual routing rule and proving how existing confidence-based cascades break under covariate drift.
Identified Literature Gap & Failure Mode
Existing two-tier model cascades (e.g., Jitkrittum et al., 2023; Dekoninck et al., 2024) assume the deferral threshold calibrated on an i.i.d. validation split remains valid at test time. When covariate shift hits, the compact Stage-1 model becomes overconfident on corrupted inputs—failing to defer right when the heavy Stage-2 model is needed most—or else triggers a cascade storm that blows past the fixed compute budget.
Proposed Direction: I want to build a two-stage routing controller between a fast compact model (f_L) and a high-capacity model (f_H, plus an abstention option) that explicitly monitors representation-space drift and updates its deferral threshold online via primal-dual Lagrange multipliers—keeping selective error below r* without exceeding a strict average FLOPs budget B_max.
Strongest industry-systems option for edge AI and smart manufacturing, directly building on my published YOLOv8+CNN textile defect inspection work (ICCA 2024). Especially attractive when paired with an industry-linked supervisor or edge hardware testbed.
Identified Literature Gap & Failure Mode
Existing edge–cloud offloading and split-computing systems (Matsubara et al., 2022) compress intermediate activations or offload frames based on static confidence thresholds measured on clean factory images. In real production lines, camera lens contamination, motion blur, or dye-lot illumination shifts interact non-linearly with quantization/bottleneck compression—causing the edge device to either drop subtle micro-defects or flood the wireless link.
Proposed Direction: Building on my real-time textile defect detection research, this project designs a collaborative edge–cloud inspection pipeline where a quantized edge stem processes 100% of high-FPS camera frames locally and uses a shift-calibrated bottleneck gate to offload only ambiguous ROIs or compressed latent tensors to a cloud server under bandwidth constraints.
Strong low-compute alternative that directly leverages my graph-learning and smart-grid/feature-correlation background (ICCIT 2024, ICEEICT 2024). Multivariate sensor datasets train in under 30 minutes on a single mid-range GPU, allowing rapid mathematical iteration.
Identified Literature Gap & Failure Mode
Graph Structure Learning (GSL) models for multivariate sensor anomaly and fault diagnosis (e.g., GDN, Deng & Hooi, AAAI 2021; MTGNN, Wu et al., KDD 2020) learn a deterministic top-k adjacency matrix from clean training data. When a subset of sensors drops out, freezes, or drifts under a new operating regime, standard GNN message passing blindly aggregates and amplifies those corrupted sensor readings across L-hop neighborhoods—causing healthy subsystems to be misclassified as faulty.
Proposed Direction: I propose an uncertainty-calibrated Graph Structure Learning (GSL) architecture where each sensor node estimates both a state representation and an aleatoric/epistemic reliability score; learned graph edges dynamically attenuate incoming messages from missing or noisy neighbors and output conformal fault localization sets.
Very manageable experimental scope and a clean efficiency–reliability research question on Vision Transformers. The novelty burden here requires careful differentiation: both Early-Exit ViTs and Evidential Deep Learning are active areas, so we must directly address recent critiques of Evidential DL under shift (Jürgens et al., 2024).
Identified Literature Gap & Failure Mode
Existing Early-Exit Vision Transformers halt based on first-order softmax entropy, which cannot separate in-distribution class overlap (aleatoric) from out-of-distribution unfamiliarity (epistemic). Conversely, standard Evidential Deep Learning (Sensoy et al., 2018) placed only at the final layer suffers from vanishing evidence gradients and fails to use intermediate block trajectory consistency to detect covariate shift.
Proposed Direction: Attach lightweight Dirichlet evidential heads at intermediate Vision Transformer blocks to decompose vacuity (epistemic ignorance) and dissonance (conflicting class evidence) along network depth—halting early on low-vacuity/low-dissonance samples, escalating high-dissonance samples to deeper blocks, and abstaining at the final block when vacuity remains high.
Excellent long-term PhD direction in robust and continual learning that directly extends my Research Note #09 (Test-Time Adaptation). Continual unsupervised adaptation over thousands of unlabeled test batches is notoriously prone to instability, requiring careful mathematical bounds.
Identified Literature Gap & Failure Mode
Existing Test-Time Adaptation methods (TENT, Wang et al., 2021; SAR, Niu et al., 2023) filter out high-entropy samples (H(x) > E_max) to avoid noisy gradients, but they continue updating affine normalization parameters on already overconfident low-entropy samples over long continual streams—gradually eroding calibration, collapsing class margins, and accumulating irreversible drift when the environment cycles back to clean data.
Proposed Direction: Develop an Entropy-Bounded Continual Test-Time Adaptation (EB-CTTA) algorithm that restricts online affine updates to a 'Goldilocks' informative entropy band [E_min, E_max] while anchoring normalization parameters to a source Fisher-information trust region.
Directly useful for real-world model deployment and efficient edge inference. To make this a genuine research contribution rather than an engineering exercise, the focus is strictly on preserving probabilistic calibration and shift robustness during low-bit quantization, not merely applying off-the-shelf PTQ.
Identified Literature Gap & Failure Mode
Standard Post-Training Quantization (PTQ, e.g., BRECQ, Li et al., 2021; AdaRound, Nagel et al., 2020) minimizes layer-wise activation MSE or clean task cross-entropy on a small calibration set. However, clipping activation outliers to fit 4-bit or 8-bit grids compresses logit dynamic range unevenly across classes—inflating Expected Calibration Error (ECE) under covariate shift and erasing the logit-norm / energy signal needed for OOD detection.
Proposed Direction: Design a Calibration-Preserving Post-Training Quantization (CP-PTQ) objective that jointly optimizes quantization rounding variables and layer-wise clipping scales to preserve both the predictive probability simplex (via Tsallis/KL divergence) and Helmholtz free-energy ordering under synthetic corruption perturbations.
Strong modern computer-vision direction combining Vision-Language Models (CLIP), LoRA (Research Note #06), and Early Exiting (Research Note #04). CLIP-based anomaly detection (WinCLIP, AnomalyCLIP) is a very crowded research area with more moving parts.
Identified Literature Gap & Failure Mode
Recent CLIP-based zero/few-shot anomaly detectors (WinCLIP, Jeong et al., CVPR 2023; AnomalyCLIP, Zhou et al., ICLR 2024) extract dense patch embeddings from multiple intermediate ViT layers, yet they still execute the entire 24-layer ViT-Large backbone on every single normal inspection frame—and their raw cosine similarity scores with 'normal'/'damaged' text prompts are severely uncalibrated across unseen product textures.
Proposed Direction: Since 95%+ of industrial inspection frames are defect-free, this project introduces shift-calibrated intermediate cross-modal exit gates into a LoRA-adapted CLIP vision encoder—halting inference at shallow ViT layers for clearly normal items while calibrating cross-modal anomaly probabilities across unseen domains.
Practical, benchmark-friendly industrial vision research with a very realistic application path and fast experimental turnaround. Focuses on patch-memory and density-based anomaly backbones rather than broader adaptive routing architectures.
Identified Literature Gap & Failure Mode
Patch-memory anomaly detectors like PatchCore (Roth et al., CVPR 2022) achieve near-perfect AUROC when given hundreds of normal training images, but in the few-shot regime (k = 1, 2, 4 normal images of a new product), the sparse memory bank cannot cover normal pose/illumination variations—causing raw L2 nearest-neighbor distances to spike on normal edges and producing uncalibrated, threshold-sensitive false alarms.
Proposed Direction: Transform raw few-shot patch nearest-neighbor distances into locally variance-normalized, conformally calibrated anomaly probabilities by estimating local feature covariance around the k normal support images and applying leave-one-patch-out conformal thresholding.
Strong LLM engineering and inference-cost relevance. Evaluating 'query difficulty' and 'domain shift' across open-ended LLM generation requires carefully controlled benchmark design so results aren't confounded by noisy LLM judges.
Identified Literature Gap & Failure Mode
Existing LLM routers (RouteLLM, Ong et al., 2024; HybridLLM, Ding et al., ICLR 2024) train a BERT/Llama classifier on 50,000+ Chatbot Arena preference pairs. When deployed in a specialized enterprise domain (e.g., medical triage, Bangladeshi legal/financial queries, or SQL generation) where labeled preference data is scarce (< 200 examples) and query distributions shift, supervised routers misestimate SLM competence and route complex domain queries to the weak SLM.
Proposed Direction: Build a label-efficient, shift-aware LLM router that combines zero-label internal SLM probes (first-token prefill entropy and hidden-state Mahalanobis density) with active few-shot calibration (< 150 labeled queries) to route between a fast local SLM (1.5B–3B) and a heavy oracle model under strict API cost budgets.
Excellent fit with my full-stack software engineering experience and builds a distinctive research identity in reliable AI-assisted programming. Integrating formal static/AST verifiers into token/block decoding requires careful engineering across language parsers.
Identified Literature Gap & Failure Mode
Current code generation benchmarks (HumanEval, MBPP, SWE-bench) evaluate functional correctness (pass@k) while ignoring security vulnerabilities (e.g., SQL injection CWE-89, path traversal CWE-22, unsafe deserialization CWE-502). Existing post-hoc repair agents run static analyzers only after generating the full program—and when the model lacks knowledge of a secure library API, iterative self-repair loops often silence the linter warning by breaking functionality or introducing a deeper logic flaw instead of abstaining.
Proposed Direction: Combine statement-level AST/taint-analysis-guided decoding with calibrated risk-aware abstention in code LLMs—steering generation away from insecure CWE patterns at statement boundaries and abstaining with an explicit security diagnostic when no verified patch satisfies both functional tests and security invariants.
Highly practical enterprise AI topic that builds on my RAG Evaluation Frameworks note (Research Note #07). Academic research value depends strictly on developing a novel cross-modal evidence-sufficiency and decision-theoretic abstention method rather than simply engineering a RAG application.
Identified Literature Gap & Failure Mode
Existing RAG faithfulness evaluators (RAGAS, Es et al., 2024; Self-RAG, Asai et al., ICLR 2024) operate purely on text passages and assume that top-k retrieval scores reflect evidence sufficiency. In multimodal technical domains—where answering a diagnostic query requires aligning a retrieved wiring/piping schematic figure, a tabular fault-code log, and a maintenance manual text chunk—VLMs frequently hallucinate missing steps or misread diagram labels when one modality is missing or conflicting.
Proposed Direction: Formulate a Cross-Modal Evidence Sufficiency Score that decomposes a maintenance query into required visual/tabular/textual sub-facts, verifies bi-directional entailment across retrieved diagrams and text chunks, and triggers conformal risk-aware abstention when multimodal evidence is insufficient.
Valuable ML-systems direction for long-context LLM serving on memory-bounded GPUs. A wave of 2024–2026 papers (H2O, SnapKV, PyramidKV, DuoAttention) makes a sharp, defensible theoretical contribution essential.
Identified Literature Gap & Failure Mode
Leading training-free KV-cache eviction methods (H2O, Zhang et al., NeurIPS 2023; SnapKV, Li et al., 2024) evict tokens based solely on cumulative attention weights A_ij summed over past queries. This suffers from three mathematical flaws: (1) early tokens accumulate higher sums purely by being present for more steps (position bias), (2) it ignores the L2 norm and directional uniqueness of the Value vector ||v_j||_2, and (3) redundant duplicate tokens split attention mass and both get evicted!
Proposed Direction: Formulate KV-cache eviction as minimizing the upper bound on attention output reconstruction error ||Attn(Q,K,V) - Attn(Q,K_S,V_S)||_2—scoring cached tokens by exposure-corrected attention × value residual norm and merging near-collinear key-value pairs instead of blindly dropping them.
Good potential fit for smart manufacturing, healthcare diagnostics, sensing, agriculture, and infrastructure projects, directly building on my Deep Multi-Modal Protein Function Prediction paper (ICEEICT 2024). Finding public datasets with genuinely synchronized high-rate vision + industrial IoT telemetry under realistic corruptions is slightly harder than vision-only or sensor-only benchmarks.
Identified Literature Gap & Failure Mode
Standard multimodal fusion architectures (concatenation, cross-attention, or static late fusion) suffer from 'modality dominance' and fail catastrophically under asymmetric sensor degradation: if the visual camera is occluded by steam/glare while vibration sensors remain clean (or vice versa), static cross-attention allows the corrupted modality to poison the joint representation—and post-hoc attributions fail to tell operators which modality drove the alarm.
Proposed Direction: Design an Uncertainty-Calibrated Dynamic Multimodal Fusion (UC-DMF) framework using Dirichlet/Normal-Inverse-Gamma precision weighting across visual and IoT sensor branches, paired with closed-form modality Shapley attributions so operators see exact modality contributions and reliability scores.
Relevant to distributed and privacy-preserving AI across hospitals or manufacturing sites. Simulating realistic multi-client domain heterogeneity, stragglers, and communication rounds is more complex to evaluate rigorously in a resource-limited MSc than single-node inference topics.
Identified Literature Gap & Failure Mode
Standard Federated Learning (FedAvg, McMahan et al., 2017; FedProx) transmits full model weights every round and suffers from client drift when each client has a distinct domain style (e.g., different X-ray scanners or factory lighting). Meanwhile, naively averaging LoRA adapters across heterogeneous clients ((1/K)Σ B_k A_k != (1/K Σ B_k)(1/K Σ A_k)) introduces cross-term interference noise and fails to generalize to an unseen (K+1)-th target client!
Proposed Direction: Combine Federated Low-Rank Adaptation (communicating < 0.8% of backbone parameters per round) with orthogonal subspace alignment (fixing the quadratic BA averaging cross-term error) and local-only normalization statistics so the aggregated global model generalizes zero-shot to unseen target domains.
Ambitious and mathematically elegant AI-safety research direction. Proving that a concept is truly erased (against adversarial re-elicitation / multi-turn probes) while preserving closely adjacent benign capabilities (e.g., erasing bioweapon synthesis while retaining undergraduate virology) is notoriously difficult.
Identified Literature Gap & Failure Mode
Linear concept erasure methods like LEACE (Belrose et al., NeurIPS 2023) guarantee zero linear decodability at a single layer, but subsequent nonlinear MLP blocks can reconstruct the concept from correlated proxy features—while gradient-ascent unlearning methods (RMU, Li et al., ICML 2024) suffer from 'collateral capability spillover' that degrades performance on closely related benign topics.
Proposed Direction: Formulate targeted concept erasure as a closed-form generalized eigenvalue / oblique subspace projection applied directly to mid-layer MLP down-projection matrices W_down—maximizing erasure of the target forget-concept covariance Σ_forget while strictly bounding perturbation on the adjacent retain-concept covariance Σ_retain.
W_down* = W_down (I - P_erase), where P_erase solves max Tr(P^T Σ_F P) s.t. Tr(P^T Σ_R P) ≤ ε
Best overall balance of reliable ML, adaptive inference, single-GPU feasibility, and industry relevance. Novelty comes from our specific shift-aware primal-dual routing rule and proving how existing confidence-based cascades break under covariate drift.
Strongest industry-systems option for edge AI and smart manufacturing, directly building on my published YOLOv8+CNN textile defect inspection work (ICCA 2024). Especially attractive when paired with an industry-linked supervisor or edge hardware testbed.
Strong low-compute alternative that directly leverages my graph-learning and smart-grid/feature-correlation background (ICCIT 2024, ICEEICT 2024). Multivariate sensor datasets train in under 30 minutes on a single mid-range GPU, allowing rapid mathematical iteration.
Very manageable experimental scope and a clean efficiency–reliability research question on Vision Transformers. The novelty burden here requires careful differentiation: both Early-Exit ViTs and Evidential Deep Learning are active areas, so we must directly address recent critiques of Evidential DL under shift (Jürgens et al., 2024).
Excellent long-term PhD direction in robust and continual learning that directly extends my Research Note #09 (Test-Time Adaptation). Continual unsupervised adaptation over thousands of unlabeled test batches is notoriously prone to instability, requiring careful mathematical bounds.
Directly useful for real-world model deployment and efficient edge inference. To make this a genuine research contribution rather than an engineering exercise, the focus is strictly on preserving probabilistic calibration and shift robustness during low-bit quantization, not merely applying off-the-shelf PTQ.
Strong modern computer-vision direction combining Vision-Language Models (CLIP), LoRA (Research Note #06), and Early Exiting (Research Note #04). CLIP-based anomaly detection (WinCLIP, AnomalyCLIP) is a very crowded research area with more moving parts.
Practical, benchmark-friendly industrial vision research with a very realistic application path and fast experimental turnaround. Focuses on patch-memory and density-based anomaly backbones rather than broader adaptive routing architectures.
Excellent fit with my full-stack software engineering experience and builds a distinctive research identity in reliable AI-assisted programming. Integrating formal static/AST verifiers into token/block decoding requires careful engineering across language parsers.
Highly practical enterprise AI topic that builds on my RAG Evaluation Frameworks note (Research Note #07). Academic research value depends strictly on developing a novel cross-modal evidence-sufficiency and decision-theoretic abstention method rather than simply engineering a RAG application.
Valuable ML-systems direction for long-context LLM serving on memory-bounded GPUs. A wave of 2024–2026 papers (H2O, SnapKV, PyramidKV, DuoAttention) makes a sharp, defensible theoretical contribution essential.
Good potential fit for smart manufacturing, healthcare diagnostics, sensing, agriculture, and infrastructure projects, directly building on my Deep Multi-Modal Protein Function Prediction paper (ICEEICT 2024). Finding public datasets with genuinely synchronized high-rate vision + industrial IoT telemetry under realistic corruptions is slightly harder than vision-only or sensor-only benchmarks.
Relevant to distributed and privacy-preserving AI across hospitals or manufacturing sites. Simulating realistic multi-client domain heterogeneity, stragglers, and communication rounds is more complex to evaluate rigorously in a resource-limited MSc than single-node inference topics.
Ambitious and mathematically elegant AI-safety research direction. Proving that a concept is truly erased (against adversarial re-elicitation / multi-turn probes) while preserving closely adjacent benign capabilities (e.g., erasing bioweapon synthesis while retaining undergraduate virology) is notoriously difficult.