Abstract
Future autonomous systems promise significant societal benefits, yet their
deployment raises concerns about safety and trustworthiness. A key concern is
assuring the reliability of robot perception, as perception seeds safe
decision-making. Failures in perception are often due to complex yet common
environmental factors and can lead to accidents that erode public trust. To
address this concern, we introduce the SET (Self, Environment, and Target)
Perceptual Factors Framework. We designed the framework to systematically
analyze how factors such as weather, occlusion, or sensor limitations
negatively impact perception. To achieve this, the framework employs SET State
Trees to categorize where such factors originate and SET Factor Trees to model
how these sources and factors impact perceptual tasks like object detection or
pose estimation. Next, we develop Perceptual Factor Models using both trees to
quantify the uncertainty for a given task. Our framework aims to promote
rigorous safety assurances and cultivate greater public understanding and trust
in autonomous systems by offering a transparent and standardized method for
identifying, modeling, and communicating perceptual risks.
Abstract
Visual reasoning is critical for a wide range of computer vision tasks that
go beyond surface-level object detection and classification. Despite notable
advances in relational, symbolic, temporal, causal, and commonsense reasoning,
existing surveys often address these directions in isolation, lacking a unified
analysis and comparison across reasoning types, methodologies, and evaluation
protocols. This survey aims to address this gap by categorizing visual
reasoning into five major types (relational, symbolic, temporal, causal, and
commonsense) and systematically examining their implementation through
architectures such as graph-based models, memory networks, attention
mechanisms, and neuro-symbolic systems. We review evaluation protocols designed
to assess functional correctness, structural consistency, and causal validity,
and critically analyze their limitations in terms of generalizability,
reproducibility, and explanatory power. Beyond evaluation, we identify key open
challenges in visual reasoning, including scalability to complex scenes, deeper
integration of symbolic and neural paradigms, the lack of comprehensive
benchmark datasets, and reasoning under weak supervision. Finally, we outline a
forward-looking research agenda for next-generation vision systems, emphasizing
that bridging perception and reasoning is essential for building transparent,
trustworthy, and cross-domain adaptive AI systems, particularly in critical
domains such as autonomous driving and medical diagnostics.