Kuaishou Technology, Peki
Abstract
Retrieval-Augmented Generation (RAG) has emerged as a promising approach to
address key limitations of Large Language Models (LLMs), such as hallucination,
outdated knowledge, and lacking reference. However, current RAG frameworks
often struggle with identifying whether retrieved documents meaningfully
contribute to answer generation. This shortcoming makes it difficult to filter
out irrelevant or even misleading content, which notably impacts the final
performance. In this paper, we propose Document Information Gain (DIG), a novel
metric designed to quantify the contribution of retrieved documents to correct
answer generation. DIG measures a document's value by computing the difference
of LLM's generation confidence with and without the document augmented.
Further, we introduce InfoGain-RAG, a framework that leverages DIG scores to
train a specialized reranker, which prioritizes each retrieved document from
exact distinguishing and accurate sorting perspectives. This approach can
effectively filter out irrelevant documents and select the most valuable ones
for better answer generation. Extensive experiments across various models and
benchmarks demonstrate that InfoGain-RAG can significantly outperform existing
approaches, on both single and multiple retrievers paradigm. Specifically on
NaturalQA, it achieves the improvements of 17.9%, 4.5%, 12.5% in exact match
accuracy against naive RAG, self-reflective RAG and modern ranking-based RAG
respectively, and even an average of 15.3% increment on advanced proprietary
model GPT-4o across all datasets. These results demonstrate the feasibility of
InfoGain-RAG as it can offer a reliable solution for RAG in multiple
applications.
Whoop, Boston, MA, USA
Abstract
We consider a streaming signal in which each sample is linked to a latent
class. We assume that multiple classifiers are available, each providing class
probabilities with varying degrees of accuracy. These classifiers are employed
following a straightforward and fixed policy. In this setting, we consider the
problem of fusing the output of the classifiers while incorporating the
temporal aspect to improve classification accuracy. We propose a state-space
model and develop a filter tailored for realtime execution. We demonstrate the
effectiveness of the proposed filter in an activity classification application
based on inertial measurement unit (IMU) data from a wearable device.
AI Insights - The filter models class probabilities with a Dirichlet prior, enabling principled Bayesian updates on streaming data.
- Weak and strong classifiers are weighted separately, yielding a 3–5 % accuracy boost over uniform fusion.
- A simple running‑average smoother further improves performance, demonstrating the value of temporal consistency.
- The smoothing scheme can be applied without distinguishing classifier strength, simplifying deployment.
- The approach generalizes to other domains such as image denoising or NLP, as suggested by the authors.
- Key references include “Bayesian Filtering and Smoothing” by S. Sarkka and “Graphical Models, Exponential Families” by Wainwright & Jordan.
- Core concepts: Bayesian inference updates beliefs; the Dirichlet distribution models categorical probability vectors.