Abstract
The Text-to-SQL task translates natural language questions into SQL queries,
enabling intuitive database interaction for non-experts. While recent methods
leveraging Large Language Models (LLMs) achieve strong performance, their
reliance on proprietary models raise concerns about deployment feasibility and
data privacy. In this work, we introduce LitE-SQL, a Lightweight and Efficient
framework with two components: (i) a Schema Retriever that performs efficient
schema linking using a vector database of pre-computed schema embeddings, and
(ii) a SQL Generator fine-tuned in two stages-supervised fine-tuning followed
by execution-guided reinforcement-enabling self-correction without costly
multi-candidate generation. On BIRD, LitE-SQL achieves 72.10% execution
accuracy, and on Spider 1.0 it reaches 88.45%, demonstrating comparable or
superior performance to LLM-based methods despite using 2x to 30x fewer
parameters. Our findings demonstrate that high-quality Text-to-SQL generation
is feasible with lightweight models, offering a practical solution for
privacy-sensitive and resource-constrained settings.
Cornell University and G
Abstract
In the business domain, where data-driven decision making is crucial,
text-to-SQL is fundamental for easy natural language access to structured data.
While recent LLMs have achieved strong performance in code generation, existing
text-to-SQL benchmarks remain focused on factual retrieval of past records. We
introduce CORGI, a new benchmark specifically designed for real-world business
contexts. CORGI is composed of synthetic databases inspired by enterprises such
as Doordash, Airbnb, and Lululemon. It provides questions across four
increasingly complex categories of business queries: descriptive, explanatory,
predictive, and recommendational. This challenge calls for causal reasoning,
temporal forecasting, and strategic recommendation, reflecting multi-level and
multi-step agentic intelligence. We find that LLM performance drops on
high-level questions, struggling to make accurate predictions and offer
actionable plans. Based on execution success rate, the CORGI benchmark is about
21% more difficult than the BIRD benchmark. This highlights the gap between
popular LLMs and the need for real-world business intelligence. We release a
public dataset and evaluation framework, and a website for public submissions.