Papers from 08 to 12 September, 2025

Here are the personalized paper recommendations sorted by most relevant
Data Science Engineering
👍 👎 ♥ Save
Abstract
Large Language Models (LLMs) have shifted in just a few years from novelty to ubiquity, raising fundamental questions for data science education. Tasks once used to teach coding, writing, and problem-solving can now be completed by LLMs, forcing educators to reconsider both pedagogy and assessment. To understand how instructors are adapting, we conducted semi-structured interviews with 42 instructors from 33 institutions in 10 countries in June and July 2025. Our qualitative analysis reveals a pragmatic mix of optimism and concern. Many respondents view LLMs as inevitable classroom tools -- comparable to calculators or Wikipedia -- while others worry about de-skilling, misplaced confidence, and uneven integration across institutions. Around 58 per cent have already introduced demonstrations, guided activities, or make extensive use of LLMs in their courses, though most expect change to remain slow and uneven. That said, 31 per cent have not used LLMs to teach students and do not plan to. We highlight some instructional innovations, including AI-aware assessments, reflective use of LLMs as tutors, and course-specific chatbots. By sharing these perspectives, we aim to help data science educators adapt collectively to ensure curricula keep pace with technological change.
👍 👎 ♥ Save
National Center for Compu
Abstract
Memory-to-memory data streaming is essential for modern scientific workflows that require near real-time data analysis, experimental steering, and informed decision-making during experiment execution. It eliminates the latency bottlenecks associated with file-based transfers to parallel storage, enabling rapid data movement between experimental facilities and HPC systems. These tightly coupled experimental-HPC workflows demand low latency, high throughput, and reliable data delivery to support on-the-fly analysis and timely feedback for experimental control. Off-the-shelf messaging frameworks are increasingly considered viable solutions for enabling such direct memory streaming due to their maturity, broad adoption, and ability to abstract core messaging and reliability functionalities from the application layer. However, effectively meeting the workflows' requirements depends on utilizing the framework's capabilities and carefully tuning its configurations. In this paper, we present a study that investigates the messaging parameters, and their configuration choices that impact the streaming requirements of two representative scientific workflows. We specifically characterize throughput trade-offs associated with reliable message transmission for these workflows. Our study is conducted through streaming simulations using synthetic workloads derived from the Deleria and LCLS workflows, employing the RabbitMQ messaging framework within the context of the Data Streaming to HPC infrastructure at OLCF. Our simulations reveal several key observations and practical insights that help users understand which configurations best meet the needs of their streaming workloads.
AI Insights
  • The study pioneers AI‑coupled HPC workflows that fuse machine‑learning inference with real‑time data streams, slashing analysis latency by an order of magnitude.
  • It proposes a unified, API‑driven research infrastructure that stitches together experimental instruments, data‑caching layers, and HPC back‑ends into a single, secure pipeline.
  • The authors demonstrate how deep‑learning pipelines can be embedded directly into the streaming fabric, enabling on‑the‑fly feature extraction for materials science and neutron crystallography.
  • A key insight is that secure, role‑based API gateways not only protect sensitive experimental data but also reduce overhead by eliminating redundant authentication hops.
  • The paper highlights that achieving seamless transitions from lab to production HPC requires coordinated tooling, not just high‑bandwidth links, and recommends a modular plug‑in architecture.
  • The authors caution that the proposed framework demands significant infrastructure investment and skilled personnel, underscoring the need for community‑wide tooling standards.
  • Finally, the study calls for collaborative R&D to refine the AI‑coupled workflow model, suggesting that shared benchmarks could accelerate adoption across scientific domains.
Managing tech teams
👍 👎 ♥ Save
Abstract
Teams now drive most scientific advances, yet the impact of absolute beginners -- authors with no prior publications -- remains understudied. Analyzing over 28 million articles published between 1971 and 2020 across disciplines and team sizes, we uncover a universal and previously undocumented pattern: teams with a higher fraction of beginners are systematically more disruptive and innovative. Their contributions are linked to distinct knowledge-integration behaviors, including drawing on broader and less canonical prior work and producing more atypical recombinations. Collaboration structure further shapes outcomes: disruption is high when beginners work with early-career colleagues or with co-authors who have disruptive track records. Although disruption and citations are negatively correlated overall, highly disruptive papers from beginner-heavy teams are highly cited. These findings reveal a "beginner's charm" in science, highlighting the underrecognized yet powerful value of beginner fractions in teams and suggesting actionable strategies for fostering a thriving ecosystem of innovation in science and technology.
👍 👎 ♥ Save
Politecnico di Torino, Un
Abstract
Software Engineering is mostly a male-dominated sector, where gender diversity is a key feature for improving equality of opportunities, productivity, and innovation. Other diversity aspects, including but not limited to nationality and ethnicity, are often understudied.In this work we aim to assess the impact of team diversity, focusing mainly on gender and nationality, in the context of an agile software development project-based course. We analyzed 51 teams over three academic years, measuring three different Diversity indexes - regarding Gender, Nationality and their co-presence - to examine how different aspects of diversity impact the quality of team project outcomes.Statistical analysis revealed a moderate, statistically significant correlation between gender diversity and project success, aligning with existing literature. Diversity in nationality showed a negative but negligible effect on project results, indicating that promoting these aspects does not harm students' performance. Analyzing their co-presence within a team, gender and nationality combined had a negative impact, likely due to increased communication barriers and differing cultural norms.This study underscores the importance of considering multiple diversity dimensions and their interactions in educational settings. Our findings, overall, show that promoting diversity in teams does not negatively impact their performance and achievement of educational goals.
AI for Data Science Management
👍 👎 ♥ Save
Google DeepMind and other
Abstract
The cycle of scientific discovery is frequently bottlenecked by the slow, manual creation of software to support computational experiments. To address this, we present an AI system that creates expert-level scientific software whose goal is to maximize a quality metric. The system uses a Large Language Model (LLM) and Tree Search (TS) to systematically improve the quality metric and intelligently navigate the large space of possible solutions. The system achieves expert-level results when it explores and integrates complex research ideas from external sources. The effectiveness of tree search is demonstrated across a wide range of benchmarks. In bioinformatics, it discovered 40 novel methods for single-cell data analysis that outperformed the top human-developed methods on a public leaderboard. In epidemiology, it generated 14 models that outperformed the CDC ensemble and all other individual models for forecasting COVID-19 hospitalizations. Our method also produced state-of-the-art software for geospatial analysis, neural activity prediction in zebrafish, time series forecasting and numerical solution of integrals. By devising and implementing novel solutions to diverse tasks, the system represents a significant step towards accelerating scientific progress.
AI Insights
  • Integration forbids cell_type usage, forcing reliance on scanpy, sklearn, numpy, scipy, tensorflow, torch, or jax.
  • Evaluation spans ASW Batch, ASW Label, ARI, NMI, kBET, iLISI, and PCR for batch‑integration quality.
  • Scalable batch removal is achieved by blending CCA, MNN, and BBKNN insights into a unified, GPU‑friendly pipeline.
  • The system’s tree search explores millions of code variants, pruning by a quality metric that balances accuracy and computational cost.
  • Future work could integrate JAX‑accelerated differentiable programming to learn batch‑removal parameters end‑to‑end.
  • A key challenge is handling datasets with many batches or complex technical noise without exploding memory usage.
👍 👎 ♥ Save
Hugging Face
Abstract
Artificial intelligence promises to accelerate scientific discovery, yet its benefits remain unevenly distributed. While technical obstacles such as scarce data, fragmented standards, and unequal access to computation are significant, we argue that the primary barriers are social and institutional. Narratives that defer progress to speculative "AI scientists," the undervaluing of data and infrastructure contributions, misaligned incentives, and gaps between domain experts and machine learning researchers all constrain impact. We highlight four interconnected challenges: community dysfunction, research priorities misaligned with upstream needs, data fragmentation, and infrastructure inequities. We argue that their roots lie in cultural and organizational practices. Addressing them requires not only technical innovation but also intentional community-building, cross-disciplinary education, shared benchmarks, and accessible infrastructure. We call for reframing AI for science as a collective social project, where sustainable collaboration and equitable participation are treated as prerequisites for technical progress.
AI Insights
  • Democratizing advanced cyberinfrastructure unlocks responsible AI research across global labs.
  • Only 5 % of Africa’s AI talent accesses sufficient compute, underscoring regional inequity.
  • Pre‑trained transformer models now generate multi‑omics, multi‑species, multi‑tissue samples.
  • Quantization‑aware training yields efficient neural PDE‑solvers showcased at recent conferences.
  • The FAIR Guiding Principles guide scientific data stewardship, enhancing reproducibility.
  • MAGE‑Tab’s spreadsheet‑based format standardizes microarray data for seamless sharing.
  • Resources like The Human Cell Atlas and pymatgen empower interdisciplinary material‑genomics research.
AI for Data Science Engineering
👍 👎 ♥ Save
Deakin University
Abstract
Organizations and educational institutions use time-bound assessment tasks to evaluate coding and problem-solving skills. These assessments measure not only the correctness of the solutions, but also their efficiency. Problem setters (educator/interviewer) are responsible for crafting these challenges, carefully balancing difficulty and relevance to create meaningful evaluation experiences. Conversely, problem solvers (student/interviewee) apply coding efficiency and logical thinking to arrive at correct solutions. In the era of Large Language Models (LLMs), LLMs assist problem setters in generating diverse and challenging questions, but they can undermine assessment integrity for problem solvers by providing easy access to solutions. This paper introduces OpenCoderRank, an easy-to-use platform designed to simulate technical assessments. It acts as a bridge between problem setters and problem solvers, helping solvers prepare for time constraints and unfamiliar problems while allowing setters to self-host assessments, offering a no-cost and customizable solution for technical assessments in resource-constrained environments.
AI Insights
  • OpenCoderRank runs on Flask with SQLite, enabling rapid deployment on modest hardware.
  • It auto‑judges MCQs and executes user code in isolated containers for SQL, Python, and Java.
  • Full‑screen mode, copy‑paste blocking, and random ordering protect against LLM‑assisted cheating.
  • Educators embed it in LMS to run timed drills that mirror real interview challenges.
  • Ideal for intra‑college contests, peer learning, and startup hiring tests, all free of charge.
  • Key reading: Bhushan et al. 2025 on LLM‑answer detection and Desmond et al. 2025 on LLM‑as‑a‑judge.

Interests not found

We did not find any papers that match the below interests. Try other terms also consider if the content exists in arxiv.org.
  • Engineering Management
  • Managing teams of data scientists
  • Data Science Engineering Management
  • Data Science Management
You can edit or add more interests any time.

Unsubscribe from these updates