Stanford University
AI Insights
- The paper presents a framework for evaluating and improving the fairness of binary classification models. (ML: 0.98)
- They demonstrate the effectiveness of their approach on several real-world datasets, including COMPAS, German Credit, and Adult. (ML: 0.98)
- It introduces three new metrics: pointwise balance, global calibration, and signed covariance, which are used to evaluate the fairness of a model. (ML: 0.98)
- The authors also provide a detailed overview of experimental results, which include comparisons with existing fair machine learning methods. (ML: 0.97)
- Their results show that the proposed framework can significantly improve the fairness of binary classification models while maintaining or improving their accuracy. (ML: 0.96)
- The authors also propose a new algorithm for optimizing these metrics, called FairBoost. (ML: 0.94)
- FairBoost: an algorithm for optimizing pointwise balance, global calibration, and signed covariance. (ML: 0.79)
- Signed covariance: Cov(π(Y), r(Y)) ≥ 0 if π and r are monotone in the same direction. (ML: 0.73)
- Global calibration: δC(z) = E[Y|Z=z, G=1]−E[Y|Z=z, G=0]. (ML: 0.72)
- Pointwise balance: δB(y) = E[Z|Y=y, G=1]−E[Z|Y=y, G=0]. (ML: 0.64)
Abstract
We derive an accounting identity for predictive models that links accuracy with common fairness criteria. The identity shows that for globally calibrated models, the weighted sums of miscalibration within groups and error imbalance across groups is equal to a "total unfairness budget." For binary outcomes, this budget is the model's mean-squared error times the difference in group prevalence across outcome classes. The identity nests standard impossibility results as special cases, while also describing inherent tradeoffs when one or more fairness measures are not perfectly satisfied. The results suggest that accuracy and fairness are best viewed as complements in binary prediction tasks: increasing accuracy necessarily shrinks the total unfairness budget and vice-versa. Experiments on benchmark data confirm the theory and show that many fairness interventions largely substitute between fairness violations, and when they reduce accuracy they tend to expand the total unfairness budget. The results extend naturally to prediction tasks with non-binary outcomes, illustrating how additional outcome information can relax fairness incompatibilities and identifying conditions under which the binary-style impossibility does and does not extend to regression tasks.
We derive an accounting identity for predictive models that links accuracy with common fairness criteria. The identity shows that for globally calibrated models, the weighted sums of miscalibration within groups and error imbalance across groups is equal to a "total unfairness budget." For binary outcomes, this budget is the model's mean-squared error times the difference in group prevalence across outcome classes. The identity nests standard impossibility results as special cases, while also describing inherent tradeoffs when one or more fairness measures are not perfectly satisfied. The results suggest that accuracy and fairness are best viewed as complements in binary prediction tasks: increasing accuracy necessarily shrinks the total unfairness budget and vice-versa. Experiments on benchmark data confirm the theory and show that many fairness interventions largely substitute between fairness violations, and when they reduce accuracy they tend to expand the total unfairness budget. The results extend naturally to prediction tasks with non-binary outcomes, illustrating how additional outcome information can relax fairness incompatibilities and identifying conditions under which the binary-style impossibility does and does not extend to regression tasks.
Why we are recommending this paper?
Due to your Interest in Data Fairness
This paper from Stanford University offers a crucial theoretical framework for understanding and quantifying fairness in algorithmic models, directly aligning with the user's interest in AI fairness and bias. It provides a concrete identity linking accuracy with fairness criteria, a key concept for addressing bias concerns.
Due to your Interest in Data Fairness
This paper from Stanford University offers a crucial theoretical framework for understanding and quantifying fairness in algorithmic models, directly aligning with the user's interest in AI fairness and bias. It provides a concrete identity linking accuracy with fairness criteria, a key concept for addressing bias concerns.
Czech Technical University in Prague
AI Insights
- (2025) is referenced for the formulation for bias detection using conjunction groups. (ML: 0.94)
- MIO: Mixed-Integer Optimization SPSF: Statistical Parity Score Fairness FPSF: False Positive Rate Fairness MSD: Maximum Statistical Disparity SD: Statistical Disparity N min : The size of a minimal subgroup to be considered The MIO formulations used in the framework are presented. (ML: 0.88)
- The formulations include the detection formulation for bias detection using conjunction groups, and the training formulations. (ML: 0.88)
- The scale of the coefficients can affect the solution process. (ML: 0.87)
- The formulations start with the detection formulation for bias detection using conjunction groups. (ML: 0.86)
- N ˇemeˇceket al. (ML: 0.84)
- The MIO formulations used in the framework are presented in their entirety. (ML: 0.63)
Abstract
The deployment of Artificial Intelligence in high-risk domains, such as finance and healthcare, necessitates models that are both fair and transparent. While regulatory frameworks, including the EU's AI Act, mandate bias mitigation, they are deliberately vague about the definition of bias. In line with existing research, we argue that true fairness requires addressing bias at the intersections of protected groups. We propose a unified framework that leverages Mixed-Integer Optimization (MIO) to train intersectionally fair and intrinsically interpretable classifiers. We prove the equivalence of two measures of intersectional fairness (MSD and SPSF) in detecting the most unfair subgroup and empirically demonstrate that our MIO-based algorithm improves performance in finding bias. We train high-performing, interpretable classifiers that bound intersectional bias below an acceptable threshold, offering a robust solution for regulated industries and beyond.
The deployment of Artificial Intelligence in high-risk domains, such as finance and healthcare, necessitates models that are both fair and transparent. While regulatory frameworks, including the EU's AI Act, mandate bias mitigation, they are deliberately vague about the definition of bias. In line with existing research, we argue that true fairness requires addressing bias at the intersections of protected groups. We propose a unified framework that leverages Mixed-Integer Optimization (MIO) to train intersectionally fair and intrinsically interpretable classifiers. We prove the equivalence of two measures of intersectional fairness (MSD and SPSF) in detecting the most unfair subgroup and empirically demonstrate that our MIO-based algorithm improves performance in finding bias. We train high-performing, interpretable classifiers that bound intersectional bias below an acceptable threshold, offering a robust solution for regulated industries and beyond.
Why we are recommending this paper?
Due to your Interest in Data Fairness
Given the user's focus on intersectional fairness, this paper from the Czech Technical University in Prague presents a novel optimization approach. The method directly tackles fairness challenges across multiple groups, offering a practical solution for addressing complex bias scenarios.
Due to your Interest in Data Fairness
Given the user's focus on intersectional fairness, this paper from the Czech Technical University in Prague presents a novel optimization approach. The method directly tackles fairness challenges across multiple groups, offering a practical solution for addressing complex bias scenarios.
Northeastern University
AI Insights
- Participants expressed concerns about becoming 'AI babysitters' or 'button-pushers,' losing their clinical intuition and autonomy. (ML: 0.98)
- Participants expressed concerns about becoming 'AI babysitters' or 'button-pushers,' losing their clinical intuition and autonomy. (ML: 0.98)
- The study's focus on a single field (radiation oncology) may limit its generalizability to other healthcare fields. (ML: 0.97)
- The study highlights the need for designers and policymakers to consider the human impact of automation in healthcare. (ML: 0.97)
- The study suggests that the use of AI in healthcare can lead to deprofessionalization, where skilled roles lose their distinctiveness and authority. (ML: 0.97)
- The study suggests that the use of AI in healthcare can have unintended consequences, including erosion of professional identity and autonomy. (ML: 0.96)
- The study highlights the risks of automation in radiation oncology, including deskilling, dependency on AI, and erosion of professional identity. (ML: 0.95)
- Deskilling: The gradual loss of sharpness in a skill due to disuse. (ML: 0.95)
- Chronic Harm: Sustained, accumulating negative impacts that manifest over time, akin to a chronic condition. (ML: 0.93)
- Identity Commoditization: The fear (or reality) that a professional's role is being reduced to a commodity, eroding their sense of identity, uniqueness, and dignity at work. (ML: 0.92)
Abstract
In the future of work discourse, AI is touted as the ultimate productivity amplifier. Yet, beneath the efficiency gains lie subtle erosions of human expertise and agency. This paper shifts focus from the future of work to the future of workers by navigating the AI-as-Amplifier Paradox: AI's dual role as enhancer and eroder, simultaneously strengthening performance while eroding underlying expertise. We present a year-long study on the longitudinal use of AI in a high-stakes workplace among cancer specialists. Initial operational gains hid ``intuition rust'': the gradual dulling of expert judgment. These asymptomatic effects evolved into chronic harms, such as skill atrophy and identity commoditization. Building on these findings, we offer a framework for dignified Human-AI interaction co-constructed with professional knowledge workers facing AI-induced skill erosion without traditional labor protections. The framework operationalizes sociotechnical immunity through dual-purpose mechanisms that serve institutional quality goals while building worker power to detect, contain, and recover from skill erosion, and preserve human identity. Evaluated across healthcare and software engineering, our work takes a foundational step toward dignified human-AI interaction futures by balancing productivity with the preservation of human expertise.
In the future of work discourse, AI is touted as the ultimate productivity amplifier. Yet, beneath the efficiency gains lie subtle erosions of human expertise and agency. This paper shifts focus from the future of work to the future of workers by navigating the AI-as-Amplifier Paradox: AI's dual role as enhancer and eroder, simultaneously strengthening performance while eroding underlying expertise. We present a year-long study on the longitudinal use of AI in a high-stakes workplace among cancer specialists. Initial operational gains hid ``intuition rust'': the gradual dulling of expert judgment. These asymptomatic effects evolved into chronic harms, such as skill atrophy and identity commoditization. Building on these findings, we offer a framework for dignified Human-AI interaction co-constructed with professional knowledge workers facing AI-induced skill erosion without traditional labor protections. The framework operationalizes sociotechnical immunity through dual-purpose mechanisms that serve institutional quality goals while building worker power to detect, contain, and recover from skill erosion, and preserve human identity. Evaluated across healthcare and software engineering, our work takes a foundational step toward dignified human-AI interaction futures by balancing productivity with the preservation of human expertise.
Why we are recommending this paper?
Due to your Interest in AI Ethics
This paper from Northeastern University directly addresses the ethical implications of AI's impact on human work, aligning with the user's interest in AI ethics and the broader societal consequences of AI deployment. It highlights potential harms beyond simple bias, focusing on human agency and dignity.
Due to your Interest in AI Ethics
This paper from Northeastern University directly addresses the ethical implications of AI's impact on human work, aligning with the user's interest in AI ethics and the broader societal consequences of AI deployment. It highlights potential harms beyond simple bias, focusing on human agency and dignity.
University of Tartu
AI Insights
- It emphasizes the need for a more comprehensive understanding of the complex relationships between technological, social, and economic factors in AI development and deployment. (ML: 0.99)
- It cites studies on AI bias, transparency, accountability, and the need for responsible AI development and deployment. (ML: 0.99)
- The study acknowledges that it has limitations due to the complexity of the topic and the need for further research. (ML: 0.98)
- It also recognizes that the development of AI governance frameworks is a dynamic process that requires ongoing evaluation and refinement. (ML: 0.98)
- The study highlights the importance of considering paradoxes when developing AI governance frameworks. (ML: 0.97)
- The paper explores these kinds of complexities in AI governance and why they need to be considered when developing frameworks for responsible AI development and deployment. (ML: 0.97)
- The paper explores the concept of paradox in the context of artificial intelligence (AI) and its governance, highlighting the importance of considering these complexities when developing AI governance frameworks. (ML: 0.97)
- The paper discusses the concept of paradox in management and organization theories, specifically focusing on artificial intelligence (AI) and its governance. (ML: 0.96)
- The paper draws on existing literature in management, organization theory, and AI ethics to inform its discussion of paradoxes in AI governance. (ML: 0.95)
- Imagine you're trying to create a system that can make decisions without being biased, but at the same time, you want it to be transparent so people understand how those decisions are made. (ML: 0.93)
- That's a paradox! (ML: 0.91)
- Paradox: a situation or condition that is contradictory or opposite to what would be expected. (ML: 0.86)
Abstract
The rapid proliferation of artificial intelligence across organizational contexts has generated profound strategic opportunities while introducing significant ethical and operational risks. Despite growing scholarly attention to responsible AI, extant literature remains fragmented and is often adopting either an optimistic stance emphasizing value creation or an excessively cautious perspective fixated on potential harms. This paper addresses this gap by presenting a comprehensive examination of AI's dual nature through the lens of strategic information systems. Drawing upon a systematic synthesis of the responsible AI literature and grounded in paradox theory, we develop the Paradox-based Responsible AI Governance (PRAIG) framework that articulates: (1) the strategic benefits of AI adoption, (2) the inherent risks and unintended consequences, and (3) governance mechanisms that enable organizations to navigate these tensions. Our framework advances theoretical understanding by conceptualizing responsible AI governance as the dynamic management of paradoxical tensions between value creation and risk mitigation. We provide formal propositions demonstrating that trade-off approaches amplify rather than resolve these tensions, and we develop a taxonomy of paradox management strategies with specified contingency conditions. For practitioners, we offer actionable guidance for developing governance structures that neither stifle innovation nor expose organizations to unacceptable risks. The paper concludes with a research agenda for advancing responsible AI governance scholarship.
The rapid proliferation of artificial intelligence across organizational contexts has generated profound strategic opportunities while introducing significant ethical and operational risks. Despite growing scholarly attention to responsible AI, extant literature remains fragmented and is often adopting either an optimistic stance emphasizing value creation or an excessively cautious perspective fixated on potential harms. This paper addresses this gap by presenting a comprehensive examination of AI's dual nature through the lens of strategic information systems. Drawing upon a systematic synthesis of the responsible AI literature and grounded in paradox theory, we develop the Paradox-based Responsible AI Governance (PRAIG) framework that articulates: (1) the strategic benefits of AI adoption, (2) the inherent risks and unintended consequences, and (3) governance mechanisms that enable organizations to navigate these tensions. Our framework advances theoretical understanding by conceptualizing responsible AI governance as the dynamic management of paradoxical tensions between value creation and risk mitigation. We provide formal propositions demonstrating that trade-off approaches amplify rather than resolve these tensions, and we develop a taxonomy of paradox management strategies with specified contingency conditions. For practitioners, we offer actionable guidance for developing governance structures that neither stifle innovation nor expose organizations to unacceptable risks. The paper concludes with a research agenda for advancing responsible AI governance scholarship.
Why we are recommending this paper?
Due to your Interest in AI Bias
Coming from the University of Tartu, this paper tackles the broad landscape of responsible AI, directly addressing the ethical and operational risks associated with AI proliferation – a core interest for the user. It provides a valuable overview of the challenges within the field.
Due to your Interest in AI Bias
Coming from the University of Tartu, this paper tackles the broad landscape of responsible AI, directly addressing the ethical and operational risks associated with AI proliferation – a core interest for the user. It provides a valuable overview of the challenges within the field.
TeMLM Foundation
AI Insights
- TeMLM-Card: A concise summary of the key aspects of a medical language model, including its purpose, methodology, and results. (ML: 0.98)
- TeMLM-Provenance: A record of the steps taken to develop and evaluate a medical language model, including data sources, preprocessing methods, and evaluation metrics. (ML: 0.97)
- The Transparency pillar of TeMLM provides a framework for making clinical NLP research more auditable, reusable, and trustworthy. (ML: 0.97)
- The authors argue that 'transparent enough' should be treated as a verifiable claim, and that documentation, provenance, and audit metrics together form a portable evidence bundle for clinical NLP. (ML: 0.97)
- TeMLM-Datasheet: A structured document that provides information about the development and evaluation of a medical language model. (ML: 0.96)
- The paper presents the Transparency pillar of TeMLM, a framework for medical language models that aims to make clinical NLP research more auditable, reusable, and trustworthy. (ML: 0.95)
- The authors introduce three artifacts: TeMLM-Datasheet, TeMLM-Card, and TeMLM-Provenance, which provide a structured way to document and report on the development and evaluation of medical language models. (ML: 0.94)
- The three artifacts introduced in this paper (TeMLM-Datasheet, TeMLM-Card, and TeMLM-Provenance) provide a structured way to document and report on the development and evaluation of medical language models. (ML: 0.94)
- Minimal transparency metric suite: A set of metrics that can be used to evaluate the transparency of a medical language model, including documentation, provenance, and audit metrics. (ML: 0.94)
- The paper also presents a minimal transparency metric suite and release thresholds that can be used to evaluate the transparency of medical language models. (ML: 0.89)
Abstract
We introduce TeMLM, a set of transparency-first release artifacts for clinical language models. TeMLM unifies provenance, data transparency, modeling transparency, and governance into a single, machine-checkable release bundle. We define an artifact suite (TeMLM-Card, TeMLM-Datasheet, TeMLM-Provenance) and a lightweight conformance checklist for repeatable auditing. We instantiate the artifacts on Technetium-I, a large-scale synthetic clinical NLP dataset with 498,000 notes, 7.74M PHI entity annotations across 10 types, and ICD-9-CM diagnosis labels, and report reference results for ProtactiniumBERT (about 100 million parameters) on PHI de-identification (token classification) and top-50 ICD-9 code extraction (multi-label classification). We emphasize that synthetic benchmarks are valuable for tooling and process validation, but models should be validated on real clinical data prior to deployment.
We introduce TeMLM, a set of transparency-first release artifacts for clinical language models. TeMLM unifies provenance, data transparency, modeling transparency, and governance into a single, machine-checkable release bundle. We define an artifact suite (TeMLM-Card, TeMLM-Datasheet, TeMLM-Provenance) and a lightweight conformance checklist for repeatable auditing. We instantiate the artifacts on Technetium-I, a large-scale synthetic clinical NLP dataset with 498,000 notes, 7.74M PHI entity annotations across 10 types, and ICD-9-CM diagnosis labels, and report reference results for ProtactiniumBERT (about 100 million parameters) on PHI de-identification (token classification) and top-50 ICD-9 code extraction (multi-label classification). We emphasize that synthetic benchmarks are valuable for tooling and process validation, but models should be validated on real clinical data prior to deployment.
Why we are recommending this paper?
Due to your Interest in Data Transparency
This paper from TeMLM Foundation focuses on transparency in medical language models, a critical area for the user's interest in AI transparency and data ethics. The emphasis on data provenance and model cards directly addresses concerns about model accountability and trustworthiness.
Due to your Interest in Data Transparency
This paper from TeMLM Foundation focuses on transparency in medical language models, a critical area for the user's interest in AI transparency and data ethics. The emphasis on data provenance and model cards directly addresses concerns about model accountability and trustworthiness.
University of Oxford
AI Insights
- Utility can be defined according to expected distortion or mutual information. (ML: 0.96)
- Expected Distortion: A measure of how much the released data deviates from the original data. (ML: 0.95)
- Progressive Neural Networks (PNNs): A type of neural network that provides responses to sequential tasks by creating a new neural network for each task. (ML: 0.94)
- Mutual Information: A measure of the amount of information that one variable contains about another variable. (ML: 0.93)
- The problem formulation handles multiple sequential data requests while balancing privacy and utility, accounting for the possibility of collusion. (ML: 0.85)
- Collusion: The act of multiple parties working together to obtain sensitive information. (ML: 0.84)
- The proposed problem formulation and procedure provide a framework for handling multiple sequential data requests while balancing privacy and utility, accounting for the possibility of collusion. (ML: 0.84)
- A procedure is outlined to adaptively release data and trace the three-dimensional distortion-privacy-collusion curve D k(ϵk, δk) or information-privacy-collusion curve Ik(ϵ, δ). (ML: 0.71)
- A numerical example demonstrates the feasibility of the procedure and confirms the convergence of the Blahut-Arimoto algorithm for the expected distortion problem. (ML: 0.59)
- The Blahut-Arimoto algorithm ensures convergence to the appropriate value. (ML: 0.58)
Abstract
The fundamental trade-off between privacy and utility remains an active area of research. Our contribution is motivated by two observations. First, privacy mechanisms developed for one-time data release cannot straightforwardly be extended to sequential releases. Second, practical databases are likely to be useful to multiple distinct parties. Furthermore, we can not rule out the possibility of data sharing between parties. With utility in mind, we formulate a privacy-utility trade-off problem to adaptively tackle sequential data requests made by different, potentially colluding entities. We consider both expected distortion and mutual information as measures to quantify utility, and use mutual information to measure privacy. We assume an attack model whereby illicit data sharing, which we call collusion, can occur between data receivers. We develop an adaptive algorithm for data releases that makes use of a modified Blahut-Arimoto algorithm. We show that the resulting data releases are optimal when expected distortion quantifies utility, and locally optimal when mutual information quantifies utility. Finally, we discuss how our findings may extend to applications in machine learning.
The fundamental trade-off between privacy and utility remains an active area of research. Our contribution is motivated by two observations. First, privacy mechanisms developed for one-time data release cannot straightforwardly be extended to sequential releases. Second, practical databases are likely to be useful to multiple distinct parties. Furthermore, we can not rule out the possibility of data sharing between parties. With utility in mind, we formulate a privacy-utility trade-off problem to adaptively tackle sequential data requests made by different, potentially colluding entities. We consider both expected distortion and mutual information as measures to quantify utility, and use mutual information to measure privacy. We assume an attack model whereby illicit data sharing, which we call collusion, can occur between data receivers. We develop an adaptive algorithm for data releases that makes use of a modified Blahut-Arimoto algorithm. We show that the resulting data releases are optimal when expected distortion quantifies utility, and locally optimal when mutual information quantifies utility. Finally, we discuss how our findings may extend to applications in machine learning.
Why we are recommending this paper?
Due to your Interest in Data Ethics
Due to your Interest in Data Ethics
Paderborn University
AI Insights
- They struggle with understanding what data to report, how to report it, and what the consequences will be if they get it wrong. (ML: 0.99)
- Our work extends this line of inquiry by exploring the challenges developers face in reporting data collection and analyzing their confidence levels. (ML: 0.98)
- The current tools that help them fill out this form may not be effective in addressing these challenges. (ML: 0.98)
- Developers face challenges when completing the DSS forms, including issues with data collection and reporting, limited understanding of the form, and concerns about app rejection. (ML: 0.97)
- The current DSS-support tools may not be effective in addressing these challenges. (ML: 0.96)
- Limited insight into data gathered by third-party SDKs Vague definitions of ephemeral data processing Ambiguity in data type categories Unclear account creation criteria involving third-party SDKs Multiple studies have examined the accuracy of DSS submissions, but none specifically investigate how developers report data collection. (ML: 0.96)
- Developers face many challenges when completing the DSS forms, including issues in identifying privacy-relevant data to complete the form, limited understanding of the form, and concerns about app rejection due to discrepancies with Google's privacy requirements. (ML: 0.94)
- Future work should systematically evaluate these tools to understand why they are underutilized and how they can be improved to better support app developers. (ML: 0.94)
- DSS: Data Safety Declaration (a form that developers must fill out to declare what data their apps collect) The current DSS-support tools may fall short in terms of accuracy, usability, or accessibility. (ML: 0.90)
- Developers are having trouble filling out a form called the Data Safety Declaration (DSS) which is required to declare what data their apps collect. (ML: 0.85)
Abstract
Current legal frameworks enforce that Android developers accurately report the data their apps collect. However, large codebases can make this reporting challenging. This paper employs an empirical approach to understand developers' experience with Google Play Store's Data Safety Section (DSS) form. We first survey 41 Android developers to understand how they categorize privacy-related data into DSS categories and how confident they feel when completing the DSS form. To gain a broader and more detailed view of the challenges developers encounter during the process, we complement the survey with an analysis of 172 online developer discussions, capturing the perspectives of 642 additional developers. Together, these two data sources represent insights from 683 developers. Our findings reveal that developers often manually classify the privacy-related data their apps collect into the data categories defined by Google-or, in some cases, omit classification entirely-and rely heavily on existing online resources when completing the form. Moreover, developers are generally confident in recognizing the data their apps collect, yet they lack confidence in translating this knowledge into DSS-compliant disclosures. Key challenges include issues in identifying privacy-relevant data to complete the form, limited understanding of the form, and concerns about app rejection due to discrepancies with Google's privacy requirements. These results underscore the need for clearer guidance and more accessible tooling to support developers in meeting privacy-aware reporting obligations.
Current legal frameworks enforce that Android developers accurately report the data their apps collect. However, large codebases can make this reporting challenging. This paper employs an empirical approach to understand developers' experience with Google Play Store's Data Safety Section (DSS) form. We first survey 41 Android developers to understand how they categorize privacy-related data into DSS categories and how confident they feel when completing the DSS form. To gain a broader and more detailed view of the challenges developers encounter during the process, we complement the survey with an analysis of 172 online developer discussions, capturing the perspectives of 642 additional developers. Together, these two data sources represent insights from 683 developers. Our findings reveal that developers often manually classify the privacy-related data their apps collect into the data categories defined by Google-or, in some cases, omit classification entirely-and rely heavily on existing online resources when completing the form. Moreover, developers are generally confident in recognizing the data their apps collect, yet they lack confidence in translating this knowledge into DSS-compliant disclosures. Key challenges include issues in identifying privacy-relevant data to complete the form, limited understanding of the form, and concerns about app rejection due to discrepancies with Google's privacy requirements. These results underscore the need for clearer guidance and more accessible tooling to support developers in meeting privacy-aware reporting obligations.
Why we are recommending this paper?
Due to your Interest in Data Ethics
Due to your Interest in Data Ethics
Anthropic
AI Insights
- The tasks were designed to be straightforward, but participants still encountered various errors, which may not reflect real-world coding scenarios. (ML: 0.99)
- The study only recruited participants who have used AI assistants before, which may limit the generalizability of the findings. (ML: 0.99)
- The study suggests that relying on AI for coding assistance may not necessarily lead to improved productivity or skill formation, especially if participants spend significant time interacting with the AI assistant. (ML: 0.99)
- The way participants adopted AI advice, whether by pasting or manually copying code, had a significant impact on task completion time and quiz score. (ML: 0.98)
- Pasting vs Manual Code Copying: Participants who directly pasted AI-written code versus those who manually typed in (copied) the AI generated code. (ML: 0.98)
- Participants who directly pasted AI-written code finished tasks faster than those who manually copied or used a hybrid method. (ML: 0.98)
- There was no notable difference between groups that typed vs directly pasted AI output for skill formation, measured by quiz score. (ML: 0.98)
- The study found that participants in the AI condition spent significant time interacting with the AI assistant, which did not translate to improved productivity. (ML: 0.96)
- Queries: User inputs into the AI assistant, categorized into 5 broad categories: explanation, generation, debugging, capabilities questions, and appreciation. (ML: 0.95)
- AI Interaction Time: The time spent interacting with the AI assistant, including typing and thinking about what to type. (ML: 0.92)
Abstract
AI assistance produces significant productivity gains across professional domains, particularly for novice workers. Yet how this assistance affects the development of skills required to effectively supervise AI remains unclear. Novice workers who rely heavily on AI to complete unfamiliar tasks may compromise their own skill acquisition in the process. We conduct randomized experiments to study how developers gained mastery of a new asynchronous programming library with and without the assistance of AI. We find that AI use impairs conceptual understanding, code reading, and debugging abilities, without delivering significant efficiency gains on average. Participants who fully delegated coding tasks showed some productivity improvements, but at the cost of learning the library. We identify six distinct AI interaction patterns, three of which involve cognitive engagement and preserve learning outcomes even when participants receive AI assistance. Our findings suggest that AI-enhanced productivity is not a shortcut to competence and AI assistance should be carefully adopted into workflows to preserve skill formation -- particularly in safety-critical domains.
AI assistance produces significant productivity gains across professional domains, particularly for novice workers. Yet how this assistance affects the development of skills required to effectively supervise AI remains unclear. Novice workers who rely heavily on AI to complete unfamiliar tasks may compromise their own skill acquisition in the process. We conduct randomized experiments to study how developers gained mastery of a new asynchronous programming library with and without the assistance of AI. We find that AI use impairs conceptual understanding, code reading, and debugging abilities, without delivering significant efficiency gains on average. Participants who fully delegated coding tasks showed some productivity improvements, but at the cost of learning the library. We identify six distinct AI interaction patterns, three of which involve cognitive engagement and preserve learning outcomes even when participants receive AI assistance. Our findings suggest that AI-enhanced productivity is not a shortcut to competence and AI assistance should be carefully adopted into workflows to preserve skill formation -- particularly in safety-critical domains.
Why we are recommending this paper?
Due to your Interest in AI Bias
Due to your Interest in AI Bias
Shanghai Jiao Tong University
AI Insights
- The paper has several limitations, including the use of a small dataset for training and evaluating the defender agent, the lack of consideration for the potential impact of the defender agent on the performance of the large language model, and the need for further research to improve the robustness and adaptability of the defense mechanism. (ML: 0.90)
- The paper has several limitations, including the use of a small dataset for training and evaluating the defender agent, the lack of consideration for the potential impact of the defender agent on the performance of the large language model, and the need for further research to improve the robustness and adaptability of the defense mechanism. (ML: 0.90)
- They also compare their approach to several baseline methods and show that it outperforms them in terms of defense success rate and service regression test success rate. (ML: 0.88)
- The use of a small dataset for training and evaluating the defender agent. (ML: 0.86)
- The paper presents several key contributions, including a novel multi-agent framework for defending large language models against jailbreak attacks, a new dataset for evaluating the effectiveness of defense mechanisms, and a set of metrics for evaluating the performance of defense mechanisms. (ML: 0.85)
- DSR (Defense Success Rate): The overall ratio of cases where the defense mechanism is successful in preventing an attack. (ML: 0.83)
- The paper discusses a novel approach to defending large language models against jailbreak attacks using a multi-agent framework. (ML: 0.81)
- FDSR (Failed Defense Success Rate): The total number of failed defenses divided by the total number of attacks. (ML: 0.81)
- The paper presents a novel approach to defending large language models against jailbreak attacks using a multi-agent framework. (ML: 0.81)
- The attacker agent is designed to mimic the behavior of a human attacker, while the defender agent uses a combination of machine learning and rule-based systems to detect and prevent attacks. (ML: 0.81)
- TDSR (Total Defense Success Rate): The total number of successful defenses divided by the total number of attacks. (ML: 0.70)
- The authors propose a system that consists of two main components: an attacker agent and a defender agent. (ML: 0.69)
- The authors demonstrate that their approach can effectively defend against jailbreak attacks and outperform several baseline methods in terms of defense success rate and service regression test success rate. (ML: 0.58)
- The authors evaluate their approach using a range of experiments and demonstrate that it can effectively defend against jailbreak attacks. (ML: 0.53)
Abstract
The dual offensive and defensive utility of Large Language Models (LLMs) highlights a critical gap in AI security: the lack of unified frameworks for dynamic, iterative adversarial adaptation hardening. To bridge this gap, we propose the Red Team vs. Blue Team (RvB) framework, formulated as a training-free, sequential, imperfect-information game. In this process, the Red Team exposes vulnerabilities, driving the Blue Team to learning effective solutions without parameter updates. We validate our framework across two challenging domains: dynamic code hardening against CVEs and guardrail optimization against jailbreaks. Our empirical results show that this interaction compels the Blue Team to learn fundamental defensive principles, leading to robust remediations that are not merely overfitted to specific exploits. RvB achieves Defense Success Rates of 90\% and 45\% across the respective tasks while maintaining near 0\% False Positive Rates, significantly surpassing baselines. This work establishes the iterative adversarial interaction framework as a practical paradigm that automates the continuous hardening of AI systems.
The dual offensive and defensive utility of Large Language Models (LLMs) highlights a critical gap in AI security: the lack of unified frameworks for dynamic, iterative adversarial adaptation hardening. To bridge this gap, we propose the Red Team vs. Blue Team (RvB) framework, formulated as a training-free, sequential, imperfect-information game. In this process, the Red Team exposes vulnerabilities, driving the Blue Team to learning effective solutions without parameter updates. We validate our framework across two challenging domains: dynamic code hardening against CVEs and guardrail optimization against jailbreaks. Our empirical results show that this interaction compels the Blue Team to learn fundamental defensive principles, leading to robust remediations that are not merely overfitted to specific exploits. RvB achieves Defense Success Rates of 90\% and 45\% across the respective tasks while maintaining near 0\% False Positive Rates, significantly surpassing baselines. This work establishes the iterative adversarial interaction framework as a practical paradigm that automates the continuous hardening of AI systems.
Why we are recommending this paper?
Due to your Interest in AI Transparency
Due to your Interest in AI Transparency
University of Notre Dame
AI Insights
- Participants may have had prior experience with computer science concepts, which could influence their performance in the study. (ML: 0.99)
- The study suggests that providing multiple representations can support the learning of data structures by BVI individuals, but it is essential to consider individual differences in learning styles and preferences. (ML: 0.98)
- The study found that BVI individuals use multiple representations to understand and reason about data structures in a way that is consistent with sighted individuals, but they may require more time and practice to develop their skills. (ML: 0.98)
- The study aimed to investigate how blind or visually impaired (BVI) individuals use multiple representations to understand and reason about data structures, specifically arrays and binary trees. (ML: 0.96)
- The participants used the tabular representation most frequently for arrays, while the navigable representation was preferred for binary trees. (ML: 0.95)
- Data Structure: A way of organizing and storing data in a computer so that it can be efficiently accessed and modified. (ML: 0.94)
- Limited sample size of 8 participants. (ML: 0.93)
- Binary Tree: A hierarchical structure where each node has at most two children (left child and right child). (ML: 0.85)
- Array: A linear collection of elements, each identified by an index. (ML: 0.82)
- Blind or Visually Impaired (BVI): Individuals who are unable to see or have limited vision. (ML: 0.77)
Abstract
Blind and visually impaired (BVI) computer science students face systematic barriers when learning data structures: current accessibility approaches typically translate diagrams into alternative text, focusing on visual appearance rather than preserving the underlying structure essential for conceptual understanding. More accessible alternatives often do not scale in complexity, cost to produce, or both. Motivated by a recent shift to tools for creating visual diagrams from code, we propose a solution that automatically creates accessible representations from structural information about diagrams. Based on a Wizard-of-Oz study, we derive design requirements for an automated system, Arboretum, that compiles text-based diagram specifications into three synchronized nonvisual formats$\unicode{x2013}$tabular, navigable, and tactile. Our evaluation with BVI users highlights the strength of tactile graphics for complex tasks such as binary search; the benefits of offering multiple, complementary nonvisual representations; and limitations of existing digital navigation patterns for structural reasoning. This work reframes access to data structures by preserving their structural properties. The solution is a practical system to advance accessible CS education.
Blind and visually impaired (BVI) computer science students face systematic barriers when learning data structures: current accessibility approaches typically translate diagrams into alternative text, focusing on visual appearance rather than preserving the underlying structure essential for conceptual understanding. More accessible alternatives often do not scale in complexity, cost to produce, or both. Motivated by a recent shift to tools for creating visual diagrams from code, we propose a solution that automatically creates accessible representations from structural information about diagrams. Based on a Wizard-of-Oz study, we derive design requirements for an automated system, Arboretum, that compiles text-based diagram specifications into three synchronized nonvisual formats$\unicode{x2013}$tabular, navigable, and tactile. Our evaluation with BVI users highlights the strength of tactile graphics for complex tasks such as binary search; the benefits of offering multiple, complementary nonvisual representations; and limitations of existing digital navigation patterns for structural reasoning. This work reframes access to data structures by preserving their structural properties. The solution is a practical system to advance accessible CS education.
Why we are recommending this paper?
Due to your Interest in Data Representation
Due to your Interest in Data Representation
University of Zaragoza
AI Insights
- The concept of usable information provides a framework for understanding how representations can be combined and utilized in different contexts. (ML: 0.99)
- Representational similarity: The similarity between representations in terms of their structure or content. (ML: 0.98)
- The relationship between functional similarity and representational similarity is explored. (ML: 0.98)
- Functional similarity: The similarity in terms of their ability to perform tasks or achieve goals. (ML: 0.98)
- Representational similarity refers to the similarity between representations in terms of their structure or content, while functional similarity refers to the similarity in terms of their ability to perform tasks or achieve goals. (ML: 0.97)
- Markov blankets are discussed as a key component in understanding functional similarity between representations. (ML: 0.96)
- The concept of usable information is introduced, which refers to the information that can be extracted and utilized by a system or model. (ML: 0.95)
- Stitchability: The ability of one representation to be perfectly combined with another representation to achieve the same task-relevant information. (ML: 0.95)
- Usable information: The information that can be extracted and utilized by a system or model. (ML: 0.94)
- Markov blanket: A set of variables that separates two sets of variables, making them conditionally independent. (ML: 0.85)
- A Markov blanket is a set of variables that separates two sets of variables, making them conditionally independent. (ML: 0.84)
- These include Proposition 3.5 (Markov Blanket-Stitchability Equivalence), Corollary 3.6 (Functional Similarity-Stitchability Equivalence), and others. (ML: 0.76)
- A set of mathematical proofs are provided to support the theoretical framework. (ML: 0.66)
Abstract
We present a unified framework for quantifying the similarity between representations through the lens of \textit{usable information}, offering a rigorous theoretical and empirical synthesis across three key dimensions. First, addressing functional similarity, we establish a formal link between stitching performance and conditional mutual information. We further reveal that stitching is inherently asymmetric, demonstrating that robust functional comparison necessitates a bidirectional analysis rather than a unidirectional mapping. Second, concerning representational similarity, we prove that reconstruction-based metrics and standard tools (e.g., CKA, RSA) act as estimators of usable information under specific constraints. Crucially, we show that similarity is relative to the capacity of the predictive family: representations that appear distinct to a rigid observer may be identical to a more expressive one. Third, we demonstrate that representational similarity is sufficient but not necessary for functional similarity. We unify these concepts through a task-granularity hierarchy: similarity on a complex task guarantees similarity on any coarser derivative, establishing representational similarity as the limit of maximum granularity: input reconstruction.
We present a unified framework for quantifying the similarity between representations through the lens of \textit{usable information}, offering a rigorous theoretical and empirical synthesis across three key dimensions. First, addressing functional similarity, we establish a formal link between stitching performance and conditional mutual information. We further reveal that stitching is inherently asymmetric, demonstrating that robust functional comparison necessitates a bidirectional analysis rather than a unidirectional mapping. Second, concerning representational similarity, we prove that reconstruction-based metrics and standard tools (e.g., CKA, RSA) act as estimators of usable information under specific constraints. Crucially, we show that similarity is relative to the capacity of the predictive family: representations that appear distinct to a rigid observer may be identical to a more expressive one. Third, we demonstrate that representational similarity is sufficient but not necessary for functional similarity. We unify these concepts through a task-granularity hierarchy: similarity on a complex task guarantees similarity on any coarser derivative, establishing representational similarity as the limit of maximum granularity: input reconstruction.
Why we are recommending this paper?
Due to your Interest in Data Representation
Due to your Interest in Data Representation
University of Oxford
AI Insights
- The authors propose several potential solutions to mitigate this bias, including using more diverse and representative data, modifying the model architecture, and using techniques such as debiasing and regularization. (ML: 0.99)
- The study found that large language models (LLMs) can be biased towards certain values and moral foundations. (ML: 0.99)
- The study highlights the importance of considering the potential biases and limitations of large language models in applications such as decision-making and policy development. (ML: 0.99)
- They suggest that this bias could have significant implications for the use of LLMs in applications such as decision-making and policy development. (ML: 0.99)
- The authors argue that this bias is not necessarily a result of the data used to train the model, but rather an inherent property of the model itself. (ML: 0.99)
- The study also found that the bias in LLMs can be influenced by factors such as the type of data used to train the model, the architecture of the model, and the optimization algorithm used. (ML: 0.98)
- Direct preference optimization: A method for optimizing a language model to maximize a specific objective function. (ML: 0.97)
- RewardBench: A benchmarking framework for evaluating the performance of reward models. (ML: 0.95)
- Inverse reinforcement learning: A technique for inferring a reward function from observed behavior. (ML: 0.95)
- Q-function: A type of value function that estimates the expected return of an action in a given state. (ML: 0.79)
Abstract
Reward models (RMs) are central to aligning large language models (LLMs) with human values but have received less attention than pre-trained and post-trained LLMs themselves. Because RMs are initialized from LLMs, they inherit representations that shape their behavior, but the nature and extent of this influence remain understudied. In a comprehensive study of 10 leading open-weight RMs using validated psycholinguistic corpora, we show that RMs exhibit significant differences along multiple dimensions of human value as a function of their base model. Using the "Big Two" psychological axes, we show a robust preference of Llama RMs for "agency" and a corresponding robust preference of Gemma RMs for "communion." This phenomenon holds even when the preference data and finetuning process are identical, and we trace it back to the logits of the respective instruction-tuned and pre-trained models. These log-probability differences themselves can be formulated as an implicit RM; we derive usable implicit reward scores and show that they exhibit the very same agency/communion difference. We run experiments training RMs with ablations for preference data source and quantity, which demonstrate that this effect is not only repeatable but surprisingly durable. Despite RMs being designed to represent human preferences, our evidence shows that their outputs are influenced by the pretrained LLMs on which they are based. This work underscores the importance of safety and alignment efforts at the pretraining stage, and makes clear that open-source developers' choice of base model is as much a consideration of values as of performance.
Reward models (RMs) are central to aligning large language models (LLMs) with human values but have received less attention than pre-trained and post-trained LLMs themselves. Because RMs are initialized from LLMs, they inherit representations that shape their behavior, but the nature and extent of this influence remain understudied. In a comprehensive study of 10 leading open-weight RMs using validated psycholinguistic corpora, we show that RMs exhibit significant differences along multiple dimensions of human value as a function of their base model. Using the "Big Two" psychological axes, we show a robust preference of Llama RMs for "agency" and a corresponding robust preference of Gemma RMs for "communion." This phenomenon holds even when the preference data and finetuning process are identical, and we trace it back to the logits of the respective instruction-tuned and pre-trained models. These log-probability differences themselves can be formulated as an implicit RM; we derive usable implicit reward scores and show that they exhibit the very same agency/communion difference. We run experiments training RMs with ablations for preference data source and quantity, which demonstrate that this effect is not only repeatable but surprisingly durable. Despite RMs being designed to represent human preferences, our evidence shows that their outputs are influenced by the pretrained LLMs on which they are based. This work underscores the importance of safety and alignment efforts at the pretraining stage, and makes clear that open-source developers' choice of base model is as much a consideration of values as of performance.
Why we are recommending this paper?
Due to your Interest in Data Bias
Due to your Interest in Data Bias
Vrije Universiteit Brussel
AI Insights
- May not capture the nuances of human reasoning and problem-solving abilities. (ML: 0.99)
- The tools use natural language processing (NLP) to analyze the student's response and provide feedback on its correctness. (ML: 0.98)
- The tool can help identify areas where students need improvement and provide guidance for better problem-solving skills. (ML: 0.98)
- Omni-Judge: An open-source math-evaluation model developed by the authors of Omni-MATH, used to assess the correctness of an answer generated by a language model or student. (ML: 0.96)
- The evaluation process using Omni-Judge provides accurate and reliable feedback on the correctness of mathematical solutions. (ML: 0.95)
- Reasoning tokens: The number of tokens used in the reasoning part of the solution, which is a measure of the complexity and quality of the solution. (ML: 0.95)
- The evaluation process involves comparing the student's answer with a reference answer, taking into account the substance and explanation of the solution. (ML: 0.95)
- Omni-MATH: A platform for creating and solving mathematical problems. (ML: 0.90)
- Omni-MATH and Omni-Judge are tools used for evaluating mathematical problems and solutions. (ML: 0.89)
- Limited to specific types of mathematical problems. (ML: 0.87)
Abstract
Benchmarks are important tools to track progress in the development of Large Language Models (LLMs), yet inaccuracies in datasets and evaluation methods consistently undermine their effectiveness. Here, we present Omni-MATH-2, a manually revised version of the Omni-MATH dataset comprising a clean, exact-answer subset ($n{=}4181$) and a tagged, non-standard subset ($n{=}247$). Each problem was audited to ensure LaTeX compilability, solvability and verifiability, which involved adding missing figures or information, labeling problems requiring a proof, estimation or image, and removing clutter. This process significantly reduces dataset-induced noise, thereby providing a more precise assessment of model performance. The annotated dataset also allows us to evaluate judge-induced noise by comparing GPT-5 mini with the original Omni-Judge, revealing substantial discrepancies between judges on both the clean and tagged problem subsets. Expert annotations reveal that Omni-Judge is wrong in $96.4\%$ of the judge disagreements, indicating its inability to differentiate between models' abilities, even well before saturation of the benchmark occurs. As problems become more challenging, we find that increasingly competent judges become essential in order to prevent judge errors from masking genuine differences between models. Finally, neither judge identifies the present failure modes for the subset of tagged problems, demonstrating that dataset quality and judge reliability are both critical to develop accurate benchmarks of model performance.
Benchmarks are important tools to track progress in the development of Large Language Models (LLMs), yet inaccuracies in datasets and evaluation methods consistently undermine their effectiveness. Here, we present Omni-MATH-2, a manually revised version of the Omni-MATH dataset comprising a clean, exact-answer subset ($n{=}4181$) and a tagged, non-standard subset ($n{=}247$). Each problem was audited to ensure LaTeX compilability, solvability and verifiability, which involved adding missing figures or information, labeling problems requiring a proof, estimation or image, and removing clutter. This process significantly reduces dataset-induced noise, thereby providing a more precise assessment of model performance. The annotated dataset also allows us to evaluate judge-induced noise by comparing GPT-5 mini with the original Omni-Judge, revealing substantial discrepancies between judges on both the clean and tagged problem subsets. Expert annotations reveal that Omni-Judge is wrong in $96.4\%$ of the judge disagreements, indicating its inability to differentiate between models' abilities, even well before saturation of the benchmark occurs. As problems become more challenging, we find that increasingly competent judges become essential in order to prevent judge errors from masking genuine differences between models. Finally, neither judge identifies the present failure modes for the subset of tagged problems, demonstrating that dataset quality and judge reliability are both critical to develop accurate benchmarks of model performance.
Why we are recommending this paper?
Due to your Interest in Data Bias
Due to your Interest in Data Bias
Interests not found
We did not find any papers that match the below interests.
Try other terms also consider if the content exists in arxiv.org.
- AI Fairness
You can edit or add more interests any time.
💬 Help Shape Our Pricing
We're exploring pricing options to make this project sustainable. Take 3 minutes to share what you'd be willing to pay (if anything). Your input guides our future investment.
Help us improve your experience!
This project is on its early stages your feedback can be pivotal on the future of the project.
Let us know what you think about this week's papers and suggestions!
Give Feedback