Integrated Approaches to Testing Data-Centric AI Systems:
Methods, Metrics, and Benchmarks

Call for Papers

You are invited to participate in the 1st International Workshop on Integrated Approaches to Testing Data-Centric AI Systems: Methods, Metrics, and Benchmarks​, to be held as part of the Artificial Intelligence x Software Engineering (aka AIxSoftware), October 1-3, 2025, The Hills Hotel, Laguna Hills, California (AIxSE).

Purpose of the Workshop

Traditional software testing techniques are insufficient for validating AI systems, which behave probabilistically and rely heavily on data quality. Moreover, most AI testing research has focused on model performance, often overlooking systematic test design, adequacy, and validation frameworks. This workshop seeks to bridge that gap by fostering discussion around experimental methods, case studies, tools, and benchmark creation to support comprehensive AI system testing. Bringing together researchers from software engineering, machine learning, data quality, and systems communities, this workshop will provide a dedicated venue for interdisciplinary collaboration and new perspectives on testing data centric AI.

This workshop aims to explore the integration of data quality evaluation with traditional software testing methodologies to improve the reliability, robustness, and trustworthiness of data-centric AI systems. As AI models become central to decision-making in high-stakes domains, there is a growing need to adapt and extend conventional testing frameworks to account for data-driven behaviors, non-determinism, and continuous learning processes. Key topics include test adequacy criteria for AI systems, evaluation of input data quality, robustness testing, explainability under test, benchmark datasets and tools for AI testing, and the emerging area of self-testing AI systems, systems capable of evaluating and improving their own reliability.

Topics of Interest (include but not limited to)

• º Data quality and test adequacy criteria for AI models.
• º Robustness and reliability testing for AI systems.
• º Evaluating Prompts for Effectiveness and Sensitivity in Large Language Models (LLMs).
• º Tools and frameworks for AI validation and debugging.
• º Benchmarks and datasets for AI testing.
• º Explainability and interpretability under test conditions.
• º Testing Large Language Models (LLMs) and generative AI applications.
• º Self-testing or self-evaluating AI systems.
• º Experimental and empirical studies on AI assurance.
• º Case studies and best practices in industry settings.
• º Other relevant topics.

Submission Information

Contributions shoul be submiited in in PDF format through EasyChair LINK by August 20, 2025. They will be reviewed for quality and relevance to the workshop’s theme. Theoretical and applied papers and papers that capture best practices and lessons learned from field studies are encouraged. Submission of preliminary results would also be considered.

Important Dates

All dates are Anywhere on Earth (AoE).

Deadline for submission: August 20, 2025

Notification of acceptance: September 15, 2025

Camera ready: September 25, 2025

Workshop date: October 1-3, 2025

Keynote

TBD

Organizing Committee

Web Chair

Program Committee

TBD