PMTBench model and datasets

General description:

The PMTBench model defines the main components that are part of a Predictive Mutation Testing (PMT) dataset, including the entities involved, the associated properties, and how all elements are connected. It provides a standarized and structured way to describe PMT datasets so that they can be reused, extended, and automatically processed. To represent this information, PMTBench uses the Resource Description Framework (RDF), as its data model, along with the Turtle syntax, a compact and readable format for writing RDF graphs in plain text.

Model description and structure:

The model's full structure can be found in the file 'standard-pmtbench.ttl'. The model groups elements into four main categories:

• Software project: Describes the basic structure of the system under test using the classes Project, File, TestSuite, and Test.
• Mutations: Represented by the Mutant class, which describes each artificial fault introduced during mutation testing.
• Collected data: Covers the information extracted from mutants and tests: StaticAnalysis, Execution, and NaturalLanguage.
• Tooling metadata: Contains information about the tools used to generate mutants and collect data: MutationTool, AnalysisTool, and CoverageTool. 

Dataset organization:

To keep datasets modular and easy to maintain, PMTBench splits them into separate Turtle files:

• One file per project: Each TTL file contains all data related to a single project.
• An index file: A central TTL file that references all individual project files and serves as the entry point to the dataset.

Datasets:

This record includes two datasets:

1. A new dataset covering the Algorithmic or AL channel and test-suite level prediction scenario. It includes 200 Java projects and it is intended for use in a cross-project setting (citation: pmtbenchAL).
2. A converted version of the dataset used by Zhao et al. [1], covering the Natural Language or NL channel and test-case level prediction scenario. It includes 6 Java projects, each with multiple versions, and supports cross-project and cross-setting evaluations (citation: zhao2024spotting). The dataset has been extended with mutation data from 3 Python projects from BugsInPy, covering a total of 12 versions, generated with MutPy 0.6.0 (citation: pmtbenchNLpy).
   
   [1] Zhao, Y., Chen, Y., Sun, Z., Liang, Q., Wang, G., Hao, D. (2024). Spotting code mutation for predictive mutation testing. Proceedings of the 39th IEEE/ACM International Conference on Automated Software Engineering (p. 1133–1145). https://doi.org/10.1145/3691620.3695491