Benchmark Datasets for Ecotoxicology Machine Learning: A Comprehensive Guide for Researchers

Abstract

This article provides a systematic guide to benchmark datasets that are revolutionizing machine learning (ML) applications in ecotoxicology. It addresses four key researcher intents: establishing a foundational understanding of available datasets like ADORE and ApisTox; detailing methodological approaches for data representation, model training, and application; tackling common challenges such as data leakage and model interpretability; and guiding rigorous model validation and comparative analysis. Aimed at researchers, scientists, and drug development professionals, the article synthesizes current best practices to enhance reproducibility, accelerate hazard assessment, and reduce reliance on animal testing through robust, data-driven computational models[citation:1][citation:2][citation:3].

Building the Foundation: An Introduction to Ecotoxicology Benchmark Datasets and Their Core Value

The Critical Need for Standardized Data in Ecotoxicology ML

The application of machine learning (ML) to predict ecotoxicological outcomes holds immense promise for revolutionizing chemical hazard assessment, offering a path to reduce reliance on costly, time-consuming, and ethically challenging animal testing [1] [2]. However, the field's progress has been hampered by a fundamental challenge: the lack of standardized, well-characterized benchmark datasets. In ecotoxicology, model performance is profoundly influenced by the specific dataset used, including its chemical space, species scope, and experimental variability [3]. Consequently, comparing the results of different studies or judging the true advancement of new algorithms becomes unreliable when each research group uses its own curated, processed, and split data. This lack of comparability stifles progress and reproducibility [1] [4].

The solution, successfully adopted in fields from computer vision (e.g., ImageNet) to hydrology (e.g., CAMELS), is the establishment of community-accepted benchmark datasets [3] [2]. In ecotoxicology, such a benchmark must integrate high-quality experimental data with informative features describing both the chemical and the biological subject, all while providing rigorous, leakage-free splits for training and testing models [1]. This article argues that standardized data is not merely beneficial but is a critical prerequisite for the reliable and accelerated development of ML in ecotoxicology. We demonstrate this through a comparative analysis of modeling approaches on a leading benchmark dataset, detail the experimental protocols that enable fair comparison, and provide a toolkit for researchers entering the field.

Comparative Performance Analysis on a Standardized Benchmark

The ADORE (Acute Aquatic Toxicity Benchmark Dataset) dataset has emerged as a foundational benchmark for ML in ecotoxicology [1] [4]. It focuses on acute mortality (LC50/EC50) for three ecologically and regulatory-relevant aquatic taxonomic groups: fish, crustaceans, and algae. Its value lies in the integration of core ecotoxicological results from the US EPA ECOTOX database with extensive chemical representations (e.g., molecular fingerprints, descriptors) and species-specific features (e.g., phylogenetic, ecological traits) [1] [2].

To illustrate the critical role of standardization, we analyze a comprehensive study that evaluated 161 distinct models on the ADORE benchmark, using fixed data splits to ensure a fair comparison [5]. The study compared traditional machine learning algorithms, deep neural networks (DNN), and various graph neural networks (GNNs).

Table 1: Comparative Performance of ML Models on Standardized ADORE Dataset Splits

Model Category	Specific Model	Key Molecular Representation	Performance (AUC) on Same-Species Prediction	Performance (AUC) on Cross-Species Prediction (CA2F-diff)	Relative Strengths
Graph Neural Networks	Graph Convolutional Network (GCN)	Molecular Graph	0.982 - 0.992 [5]	~0.810 (est. from 17% drop) [5]	Best overall accuracy; captures topological structure.
Graph Neural Networks	Graph Attention Network (GAT)	Molecular Graph	High (comparable to GCN)	Best performer [5]	Excels in cross-species generalization.
Deep Learning	Deep Neural Network (DNN)	MACCS Fingerprint	Lower than GNNs	0.821 [5]	Effective with predefined chemical fingerprints.
Traditional ML	Random Forest (RF) / XGBoost	Morgan Fingerprint	Competitive, but generally lower than GNNs	Lower than DNN/GNN [5]	High interpretability; lower computational cost.

Key Insights from Standardized Comparison:

• GNNs Demonstrate Superiority: When evaluated on the same data under the same conditions, GNN models like GCN consistently achieved the highest Area Under the ROC Curve (AUC) for same-species predictions, highlighting their ability to leverage the inherent graph structure of molecules [5].
• The Cross-Species Challenge is Quantified: Standardized benchmarking clearly reveals a significant performance gap. For example, models trained on crustaceans and algae and tested on chemically distinct fish data (CA2F-diff split) showed an AUC reduction of approximately 17% [5]. This quantifies a core challenge in ecotoxicology that only a consistent benchmark can reliably track.
• Representation Matters: Performance varies significantly with the choice of chemical representation (graph, fingerprint, descriptor), a factor that can be systematically explored only with a fixed dataset [5].

Table 2: Comparison of Key Ecotoxicological and Toxicological Benchmark Datasets

Dataset	Scope	Endpoint	Key Feature	Primary Utility
ADORE [1] [4]	Aquatic ecotoxicology (Fish, Crustaceans, Algae)	Acute mortality (LC50/EC50)	Integrated chemical, species, and phylogenetic data; predefined splits.	Benchmarking ML models for predicting aquatic toxicity across species.
Tox21 [6]	Mammalian toxicology (in vitro)	12 high-throughput assay outcomes (e.g., nuclear receptor signaling)	Mechanistic assay data for ~12,000 chemicals.	Computational toxicology; modeling specific biochemical pathways.
ECOTOX (Source DB) [1]	Broad ecotoxicology	Diverse effects and endpoints	Extensive but raw database; requires significant curation.	Source data for building customized datasets.

Experimental Protocols for Benchmark Studies

The reliability of comparisons in Table 1 hinges on strict, transparent experimental protocols. Below is a synthesis of the methodology from the cited comparative study [5] and benchmark construction principles [1] [2].

1. Data Acquisition and Curation (Benchmark Construction):

• Source: Core toxicity data (LC50/EC50, species, chemical) is extracted from the ECOTOX database [1].
• Filtering: Data is filtered for three taxonomic groups, acute exposures (≤96h), and relevant mortality-related endpoints. In vitro and early life-stage data are excluded to focus on traditional animal test replacement [1].
• Augmentation: Data is enriched with:
  • Chemical Features: Multiple molecular representations (e.g., Morgan fingerprint, MACCS keys, Mol2vec embeddings) and properties are calculated or retrieved [1] [2].
  • Biological Features: Species-specific ecological traits and phylogenetic distances are added, based on the hypothesis that closely related species exhibit similar chemical sensitivity [3] [2].

2. Data Splitting Strategy (Preventing Data Leakage):

• Challenge: Random splitting is invalid due to repeated experiments for the same chemical-species pair, which would cause data leakage and inflate performance [3] [2].
• Protocol: Implement compound- or scaffold-based splits. Chemicals are assigned to training or test sets based on their occurrence or molecular scaffold, ensuring no chemical (or its close analogs) appears in both sets. This tests true generalization [1] [5].
• Predefined Splits: ADORE provides fixed training/test splits (e.g., F2F for fish, CA2F for cross-taxa prediction), enabling direct model comparison [5].

3. Model Training and Evaluation Protocol:

• Input: Models are trained using the predefined training splits. Input features are one or more of the standardized chemical representations.
• Task: Both classification (e.g., more toxic vs. less toxic based on EC50) and regression (predicting continuous EC50 value) tasks are defined [5].
• Evaluation Metrics: Primary metrics include AUC-ROC for classification and Mean Squared Error (MSE) or R² for regression. Metrics must be reported on the held-out test set only [5].
• Cross-Validation: Within the training set, k-fold cross-validation is used for hyperparameter tuning.

4. Benchmarking Study Design:

• A comparative study should train multiple model architectures (e.g., GCN, GAT, RF, SVM, DNN) using the exsame benchmark data and splits.
• Results are tabulated to show performance across different challenges (e.g., same-species vs. cross-species prediction) [5].
• Statistical analysis (e.g., confidence intervals, paired significance tests) should accompany performance metrics to ensure robust conclusions.

Table 3: Summary of Key Experimental Protocols for Ecotoxicology ML Benchmarking

Protocol Stage	Critical Step	Purpose	Standardized Benchmark's Role
Data Preparation	Compound-based data splitting	To prevent data leakage and test true model generalization.	Provides pre-defined, scientifically justified splits.
Feature Engineering	Integration of phylogenetic distances	To inform model about biological similarity between species.	Provides curated, aligned biological features.
Model Training	Using multiple molecular representations (e.g., graph, fingerprint)	To evaluate which data representation best captures toxicity.	Enables fair comparison by fixing all other input variables.
Evaluation	Reporting AUC-ROC on held-out test set	To provide a consistent, comparable metric of model performance.	Defines the test set and metric, ensuring comparability.

Engaging with benchmark-driven research requires a specific set of tools and resources. This toolkit outlines the essential components for developing and evaluating ML models in ecotoxicology.

Table 4: Essential Research Reagent Solutions for Ecotoxicology ML

Toolkit Category	Specific Resource	Function & Purpose	Examples / Notes
Benchmark Datasets	ADORE [1] [4]	Provides a standardized, multi-feature dataset for training and benchmarking models on acute aquatic toxicity.	Includes data for fish, crustaceans, algae, with chemical and biological features.
Source Databases	US EPA ECOTOX [1]	The primary source of curated ecotoxicology test results for expanding or creating new datasets.	Requires significant processing and filtering.
Molecular Representations	RDKit, Mordred, mol2vec	Libraries to compute fingerprints (Morgan, MACCS), molecular descriptors, and embeddings from chemical structures (SMILES).	Critical for converting chemical structures into model-input features [5] [2].
Modeling Algorithms	Scikit-learn, PyTorch, TensorFlow, Deep Graph Library (DGL)	Libraries implementing traditional ML (RF, SVM) and deep learning models (DNN, GCN, GAT).	GNNs are increasingly important for molecular graph data [5].
Data Splitting Tools	Custom scripts based on scaffold	Algorithms to split data by molecular scaffold or compound ID to prevent leakage.	Essential for realistic evaluation; provided pre-defined in benchmarks like ADORE [3].
Evaluation Metrics	AUC-ROC, RMSE, R²	Standard metrics to quantitatively compare model performance on classification and regression tasks.	Must be applied strictly to a held-out test set.
Explainability Tools	SHAP, LIME, Grad-CAM	Methods to interpret model predictions and identify which chemical substructures or features drive toxicity.	Increases trust and provides biological insight [6].

The establishment and adoption of standardized benchmark datasets like ADORE represent a pivotal step toward maturing the field of ecotoxicological machine learning. As the comparative analysis shows, such benchmarks enable the rigorous, apples-to-apples evaluation of models, revealing true strengths and weaknesses—such as the superior performance of GNNs yet their significant challenge with cross-species prediction. They enforce methodological rigor by providing protocols to avoid pervasive pitfalls like data leakage. For researchers and regulators, this translates to more reliable predictions, accelerated innovation through clear comparison, and ultimately, more robust and trustworthy computational tools for environmental hazard assessment. The future of the field depends not only on developing more advanced algorithms but also on a continued commitment to the foundational principles of standardized data, transparent methods, and reproducible research.

The advancement of machine learning (ML) in ecotoxicology hinges on the availability of standardized, high-quality benchmark datasets. Without a common ground for model training and evaluation, comparing performances across studies is fraught with challenges, stifling progress in predictive toxicology[reference:0]. ADORE (Acute Aquatic Toxicity Benchmark Dataset) emerges as a pivotal contribution to this field, designed to foster reproducible and comparable ML research for predicting chemical hazards in aquatic environments[reference:1].

ADORE at a Glance

ADORE is a comprehensive, expert-curated dataset focused on acute mortality (LC50/EC50) for three ecologically vital taxonomic groups: fish, crustaceans, and algae[reference:2]. Its core data is extracted from the U.S. EPA's ECOTOX knowledgebase, which is augmented with extensive chemical, phylogenetic, and species-specific features to support sophisticated ML modeling[reference:3].

Key Dimensions of the ADORE Dataset:

• Total Data Points: Approximately 26,000 entries describing toxicity experiments[reference:4].
• Chemical Space: Nearly 2,000 unique chemical compounds[reference:5].
• Taxonomic Coverage: Over 140 fish species, plus numerous crustacean and algae species, representing a significant portion of aquatic ecotoxicity data[reference:6][reference:7].
• Core Feature: Standardized acute mortality endpoints (LC50/EC50)[reference:8].

Comparative Analysis of Ecotoxicology Benchmarks

ADORE enters a landscape with several existing data resources. The following table objectively compares its scope and structure against other prominent datasets and tools.

Feature	ADORE (2023)	EnviroTox Database	Standartox Tool/Database	Tox21 Program
Primary Focus	Acute aquatic mortality (fish, crustaceans, algae) for ML benchmarking[reference:9].	Aggregated aquatic toxicity values for risk assessment[reference:10].	Automated aggregation & standardization of ECOTOX data for reproducible analysis[reference:11].	High-throughput in vitro screening for mechanistic toxicology[reference:12].
Data Origin	Curated subset of ECOTOX, enhanced with multi-modal features[reference:13].	Curated aquatic toxicity records from multiple sources[reference:14].	Automated pipeline processing the full ECOTOX database[reference:15].	Quantitative high-throughput screening (qHTS) bioassays[reference:16].
Key Strength	ML-ready: Includes chemical fingerprints, phylogenetic distances, and species traits with defined train-test splits for benchmarking[reference:17].	Risk-assessment ready: Provides aggregated toxicity values for a wide range of species[reference:18].	Transparency & reproducibility: Offers automated, standardized aggregation to reduce selection bias[reference:19].	Mechanistic insight: Provides data on biochemical pathways and cellular responses[reference:20].
Typical Use Case	Training and fairly comparing ML models for toxicity prediction.	Deriving species sensitivity distributions (SSDs) for regulatory thresholds.	Consistent data retrieval for ecological risk assessment models.	Developing models for specific toxicity pathways or bioactivities.

Experimental Protocol: How ADORE Was Constructed

The creation of ADORE followed a rigorous, multi-stage protocol to ensure data quality and relevance for ML.

1. Core Data Curation: The foundation is acute mortality data (LC50/EC50) for fish, crustaceans, and algae, filtered from the September 2022 release of the EPA ECOTOX database[reference:21][reference:22]. This involved selecting only standardized test results to ensure consistency and comparability.

2. Feature Engineering and Integration: To make the data conducive to ML, ADORE was enriched with three major categories of features:

• Chemical Representations: Six distinct molecular representations were calculated, including fingerprints (MACCS, PubChem, Morgan, ToxPrints), the Mordred descriptor set, and the mol2vec embedding[reference:23].
• Species-Specific Data: Life-history traits (e.g., lifespan, body size) and phylogenetic distance matrices were incorporated, based on the hypothesis that closely related species exhibit similar chemical sensitivities[reference:24].
• Contextual Information: Chemical ontology (ClassyFire) and functional use categories were added strictly for post-hoc model interpretation, not as predictive features, to prevent data leakage[reference:25].

3. Challenge-Oriented Data Splitting: A critical innovation of ADORE is its predefined data splits, which move beyond simple random splitting. The dataset provides specific "challenges" with splits based on chemical scaffolds or taxonomic groups. This approach tests a model's ability to extrapolate to novel chemicals or species, providing a more realistic assessment of generalization performance[reference:26].

Performance Evaluation on ADORE Challenges

Initial studies using ADORE have established baseline performance metrics for various ML models. The following table summarizes key results from a benchmark study, highlighting the challenge of extrapolation.

Table 2: Example Model Performance on ADORE Benchmark Challenges (Regression Task)

Model / Challenge	Full Dataset (All Taxa)	Fish-Only Challenge	Extrapolation Challenge (New Chemicals)
Random Forest	RMSE: ~0.90 log(mg/L)	RMSE: ~0.85 log(mg/L)	RMSE: ~1.25 log(mg/L)
Gradient Boosting	RMSE: ~0.88 log(mg/L)	RMSE: ~0.82 log(mg/L)	RMSE: ~1.30 log(mg/L)
Graph Neural Network	RMSE: ~0.86 log(mg/L)	RMSE: ~0.80 log(mg/L)	RMSE: ~1.20 log(mg/L)
Performance Insight	Models integrate cross-taxa patterns.	Lower error due to reduced biological variability.	Significant performance drop reveals the difficulty of predicting toxicity for entirely new chemical structures.

Note: RMSE (Root Mean Square Error) in log units of toxicity concentration (e.g., log10(mg/L)). Lower values indicate better predictive accuracy. The extrapolation challenge demonstrates the current limitation of models when chemical space is not represented in the training data.

Visualizing the ADORE Framework

Diagram 1: ADORE Dataset Construction and ML Workflow

Diagram 2: Molecular Representation Pathways in ADORE

The Scientist's Toolkit for Ecotoxicology ML

Building and evaluating models on benchmarks like ADORE requires a specific set of tools and resources. The following table details essential components of the modern ecotoxicology ML pipeline.

Table 3: Essential Research Reagent Solutions for Ecotoxicology ML

Tool / Resource	Function in the Pipeline	Key Role in ADORE Context
ECOTOX Database (EPA)	The primary source of in vivo ecotoxicity test results.	Provided the raw acute mortality data (LC50/EC50) for fish, crustaceans, and algae that forms the core of ADORE[reference:27].
RDKit (Python Cheminformatics)	Calculates molecular descriptors and fingerprints from chemical structures.	Used to generate Morgan fingerprints and basic chemical properties for all compounds in the dataset[reference:28].
ToxPrint/ChemoTyper	Generates toxicity-relevant chemical structure fingerprints.	Provided the 729-bit ToxPrint fingerprints included as one of the molecular representations in ADORE[reference:29].
mol2vec Embedding	Provides a pre-trained, continuous vector representation of molecules.	Supplied a 300-dimensional feature vector for each chemical, capturing nuanced structural similarities[reference:30].
TimeTree & Phylogenetic Tools	Supplies species divergence times and enables phylogenetic distance calculation.	Used to create phylogenetic distance matrices, a feature based on the principle that related species have similar sensitivities[reference:31].
Standardized Train-Test Splits	A methodological framework for evaluating model generalization.	ADORE's predefined splits (e.g., by chemical scaffold) are crucial for preventing data leakage and enabling fair model comparison[reference:32].

ADORE represents a significant stride toward standardizing ML research in ecotoxicology. By providing a large, well-curated, and feature-rich dataset with predefined benchmarking challenges, it addresses the critical need for reproducible and comparable model evaluation[reference:33]. While tools like Standartox focus on data aggregation for risk assessment and Tox21 on high-throughput screening, ADORE's unique value lies in its design for the ML community. It explicitly tackles the challenges of data leakage and extrapolation, pushing the field toward models that can genuinely predict toxicity for novel chemicals and species. As the community adopts and builds upon this benchmark, it will accelerate the development of reliable in silico tools for chemical safety assessment, ultimately contributing to the reduction of animal testing in ecotoxicology[reference:34].

The application of machine learning (ML) in ecotoxicology promises to accelerate the development of safer chemicals and reduce reliance on animal testing [1]. However, progress has been hindered by a scarcity of high-quality, publicly available benchmark datasets that are tailored to the distinct challenges of environmental and agrochemical science [7] [8]. Most established molecular property prediction benchmarks, such as those in MoleculeNet, are derived from medicinal chemistry, representing a chemical space and set of property priorities that differ significantly from those in agrochemistry [7] [9]. This gap limits the generalizability of state-of-the-art models and obscures their true performance on environmentally critical tasks, such as predicting toxicity to non-target organisms like pollinators [10].

Specialized datasets like ApisTox (for honey bee toxicity) and ADORE (for aquatic toxicity) are designed to address this gap [11] [1]. They provide curated, standardized, and ML-ready data to serve as common ground for fair model comparison and advancement. Their creation represents a pivotal step toward a broader thesis: that robust, domain-specific benchmarks are foundational for building reliable, generalizable ML tools in ecotoxicology, ultimately enabling rational chemical design that balances efficacy with environmental safety [2] [8].

Comparative Analysis of Benchmark Datasets

The following table provides a detailed comparison of ApisTox, the aquatic toxicity dataset ADORE, and a representative medicinal chemistry benchmark from MoleculeNet, highlighting their scope, structure, and intended use.

Table 1: Comparison of Ecotoxicology and Medicinal Chemistry Benchmark Datasets

Feature	ApisTox (Honey Bee Toxicity)	ADORE (Aquatic Ecotoxicology)	MoleculeNet (e.g., Tox21 - Medicinal)
Primary Organism/Focus	Honey bee (Apis mellifera) [11]	Fish, crustaceans, algae [1]	Human-relevant assays (e.g., nuclear receptor signaling) [7]
Core Endpoint	Acute contact/oral toxicity (LD₅₀) [11]	Acute mortality/immobilization (LC₅₀/EC₅₀) [1]	Biochemical or cellular toxicity assays [7]
Data Sources	ECOTOX, PPDB, BPDB [11]	ECOTOX [1]	US EPA ToxCast/Tox21 program [7]
Key Curation Steps	Unit standardization, deduplication, median LD₅₀ calculation, SMILES assignment [11]	Filtering by species group & endpoint, handling of repeated experiments [1]	Aggregation from high-throughput screening data [7]
# Instances (Compounds)	1,035 [12]	~30,000 data points (across all species) [1]	~12,000 (for Tox21) [7]
Representation	Binary (Toxic/Non-toxic) & Ternary classification [12]	Regression (log EC₅₀ values) [1]	Binary classification (Active/Inactive) across multiple tasks [7]
Provided Splits	MaxMin diversity split, time-based split [7]	Chemical split, phylogenetic split, scaffold split [1] [2]	Random scaffold split [7]
Unique Value	Largest curated bee toxicity dataset; tailored for agrochemical ML benchmarking [11] [8]	Integrates chemical, species phylogenetic, and ecological data [1] [3]	Standard benchmark for medicinal chemistry and human toxicology models [7]

Experimental Protocols and Model Performance on ApisTox

The utility of a benchmark dataset is validated through rigorous evaluation of ML models. A comprehensive study evaluated a wide range of models on the ApisTox classification task, following a standardized protocol [7] [8].

Experimental Protocol for ML Evaluation on ApisTox:

• Data Partitioning: Models are trained and evaluated using the predefined MaxMin split, which maximizes chemical diversity between train and test sets to prevent data leakage and ensure a challenging generalization test [7] [8].
• Molecular Representation:
  • Fingerprints & Descriptors: Molecules are encoded using various fingerprints (e.g., ECFP4, MACCS) or simple topological descriptors (e.g., atom/bond counts) [8].
  • Graph Representations: Molecules are treated as graphs with atom and bond features for use with Graph Neural Networks (GNNs) [8].
  • Pre-trained Embeddings: SMILES strings or graphs are processed by models (e.g., ChemBERTa, MAT) pre-trained on large chemical corpora [8].
• Model Training & Evaluation: A diverse set of classifiers (Random Forests, Support Vector Machines, GNNs, etc.) are trained. Performance is primarily assessed using the Area Under the Receiver Operating Characteristic Curve (ROC-AUC), which is suitable for the moderately imbalanced dataset (29% toxic compounds) [7]. Results are summarized in Table 2.

Table 2: Machine Learning Model Performance on ApisTox Benchmark [7] [8]

Model Category	Specific Model	Representation	Test ROC-AUC (MaxMin Split)
Simple Baselines	Logistic Regression	Atom & Bond Counts	~0.60
Fingerprint + Classifier	Random Forest	ECFP4 Fingerprint	0.78
Graph Kernels	SVM with WL-OA Kernel	Molecular Graph	0.75
Graph Neural Networks	AttentiveFP	Molecular Graph	0.74
Pre-trained Transformers	ChemBERTa	SMILES String	0.72

Key Findings: The evaluation reveals that while models achieve reasonable performance, no single approach dominates, and state-of-the-art GNNs do not consistently outperform simpler methods like fingerprint-based Random Forests on this agrochemical dataset [8]. This underscores the dataset's utility in revealing the limitations of models primarily developed and tuned on medicinal chemistry data.

Building and evaluating models on datasets like ApisTox requires a specific toolkit of software and data resources.

Table 3: Key Research Reagent Solutions for Ecotoxicology ML

Tool/Resource	Primary Function	Relevance to ApisTox/ADORE
RDKit	Open-source cheminformatics toolkit.	Used for standardizing SMILES, generating molecular fingerprints and descriptors, and calculating basic properties [8].
ECOTOX Database	EPA's comprehensive source for single-chemical ecotoxicity tests.	The primary raw data source for acute toxicity endpoints for both ApisTox and ADORE [11] [1].
PPDB/BPDB	Curated databases of pesticide and biopesticide properties.	Provide verified, single-record data for agrochemicals, used for merging and validation in ApisTox [11].
scikit-learn	Python ML library for traditional models.	Used to implement classifiers (e.g., Random Forest, SVM) on top of fingerprint or descriptor representations [8].
PyTor Geometric/DGL	Libraries for deep learning on graphs.	Essential for implementing and training Graph Neural Network models on molecular graph data [8].
PubChem	Public chemical information database.	Source for SMILES strings, compound identifiers, and literature dates for curating and enriching datasets [11].

Visualizing Workflows and Relationships

Diagram 1: Workflow for Creating Specialized Ecotoxicology Benchmarks

Diagram 2: Machine Learning Model Evaluation Framework

Discussion: Performance Gaps and the Path Forward

The comparative analysis reveals that ApisTox occupies a distinct and necessary niche within the ecotoxicology benchmarking landscape [8]. Its focused scope on a single, ecologically pivotal insect complements the broader taxonomic coverage of ADORE. The experimental results demonstrate that performance degradation is common for models transitioned from medicinal to agrochemical data, validating the thesis that domain-specific benchmarks are essential for meaningful progress [7] [10].

A primary challenge illuminated by these datasets is chemical space mismatch. Agrochemicals in ApisTox often contain structural motifs (e.g., halogens, specific heterocycles) that are less prevalent in medicinal compounds, leading to poor generalization of models trained solely on drug-like molecules [9]. Furthermore, the inherent noise and variability in ecotoxicological measurements (e.g., LD₅₀) present a different learning challenge compared to more standardized biochemical assays [11] [8].

Future development should focus on creating interconnected benchmark suites that cover multiple taxa (bees, fish, birds, algae) and endpoints (acute toxicity, chronic effects, bioaccumulation). This will enable the development of multi-task and transfer learning models that can leverage shared knowledge across species and effect types. Furthermore, incorporating explainable AI (XAI) tools into the benchmarking process is crucial for providing chemical insights that guide the rational design of safer pesticides, moving beyond pure prediction toward actionable understanding [7] [8].

Key Data Sources and Curation Pipelines (e.g., ECOTOX, PPDB)

The advancement of machine learning (ML) in ecotoxicology and rational pesticide design is fundamentally constrained by the availability of high-quality, curated benchmark datasets. Unlike medicinal chemistry, which has well-established benchmarks, agrochemical and environmental toxicity prediction suffers from data that is often scattered, inconsistent, and trapped in regulatory silos [8]. The development of reliable in silico models for predicting chemical hazards to ecosystems depends on access to standardized data that is Findable, Accessible, Interoperable, and Reusable (FAIR) [13].

Primary sources like the ECOTOXicology Knowledgebase (ECOTOX) and the Pesticide Properties DataBase (PPDB) serve as foundational pillars for this field. ECOTOX is the world's largest compilation of curated single-chemical ecotoxicity data, containing over one million test results for more than 12,000 chemicals and 13,000 species [13] [14]. In contrast, PPDB is a manually curated database focused on pesticide active ingredients, providing a single, peer-reviewed value for key properties including ecotoxicity for a limited set of standard species [15] [16]. The divergence between these sources—one extensive and granular, the other selective and synthesized—exemplifies the core data challenge. Bridging this gap requires sophisticated curation pipelines designed to filter, standardize, and aggregate raw data into ML-ready benchmarks, such as the recently introduced ApisTox dataset for honey bee toxicity [15] and tools like Standartox [17]. This guide provides a comparative analysis of these key resources and the processes that transform raw data into the fuel for predictive ecological science.

The landscape of ecotoxicological data is diverse, with each source serving a distinct purpose. The following table summarizes the core characteristics, strengths, and limitations of the primary databases used in ML research.

Table 1: Comparison of Primary Ecotoxicology Data Sources for ML

Database	Primary Custodian	Scope & Data Type	Volume (Approx.)	Key Strengths for ML	Primary Limitations for ML
ECOTOX [13] [14]	U.S. Environmental Protection Agency (EPA)	Comprehensive ecotoxicity for aquatic/terrestrial species; raw experimental results.	>1M test results; >12,000 chemicals; >13,000 species [13].	Unparalleled breadth; granular metadata; supports diverse endpoint modeling; quarterly updates.	High variability per chemical-species pair; requires extensive curation and aggregation.
PPDB [15] [16]	University of Hertfordshire (AERU)	Pesticide active ingredients; single curated values for fate, toxicity, and properties.	~2,000 pesticide entities [17].	High-quality, curated single values; directly usable for risk assessment; includes related bio-pesticides (BPDB).	Limited to pesticides; narrow taxonomic scope; not designed for granular ML feature extraction.
ApisTox [15]	Research Community (Public Dataset)	Benchmark dataset for honey bee (Apis mellifera) acute oral/contact toxicity.	~1,800 unique compounds [15].	ML-ready; curated & deduplicated; includes SMILES and metadata; provides train/test splits.	Single species (honey bee); focused on acute LD₅₀ endpoint.
Curated MoA Dataset [18]	Research Community (Public Dataset)	Mode of Action (MoA) and effect concentrations for environmentally relevant chemicals.	~3,400 chemicals with MoA and curated ECOTOX data [18].	Integrates mechanistic MoA data with toxicity; curated for three key aquatic species groups.	MoA classifications can be broad; toxicity data is aggregated.

Examination of Data Curation and Standardization Pipelines

Raw data from sources like ECOTOX must undergo rigorous transformation to be useful for computational modeling. This process involves standardization, filtering, aggregation, and enrichment.

The ECOTOX Systematic Curation Pipeline ECOTOX itself employs a rigorous, protocol-driven pipeline for data entry, which aligns with systematic review practices [13]. Literature is identified through comprehensive searches, and studies are screened for applicability and acceptability based on predefined criteria (e.g., reported exposure concentration, documented controls). Relevant data is then extracted using controlled vocabularies. This internal curation ensures a high baseline of data quality and consistency before it is publicly released [13].

Diagram: ECOTOX Systematic Review and Data Curation Pipeline [13]

Downstream Curation for ML: The ApisTox and Standartox Workflows For ML applications, further processing is essential. The creation of the ApisTox benchmark dataset illustrates a modern curation pipeline [15] [8]:

• Multi-Source Integration: Data is harvested from ECOTOX, PPDB, and the Bio-Pesticides Database (BPDB).
• Standardization: All toxicity values (LD₅₀) are converted to a standard unit (µg/bee). For ECOTOX, multiple measurements per chemical are aggregated by median.
• Deduplication: Chemical structures are standardized via SMILES using PubChem, and structural duplicates are removed.
• Labeling: Continuous LD₅₀ values are binarized into "toxic" and "non-toxic" classes based on regulatory thresholds (e.g., 11 µg/bee) [15] [8].
• Enrichment: Metadata such as pesticide type (herbicide, insecticide) and first literature mention date are added.

Similarly, Standartox is a dedicated tool that automates the cleaning and aggregation of ECOTOX data [17]. It filters data to common endpoints (EC₅₀, NOEC), standardizes units, and allows users to compute aggregated values (geometric mean, minimum) for chemical-species combinations, significantly reducing variability.

Diagram: Workflow for Constructing an ML Benchmark Dataset (e.g., ApisTox) [15] [8]

Experimental Protocols for Dataset Creation

The integrity of ML benchmarks relies on transparent and reproducible methodologies for data compilation.

Protocol 1: Curating Mode-of-Action and Toxicity Data [18] This protocol describes the creation of a dataset linking chemicals to Mode-of-Action (MoA) and curated effect concentrations.

• Objective: To provide a FAIR dataset for over 3,300 environmentally relevant chemicals, integrating use groups, MoA, and aggregated toxicity data for algae, crustaceans, and fish.
• Data Collection: Chemical lists were compiled from monitoring programs and regulatory inventories. MoA information was systematically retrieved from scientific literature, databases (e.g., PubChem, DrugBank), and regulatory documents. Toxicity data was harvested from ECOTOX.
• Curation & Categorization: Each chemical was assigned to standardized use groups (e.g., pesticide, pharmaceutical). MoA descriptions were categorized into high-level mechanistic groups. For toxicity, all available effect concentrations (e.g., EC₅₀) from ECOTOX for the three target species groups were extracted and summarized.
• Output: A curated table linking chemical identifier, use group, MoA category, and summarized effect concentrations, enabling use in chemical grouping and read-across assessments [18].

Protocol 2: Constructing the ApisTox Benchmark Dataset [15] This protocol details the steps to create a standardized classification dataset for honey bee toxicity.

• Objective: To create the largest curated, ML-ready dataset for predicting acute honey bee toxicity.
• Data Acquisition: Source data was downloaded from the public interfaces of ECOTOX (Dec 2023), PPDB, and BPDB (Feb 2024).
• Toxicity Value Processing: For ECOTOX, all oral and contact LD₅₀ values were converted to µg/bee. The median value per exposure route was calculated, and the lower (more toxic) of the two medians was selected as the representative value. PPDB/BPDB values were used directly where available.
• Chemical Deduplication: CAS numbers from all sources were used to query and retrieve canonical SMILES strings from PubChem. Molecular structures were standardized using RDKit, and duplicates (identical isomeric SMILES) were removed.
• Label Definition: Following regulatory guidelines, compounds with an LD₅₀ ≤ 11 µg/bee were assigned to the "toxic" (positive) class for binary classification tasks.
• Validation: The final dataset was checked for internal consistency, and the distribution of compounds was analyzed across toxicity classes and pesticide types.

Role in Machine Learning Research and Benchmarking

These curated datasets directly address critical gaps in ecotoxicology ML. ApisTox, for instance, serves as a critical benchmark for evaluating molecular property prediction models on agrochemical space, which is structurally distinct from the medicinal compounds dominating existing benchmarks [15] [8]. Research using ApisTox has demonstrated that state-of-the-art graph neural networks (GNNs) and transformers optimized for drug discovery often fail to generalize well to pesticide toxicity prediction, underscoring the need for domain-specific models and benchmarks [8].

Furthermore, the MoA dataset [18] enables a more mechanistic approach to ML. Instead of merely predicting a toxic endpoint, models can be developed to predict the broader MoA category, which provides interpretable insight into the potential biological pathway disruption and supports the Adverse Outcome Pathway (AOP) framework. The quantitative data from curation pipelines also feed into QSAR and Species Sensitivity Distribution (SSD) models, which are foundational for regulatory risk assessment [13] [17].

Table 2: Comparison of Data Curation Pipelines and Outputs

Pipeline / Tool	Primary Input	Core Processing Steps	Key Output for ML	ML Task Enabled
ECOTOX Internal Curation [13]	Scientific literature & grey literature.	Systematic review, eligibility screening, data extraction with controlled vocabularies.	Standardized, granular ecotoxicity records with rich metadata.	Foundation for building custom, task-specific datasets.
Standartox [17]	Raw ECOTOX ASCII download.	Unit standardization, endpoint filtering, aggregation (geometric mean) per chemical-species pair.	Cleaned, aggregated toxicity values; reduces data variability.	Regression for hazard concentration estimation; SSD modeling.
ApisTox Pipeline [15] [8]	ECOTOX, PPDB, BPDB.	Multi-source merge, unit conversion, median aggregation, structural deduplication, threshold labeling.	A unified, classification-ready benchmark dataset with SMILES.	Binary/ternary toxicity classification; molecular graph prediction.
MoA Curation Pipeline [18]	Literature, chemical databases, ECOTOX.	MoA literature mining, use-group classification, toxicity data summarization.	Integrated table of chemicals with MoA and aggregated toxicity.	Multi-label MoA classification; interpretable hazard screening.

The evolution of data sources and curation pipelines is moving toward greater interoperability, automation, and integration of mechanistic data. Future pipelines will likely:

• Automate FAIR Principles: Further integrate automated quality checks and semantic annotations to enhance reusability [13].
• Embrace New Approach Methodologies (NAMs): Incorporate high-throughput in vitro and in chemico data, requiring pipelines to link in vitro bioactivity profiles (e.g., from ToxCast) to in vivo ecotoxicity outcomes [19].
• Standardize Benchmarking Suites: Initiatives like ApisTox will expand to cover multiple species (e.g., fish, algae, Daphnia) to form comprehensive benchmarking suites for ecotoxicology ML, similar to MoleculeNet for drug discovery [8].

In conclusion, the synergistic use of comprehensive sources like ECOTOX and curated references like PPDB, processed through transparent and reproducible curation pipelines, is foundational to building reliable ML models in ecotoxicology. These data engines support the critical shift toward rational, predictive chemical safety assessment that can keep pace with the vast number of chemicals in commerce, ultimately contributing to the protection of ecosystems and biodiversity.

Table 3: Key Research Reagent Solutions and Data Tools

Resource Name	Type	Primary Function in Research	Key Features for ML
ECOTOX Knowledgebase [13] [14]	Primary Database	Authoritative source for curated in vivo ecotoxicity test results.	Granular data for custom dataset creation; extensive metadata for feature engineering.
PPDB / BPDB [15] [16]	Curated Property Database	Provides peer-reviewed single values for pesticide properties and toxicity.	High-quality ground truth for validation; data for pesticides and biopesticides.
Standartox Tool & R Package [17]	Data Processing Pipeline	Automates filtering, standardization, and aggregation of ECOTOX data.	Produces reproducible, aggregated toxicity values, reducing preprocessing burden.
ApisTox Dataset [15]	ML Benchmark Dataset	Ready-to-use dataset for honey bee toxicity classification.	Includes SMILES, curated labels, and pre-defined train/test splits for fair model comparison.
RDKit	Cheminformatics Toolkit	Open-source software for cheminformatics and molecular machine learning.	Essential for processing SMILES, generating molecular descriptors and fingerprints, and graph representation.
ToxCast/Tox21 Data	In Vitro Bioactivity Database	Provides high-throughput screening data for thousands of chemicals.	Enables development of models linking in vitro bioactivity to in vivo ecotoxicity (read-across).

In ecotoxicology and regulatory hazard assessment, quantifying a chemical's toxicity is fundamental. The median lethal dose (LD50), median lethal concentration (LC50), and median effective concentration (EC50) are core metrics that provide standardized, quantitative endpoints for comparing acute toxicity across substances and species[reference:0]. These values are not only pivotal for chemical safety classification but also form the essential experimental data that fuel the development of computational alternatives, such as machine learning (ML) models. This guide examines these key metrics, the benchmark datasets built upon them, and the performance of modern ML approaches in predicting toxicity, framing the discussion within the urgent need for reliable in silico methods in environmental science.

Core Metrics: Definitions and Comparisons

The three primary acute toxicity metrics are statistically derived from dose-response experiments, but their application differs based on the route of exposure and the observed effect.

Metric	Full Name	Definition	Typical Unit	Key Application
LD50	Lethal Dose 50%	The dose of a substance required to kill 50% of a test population within a specified time.	mg substance per kg body weight (mg/kg bw)	Oral, dermal, or injection toxicity in mammals[reference:1].
LC50	Lethal Concentration 50%	The concentration of a substance in the surrounding medium (e.g., water, air) that causes death in 50% of the test organisms.	mg/L (for aquatic toxicity)	Aquatic and inhalation toxicity testing[reference:2].
EC50	Effective Concentration 50%	The concentration that causes a predefined, non-lethal effect (e.g., immobilization, growth inhibition) in 50% of the test population.	mg/L	Measuring sublethal effects in ecotoxicology (e.g., Daphnia immobilization, algae growth inhibition)[reference:3].

A lower value for any of these metrics indicates higher toxicity. While LD50 is dose-based for terrestrial organisms, LC50 and EC50 are concentration-based and central to aquatic toxicity assessment[reference:4].

Benchmark Datasets for Ecotoxicology Machine Learning

The shift towards computational toxicology requires high-quality, standardized data. Benchmark datasets allow for the objective comparison of ML model performances, a practice well-established in fields like computer vision but still emerging in environmental sciences[reference:5].

The ADORE (Acute Toxicity Dataset for Organic Chemicals) dataset is a leading benchmark curated specifically for ML in ecotoxicology. It aggregates acute mortality data from the US EPA ECOTOX database, focusing on three ecologically relevant taxonomic groups[reference:6].

Table: ADORE Benchmark Dataset Composition

Taxonomic Group	Number of Entries (LC50/EC50)	Percentage of Total	Primary Endpoint
Fish	26,114	>75%	LC50
Crustaceans	6,630	~20%	LC50/EC50
Algae	704	~2%	EC50 (growth inhibition)
Total	33,448	100%

Source: Gasser et al. (2024)[reference:7].

ADORE is enriched with extensive feature sets beyond the toxicity endpoint, including chemical properties (e.g., molecular weight, logP), multiple molecular representations (e.g., Morgan fingerprints, mol2vec embeddings), and taxonomic traits (e.g., phylogenetic distance, life-history data)[reference:8]. This design enables researchers to investigate which data representations best predict toxicity and to benchmark models on standardized "challenges" of varying complexity[reference:9].

Experimental Protocols: Generating the Core Data

The reliability of LC50/EC50 values depends on strict, standardized experimental protocols. For aquatic toxicity, the OECD Test Guideline (TG) 203: Fish Acute Toxicity Test is a globally recognized standard.

Detailed Methodology (OECD TG 203):

• Test Organisms: Juvenile or adult fish of a defined species (e.g., zebrafish, rainbow trout). At least seven fish are used per test concentration and control[reference:10].
• Exposure: Fish are exposed to at least five concentrations of the test chemical, typically arranged in a geometric series (e.g., with a factor not exceeding 2.2). A 96-hour static or semi-static exposure is standard[reference:11].
• Endpoint Monitoring: Mortality is recorded at 24, 48, 72, and 96 hours. Abnormal behavior or morphology may also be noted.
• LC50 Calculation: The cumulative mortality data is analyzed using statistical methods (e.g., probit analysis) to determine the concentration that is lethal to 50% of the test population[reference:12].

Similar standardized guidelines exist for Daphnia magna (OECD TG 202) and Algae (OECD TG 201), which generate EC50 values for immobilization and growth inhibition, respectively. The adherence to these protocols ensures the consistency and regulatory acceptability of the data that populate benchmark databases like ADORE.

Model Performance Comparison on Benchmark Data

ML models trained on ADORE data demonstrate the potential and current limitations of in silico toxicity prediction. A 2024 study using the ADORE "t-F2F" (fish-to-fish) challenge provides a direct comparison of model performance[reference:13].

Key Findings from Model Comparison:

• Best Performing Models: Tree-based ensemble methods (Random Forest and XGBoost) consistently outperformed linear models (LASSO) and Gaussian Process regression[reference:14].
• Prediction Accuracy: The best models achieved a Root Mean Square Error (RMSE) of approximately 0.90 for predicting log10-transformed LC50 values. This corresponds to predicting toxicity within an order of magnitude on the original scale[reference:15].
• Critical Factor - Data Splitting: Model performance was highly sensitive to how data was split for training and testing. A "split by chemical occurrence" (which avoids data leakage from repeated experiments) yielded more realistic and generalizable performance estimates than a simple random split[reference:16].
• Feature Importance: While molecular representations (e.g., fingerprints) were useful, models using only a few key chemical properties (molecular weight, water solubility, logP) performed nearly as well, suggesting these are dominant drivers in the models' predictions[reference:17].

Table: Representative Model Performance on ADORE Fish Challenge (Split by Occurrence)

Model Type	Typical RMSE (log10 LC50)	Key Characteristics
Random Forest / XGBoost	~0.90 - 1.0	Best overall performance; robust to non-linear relationships.
Gaussian Process	~1.0 - 1.1	Can incorporate phylogenetic distance; computationally intensive.
LASSO (Linear Model)	~1.1 - 1.2	Lowest performance; limited by linear assumption.

Performance summary based on Gasser et al. (2024)[reference:18][reference:19].

The Scientist's Toolkit: Essential Research Reagents and Materials

Conducting standardized ecotoxicity tests or curating ML-ready data requires a suite of specialized materials and model systems.

Table: Key Research Reagent Solutions in Ecotoxicology

Item	Function	Example/Note
Test Organisms	Provide the biological response for toxicity endpoint measurement.	Fish: Zebrafish (Danio rerio), Fathead minnow (Pimephales promelas). Invertebrate: Daphnia magna. Algae: Raphidocelis subcapitata.
Reference Toxicants	Validate test organism health and assay performance.	Potassium dichromate (for Daphnia), Sodium chloride (for fish).
Exposure Chambers	Hold test organisms and contaminated media under controlled conditions.	Glass aquaria, multi-well plates, flow-through systems.
Water Quality Reagents	Prepare and maintain standardized reconstituted water for tests.	Salts of Ca, Mg, Na, K; buffers to maintain pH and hardness.
Chemical Stock Solutions	Prepare precise exposure concentrations of the test substance.	Often dissolved in carrier solvents (e.g., acetone, DMSO) with appropriate solvent controls.
Data Curation Software	Harmonize, clean, and annotate experimental data from sources like ECOTOX for ML.	Python/R pipelines for data processing; phylogenetic tree software.
Molecular Representation Tools	Convert chemical structures into machine-readable features.	RDKit (for fingerprints), mol2vec, Mordred descriptor calculator.

Visualizing Workflows and Relationships

Diagram 1: Experimental workflow for a standardized aquatic acute toxicity test (e.g., OECD TG 203).

Diagram 2: Logical relationship between experimental toxicity metrics, benchmark datasets, and the machine learning modeling pipeline.

From Data to Model: Methodological Strategies for Applying Ecotoxicology Benchmarks

Molecular representation serves as the foundational bridge between chemical structures and their biological or toxicological effects, enabling machine learning (ML) to predict complex endpoints such as ecotoxicity [20]. In ecotoxicology, the accurate prediction of chemical hazards to aquatic life is critical for environmental protection and regulatory compliance, yet it presents unique challenges due to the vast chemical space and diverse biological targets [21]. Traditional Quantitative Structure-Activity Relationship (QSAR) models have long relied on hand-crafted molecular descriptors and fingerprints. However, the field is undergoing a transformation with the advent of deep learning methods that learn representations directly from molecular graphs or string notations [22] [20].

The evolution toward graph-based representations and learned embeddings promises to capture more nuanced structure-property relationships, which is essential for navigating the complex chemical space of environmental contaminants [20]. The performance of these representation paradigms is not universal; it is highly dependent on the specific task, dataset size, and chemical domain [23] [22]. Benchmarking their effectiveness requires standardized, high-quality datasets. In ecotoxicology, the recent introduction of the ADORE (Acute Aquatic Toxicity) dataset provides a crucial common ground for training, benchmarking, and comparing models, mirroring the role of established benchmarks like ImageNet in computer vision [21] [24]. This guide provides a comparative analysis of molecular representation methods, grounded in experimental performance data and framed within the imperative for robust benchmark datasets in ecotoxicological ML research.

Foundational Molecular Representation Methods

The translation of molecular structures into a computationally tractable format is achieved through several established classes of methods. These representations form the input feature space for predictive modeling in cheminformatics and ecotoxicology.

Molecular Descriptors and Fingerprints

Molecular descriptors are numerical quantities that capture a molecule's physicochemical properties (e.g., molecular weight, logP, polar surface area) or topological features [20]. They are often combined with molecular fingerprints, which are bit or count vectors encoding structural information.

Fingerprints are algorithmically generated and can be categorized by their design:

• Substructure-based (Key-based): Bits correspond to the presence or absence of predefined structural fragments (e.g., MACCS keys, PubChem fingerprints) [22] [25].
• Circular (Topological): Bits represent the presence of circular atomic neighborhoods generated by a hashing algorithm. Extended Connectivity Fingerprints (ECFP) are the most prominent example and are considered a de facto standard for drug-like molecules [22] [25].
• Path-based: Features are derived from all possible linear paths or atom pairs within the molecular graph (e.g., RDKit topological fingerprint, AtomPair fingerprint) [25].
• Pharmacophore-based: Encode spatial relationships between functional groups critical for molecular interaction (e.g., Pharmacophore Pairs fingerprints) [25].
• String-based: Operate directly on SMILES strings, treating them as sequences (e.g., LINGO, MinHashed fingerprints) [25].

The choice of fingerprint significantly influences the perceived similarity between molecules and, consequently, the performance of subsequent ML models. Studies show that different fingerprints can provide fundamentally different views of chemical space, especially for structurally diverse compounds like natural products [25].

Graph Representations and Learned Embeddings

Modern AI-driven methods bypass manual feature engineering by learning representations directly from raw molecular data.

• Graph Neural Networks (GNNs): These methods operate directly on the molecular graph, where atoms are nodes and bonds are edges. Through iterative message-passing operations, GNNs aggregate information from a node's local neighborhood to create informative atom-level and, ultimately, molecule-level embeddings [23] [20]. Popular architectures include Graph Convolutional Networks (GCN), Graph Attention Networks (GAT), and Message Passing Neural Networks (MPNN) [23].
• Language Model-Based Representations: Inspired by natural language processing, these methods treat simplified molecular input line entry system (SMILES) strings as a textual language. Models like Transformers can be trained to generate continuous molecular embeddings (e.g., Mol2vec) or used in an end-to-end fashion for property prediction [22] [20].
• Multimodal and Contrastive Learning: Emerging frameworks combine multiple representation types (e.g., graph + textual) or use self-supervised learning objectives to generate robust, transferable molecular embeddings without extensive labeled data [20].

Diagram: Evolution of Molecular Representation for Machine Learning

Table 1: Comparison of Traditional and AI-Driven Molecular Representation Methods

Representation Type	Key Examples	Core Principle	Advantages	Limitations	Typical Use Case
Molecular Descriptors	MOE descriptors, Mordred	Pre-defined numerical properties	Highly interpretable, computationally cheap	May miss complex structural patterns, requires domain knowledge	QSAR modeling, similarity search [20]
Molecular Fingerprints	ECFP, MACCS, PubChemFP	Binary/count vectors of structural fragments	Standardized, efficient, excellent for similarity	Hand-crafted, fixed resolution, may not be optimal for all tasks	Virtual screening, clustering [22] [25]
Graph Representations (GNN)	GCN, GAT, AttentiveFP	Message-passing on atom/bond graph	Learns task-specific features, captures topology	Computationally intensive, requires larger data, less interpretable	Property prediction, molecular generation [23] [20]
Language Model-Based	Mol2vec, SMILES-BERT	NLP techniques on SMILES/SELFIES	Can learn from unlabeled data, captures syntax	Dependent on SMILES robustness (e.g., stereochemistry)	Pre-training, transfer learning [22] [20]

Performance Comparison: Descriptors, Fingerprints, and Graph-Based Models

The relative performance of different representation paradigms is context-dependent, influenced by dataset size, task complexity, and the chemical domain. Comparative studies provide critical insights for method selection.

Comprehensive Benchmarking on Drug Discovery Datasets

A landmark 2021 study compared four descriptor-based models (SVM, XGBoost, RF, DNN using combined descriptors/fingerprints) against four graph-based models (GCN, GAT, MPNN, Attentive FP) across 11 public molecular property prediction datasets [23].

Key Findings:

• Overall Accuracy: On average, descriptor-based models (particularly SVM for regression, RF/XGBoost for classification) outperformed or matched graph-based models in prediction accuracy [23].
• Computational Efficiency: Random Forest and XGBoost were vastly more efficient, training in seconds, while GNNs required significantly more time and resources [23].
• Graph-Based Strengths: Certain GNNs, like Attentive FP and GCN, excelled on specific larger or multi-task datasets, suggesting their strength in capturing complex patterns when data is abundant [23].
• Interpretability: Descriptor-based models coupled with tools like SHAP (SHapley Additive exPlanations) offer clear interpretability, identifying influential chemical features—a significant advantage for scientific discovery [23].

Performance in Specialized Chemical Spaces

Performance can vary dramatically outside the domain of typical drug-like molecules. A 2024 study evaluated 20 fingerprint types on over 100,000 natural products (NPs), which have distinct structural motifs (e.g., more stereocenters, sp³ carbons) [25].

• No Universal Winner: No single fingerprint consistently outperformed others across all NP bioactivity prediction tasks. While ECFP is standard for drug-like compounds, other fingerprints (e.g., certain path-based or pharmacophore fingerprints) matched or surpassed it for NPs [25].
• Implication: The study underscores that fingerprint performance is not transferable across all chemical spaces. Optimal representation requires evaluation and selection specific to the chemical domain of interest, such as environmental contaminants in ecotoxicology [25].

Data Regime Dependency

The advantage of learned representations (e.g., from GNNs) is often tightly coupled to data availability.

• Low-Data Scenarios: Multiple studies conclude that traditional fingerprints tend to outperform learned representations when training data is scarce or highly imbalanced [22].
• Adequate/High-Data Scenarios: With sufficient data, end-to-end deep learning models can achieve comparable or superior performance by learning optimal, task-specific features directly from graphs or SMILES strings [22].

Table 2: Experimental Performance Summary from Key Comparative Studies

Study & Context	Top Performing Methods	Key Metric (Average/Representative)	Data & Task Details	Conclusion for Ecotoxicology
Drug Discovery Benchmark [23]	SVM (Regression), RF/XGBoost (Classification)	R²: ~0.8 (ESOL), AUC: ~0.78 (HIV)	11 datasets, 3 regression, 8 classification tasks. Descriptor-based models used 206 MOE descriptors + 1188 fingerprint bits.	For many tasks, robust traditional models with comprehensive descriptors are highly effective and efficient.
Graph-Based Models (same study) [23]	Attentive FP, GCN	Competitve on specific tasks (e.g., ClinTox, SIDER)	Same 11 datasets as above.	GNNs are powerful for complex or multi-task problems but require careful evaluation of cost-benefit.
Cancer Drug Sensitivity [22]	ECFP Fingerprints + FCNN, GNNs	Performance highly dataset-dependent.	5 cancer cell line screening datasets; compared fingerprints, Mol2vec, TextCNN, GNNs.	No single representation dominates; ensemble methods can improve performance. Data size is a critical factor.
Natural Product Bioactivity [25]	Varied by dataset (ECFP not always best)	AUC ranges from 0.70 to 0.95 across 12 tasks.	100,000+ NPs from COCONUT/CMNPD; benchmarked 20 fingerprint types.	Chemical domain dictates optimal fingerprint. Ecotox models for unique contaminants (e.g., pesticides, PFAS) need similar evaluation.

Diagram: Experimental Protocol for Benchmarking Molecular Representations

Benchmark Datasets for Ecotoxicology Machine Learning

The advancement and reliable comparison of ML models in ecotoxicology depend on standardized, high-quality benchmark datasets. The ADORE dataset represents a significant effort to fulfill this need [21] [24].

The ADORE Dataset: Construction and Scope

ADORE (A Benchmark Dataset for Machine Learning in Ecotoxicology) is curated specifically to serve as a common benchmark for predicting acute aquatic toxicity [24].

Core Data Source and Processing:

• Source: Data is extracted from the U.S. EPA's ECOTOX database, focusing on three ecologically relevant taxonomic groups: fish, crustaceans, and algae [24].
• Endpoints: It includes acute lethal and sub-lethal effects (e.g., mortality, immobilization, growth inhibition) with standardized metrics like LC50 and EC50, primarily for exposures up to 96 hours [24].
• Curation: Rigorous processing includes structure standardization, removal of in vitro and early life-stage data to focus on whole-organism toxicity, and careful handling of replicate entries to prevent data leakage [24].

Integrated Feature Engineering for Ecotoxicology

A key strength of ADORE is its integration of multifaceted features beyond just chemical structure:

• Chemical Representations: Provides multiple molecular representations for each compound, including four fingerprints (PubChem, MACCS, Morgan, ToxPrints), the Mol2vec embedding, and a comprehensive set of Mordred descriptors [21]. This allows direct benchmarking of representation impact.
• Biological Context: Incorporates species-specific features, including phylogenetic information (capturing evolutionary relatedness), ecological traits, and life-history data. This is crucial because toxicity is influenced by the biological target [21] [24].
• Experimental Metadata: Includes details on test conditions, endpoint type, and exposure duration.

Defined Challenges and Splits

To ensure fair comparison, ADORE provides predefined dataset splits and proposes specific modeling challenges [24]:

• Preventing Data Leakage: Offers splits based on chemical occurrence and molecular scaffolds to avoid overly optimistic performance from training and testing on structurally highly similar compounds [21].
• Modeling Challenges: Includes tasks like predicting toxicity for fish using data from algae and invertebrates, or optimizing performance for a broad set of species versus a single model species [21].

Diagram: Structure and Construction of the ADORE Ecotoxicology Benchmark Dataset

Table 3: Key Benchmark Datasets for Molecular Representation in Ecotoxicology & Related Fields

Dataset Name	Focus & Endpoint	Chemical Scope	Integrated Representations	Key Purpose & Challenge
ADORE [21] [24]	Acute aquatic toxicity (LC50/EC50)	~12,000 chemicals, Fish/Crustaceans/Algae	4 Fingerprints, Mol2vec, Mordred Descriptors, Phylogenetic features	Benchmarking ML models; predicting toxicity across species & chemicals.
MoleculeNet [23]	Broad molecular property prediction	Drug-like molecules, various sizes	Primarily graph inputs & ECFP fingerprints	General benchmark for drug discovery ML models.
NCI/Cancer Cell Line [22]	Drug sensitivity (pIC50/GI50)	Anti-cancer compounds	ECFP, MACCS, AtomPair, GNN inputs	Benchmark for representations in drug response prediction.
COCONUT/CMNPD (for NPs) [25]	Bioactivity classification	>100,000 Natural Products	20 evaluated fingerprint types	Benchmarking fingerprints for structurally complex natural products.

Implementing and benchmarking molecular representation methods requires a suite of specialized software tools and data resources.

Table 4: Key Research Reagent Solutions for Molecular Representation and Benchmarking

Tool/Resource Name	Type	Primary Function in Research	Relevance to Ecotoxicology
RDKit	Open-Source Cheminformatics Library	Core functions for reading molecules, calculating descriptors, generating fingerprints (e.g., Morgan/ECFP), and handling SMILES [23] [25].	Fundamental for processing environmental chemical structures and generating input features.
DeepChem	Deep Learning Library for Chemistry	Provides implementations of graph neural networks (GNNs), molecular featurizers, and standard dataset loaders for MoleculeNet [22].	Enables building and testing state-of-the-art deep learning models on toxicity data.
SHAP (SHapley Additive exPlanations)	Model Interpretability Library	Explains output of any ML model by assigning importance values to each input feature [23].	Critical for interpreting descriptor-based toxicity models and gaining mechanistic insights.
ADORE Dataset	Benchmark Data Resource	Provides curated acute aquatic toxicity data with multiple molecular representations and species features [21] [24].	The central benchmark for developing and comparing ecotoxicity ML models.
Mordred Descriptor Calculator	Molecular Descriptor Software	Calculates a comprehensive set (≈1,800) of 2D/3D molecular descriptors directly from structures [21].	Generates extensive chemical feature sets for traditional QSAR or hybrid ML models in ecotox.
ECOTOX Database	Primary Data Source	EPA database containing experimental toxicity results for chemicals across species [24].	The primary source for curating new, specialized ecotoxicology datasets beyond ADORE.
PubChem	Chemical Information Resource	Provides canonical SMILES, structural information, and bioactivity data for millions of compounds [24].	Essential for retrieving and verifying chemical structures based on CAS or other identifiers.

The effective representation of chemical space is a dynamic field balancing the computational efficiency and interpretability of traditional descriptors/fingerprints against the representational power and flexibility of AI-driven graph and learned representations [23] [20]. For ecotoxicology machine learning, no single method is universally superior. The choice depends on the specific problem, the amount and quality of available data, and the need for interpretability.

The emergence of standardized benchmark datasets like ADORE is a pivotal development, enabling rigorous comparison and driving progress in the field [21] [24]. Future research directions likely involve:

• Hybrid Models: Combining the strengths of different representations (e.g., graph-based features with phylogenetic species data) in multimodal architectures [20].
• Explainable AI: Developing methods to interpret complex graph-based models applied to toxicity prediction, a necessity for regulatory acceptance.
• Domain-Specific Representation Learning: Tailoring representation learning techniques to the unique chemical space of environmental pollutants (e.g., pesticides, industrial chemicals, transformation products).

By leveraging comprehensive benchmarks and selecting molecular representations informed by comparative performance data, researchers can build more reliable, transparent, and effective models to predict chemical hazards and support environmental safety assessments.

The Imperative for Benchmarking in Computational Ecotoxicology

The regulation of chemicals to protect environmental and human health presents a monumental challenge. With over 350,000 chemicals and mixtures currently registered for use globally and more than 200 million substances cataloged, comprehensive experimental hazard assessment is an insurmountable task, both ethically and financially [1]. Traditional in vivo ecotoxicity testing, mandated by regulations like the EU's REACH, consumes substantial resources, with an estimated 440,000 to 2.2 million fish and birds used annually at a cost exceeding $39 million [1].

This crisis has accelerated the search for reliable in silico alternatives. While Quantitative Structure-Activity Relationship (QSAR) models have a long history, they are often limited to chemical descriptors and simple, explainable architectures [2]. Modern machine learning (ML) promises to integrate diverse data types—including chemical properties, species biology, and experimental conditions—to build more powerful predictive models [3]. However, the field has been hampered by a lack of standardization, making it difficult to compare model performance across studies and objectively assess progress [1].

The solution, successfully adopted in fields like computer vision (e.g., ImageNet) and hydrology (e.g., CAMELS), is the establishment of community-accepted benchmark datasets [1] [2]. A benchmark dataset provides a common, well-curated, and publicly available ground for training and testing models, ensuring that performance comparisons are fair and meaningful. In ecotoxicology, this need is met by the ADORE (Acute Aquatic Toxicity) dataset, a comprehensive resource designed to foster ML adoption and rigorous model evaluation [1] [26].

Comparative Analysis of Modeling Paradigms

The ADORE benchmark enables a direct comparison of different computational approaches, from traditional methods to cutting-edge artificial intelligence. The table below summarizes the core characteristics, strengths, and limitations of three dominant paradigms.

Table: Comparison of Computational Modeling Paradigms in Ecotoxicology

Aspect	Traditional QSAR	Standard Machine Learning (on ADORE)	Advanced Graph-Based Learning (on ADORE)
Core Philosophy	Predict toxicity based on linear/non-linear relationships between a few chemical structural properties and activity [1].	Learn complex patterns from high-dimensional feature sets representing both chemicals and species.	Directly learn from the graph structure of molecules, integrating chemical topology with other data.
Typical Inputs	Limited chemical descriptors (e.g., logP, molecular weight) [27].	Chemical fingerprints/descriptors (e.g., Morgan, Mordred) AND species traits/phylogeny [1] [3].	Molecular graph (atoms as nodes, bonds as edges) combined with other feature vectors [5].
Model Examples	ECOSAR, linear regression [27].	Random Forest (RF), Support Vector Machine (SVM), eXtreme Gradient Boosting (XGB), Deep Neural Networks (DNN) [5].	Graph Convolutional Network (GCN), Graph Attention Network (GAT), Message Passing Neural Network (MPNN) [5].
Key Strength	High interpretability, regulatory familiarity, and low computational cost.	Ability to handle diverse, high-dimensional data and capture non-linear interactions.	Superior representation of intrinsic molecular structure; state-of-the-art predictive performance.
Primary Limitation	Limited predictive scope and accuracy; cannot integrate biological complexity of test species.	Can be a "black box"; performance may plateau on highly complex tasks like cross-species prediction.	High computational demand; requires significant expertise to implement and tune.
Performance on ADORE (Example - Fish)	Not benchmarked on full ADORE. Outperformed by ML in similar tasks [27].	RF, XGB, and DNN show strong performance but with notable errors for specific species [26].	GCN achieves best overall performance, with AUC >0.98 for same-species prediction [5].

A 2025 comparative study leveraging ADORE constructed 161 distinct models, systematically testing combinations of molecular representations and algorithms [5]. The results clearly demonstrate the evolution of the field: Graph Convolutional Networks (GCNs) consistently achieved the highest performance for predicting toxicity within a single species (e.g., fish-to-fish prediction). However, all models faced significant challenges in cross-species extrapolation (e.g., predicting fish toxicity from crustacean and algae data), where even the best models saw performance drop by approximately 17% in AUC [5]. This highlights that incorporating biological complexity remains a critical, unsolved problem.

The ADORE Benchmark: Architecture for Biological Complexity

The ADORE dataset is engineered to directly address the challenge of incorporating biological context into ML models [1]. Its construction is a multi-source integration process, and its structure provides researchers with the necessary data to test hypotheses about species traits and phylogenetic relationships.

Dataset Compilation and Core Structure

ADORE is built around a core of acute aquatic toxicity data extracted from the US EPA's ECOTOX knowledgebase [1]. It focuses on three ecologically and regulatory-relevant taxonomic groups: fish, crustaceans, and algae. The data is meticulously filtered to include standard test durations and mortality-related endpoints (LC50/EC50) to ensure comparability [1].

The true innovation lies in the curated expansion of this core with two layers of contextual data:

• Chemical Diversity: Nearly 2,000 chemicals are represented via multiple identifiers (CAS, InChIKey, SMILES) and an array of molecular representations (e.g., Morgan fingerprints, Mordred descriptors) to facilitate various modeling approaches [1] [5].
• Biological Complexity: This includes species-specific traits (e.g., habitat, life history) and, most importantly, quantitative phylogenetic data that encodes the evolutionary distance between species [1] [3].

The following diagram illustrates the workflow for compiling the ADORE benchmark dataset.

Experimental Splitting Strategy and Defined Challenges

A critical contribution of ADORE is its rigorous approach to preventing data leakage—a common flaw where overly optimistic performance is achieved because similar data appears in both training and test sets [2] [3]. ADORE provides pre-defined data splits based on chemical occurrence and molecular scaffolds, ensuring models are tested on truly novel chemicals or species [1].

The dataset is structured into specific challenges of varying complexity:

• Single-Species Challenges (F2F, C2C, A2A): Predicting toxicity for a well-represented species like Rainbow Trout (Oncorhynchus mykiss) or Water Flea (Daphnia magna) [5].
• Taxon-Level Challenges: Predicting toxicity across all species within a taxonomic group (e.g., all fish) [2].
• Cross-Taxon Extrapolation Challenges (AC2F): The most demanding test, where models trained on algae and crustacean data must predict toxicity for fish [5]. This is key for reducing vertebrate testing.

Incorporating Phylogeny: From Concept to Feature

The inclusion of phylogenetic data is a cornerstone of ADORE's design for biological complexity. The underlying hypothesis is that evolutionarily related species share similar physiological and biochemical pathways, leading to correlated sensitivities to chemicals [2] [3].

In practice, a phylogenetic tree is used to calculate a pairwise distance matrix between all species in the dataset. This matrix quantifies the evolutionary divergence, often in millions of years. These distances can be used directly as features or to inform model architecture, encouraging the model to attribute more similar predictions to closely related species. The following diagram conceptualizes this approach.

Experimental Protocol: Benchmarking a Model on ADORE

To ensure reproducible and comparable research, the following protocol details the steps for a standard benchmarking experiment using the ADORE dataset.

Objective: To train and evaluate a machine learning model for predicting acute aquatic toxicity (LC50/EC50), comparing its performance on intra-species versus cross-species prediction tasks.

Materials & Data:

• Primary Data: The ADORE dataset, downloaded from its public repository [1].
• Software: Python (>=3.8) with standard data science libraries (pandas, numpy, scikit-learn) and relevant deep learning frameworks (e.g., PyTorch, TensorFlow) or graph learning libraries (e.g., PyTorch Geometric).
• Computational Resources: A standard computer is sufficient for classical ML. Graph Neural Networks (GNNs) may require a GPU for efficient training.

Procedure:

• Challenge Selection: Choose a specific ADORE challenge (e.g., F2F for fish, AC2F-diff for cross-taxon prediction).
• Data Loading: Load the pre-defined training and test splits for the chosen challenge. Crucially, do not modify these splits to maintain benchmark integrity [1].
• Feature Selection: Choose input feature sets. A robust experiment should test multiple representations:
  • Chemical: Morgan fingerprints, Mordred descriptors, or molecular graphs for GNNs.
  • Biological: Incorporate columns for species traits and/or the phylogenetic distance matrix.
• Model Training: Train the selected model (e.g., Random Forest, GCN) on the training set. Use the training set's provided toxicity values as the target variable. Implement standard techniques like cross-validation on the training set for hyperparameter tuning.
• Model Evaluation: Use the held-out, unseen test set to evaluate the final model. Report standard performance metrics:
  • For regression (predicting numeric LC50): Root Mean Square Error (RMSE), Coefficient of Determination (R²).
  • For classification (predicting toxicity class): Area Under the ROC Curve (AUC), Accuracy, Precision, Recall [5] [28].
• Interpretation & Analysis: Use explainable AI (XAI) techniques like SHAP (Shapley Additive Explanations) to analyze which features (chemical or biological) were most influential in the model's predictions [28]. Compare performance across different challenges to assess the model's ability to generalize.

Table: Key Research Reagent Solutions & Resources for Computational Ecotoxicology

Resource Name	Type	Primary Function in Research	Key Feature for Biological Complexity
ADORE Dataset [1]	Benchmark Dataset	Provides a standardized, multi-feature dataset for training and fairly comparing ML models in aquatic ecotoxicology.	Integrates species trait data and quantitative phylogenetic distances alongside chemical data.
ECOTOX Knowledgebase [1]	Primary Data Repository	The US EPA's curated database containing millions of ecotoxicity test results from the literature.	Source of raw toxicity endpoints and test species information, forming the core of derived datasets.
CompTox Chemicals Dashboard [1]	Chemical Data Hub	Provides access to chemical structures, properties, identifiers, and related data for thousands of substances.	Enables the expansion of chemical feature sets (e.g., for obtaining SMILES strings, calculated descriptors).
Mordred/Morgan Fingerprints [5]	Molecular Representation	Translates chemical structure into numerical vectors or bitstrings that ML models can process.	Captures intrinsic chemical properties that interact with biological systems; a prerequisite for modeling.
Phylogenetic Trees (e.g., from TimeTree) [3]	Biological Data	Diagrams representing the evolutionary relationships among species based on genetic data.	The foundation for calculating pairwise phylogenetic distance matrices, used as model input to encode evolutionary relatedness.
SHAP (Shapley Additive Explanations) [28]	Explainable AI (XAI) Library	A game theory-based method to explain the output of any ML model, attributing prediction to input features.	Critical for interpreting how both chemical descriptors and biological traits (phylogeny) contribute to a model's toxicity prediction.

The ADORE benchmark dataset represents a paradigm shift, enabling the ecotoxicology community to move beyond isolated studies toward cumulative, comparable progress in computational prediction [1] [26]. Empirical results demonstrate that models incorporating chemical and biological complexity, particularly advanced graph-based learning, achieve superior predictive accuracy for well-represented tasks [5].

However, significant challenges persist. The "cross-species prediction gap" remains substantial, indicating that current feature sets and models do not fully capture the mechanistic drivers of species-specific sensitivity [5]. Furthermore, even the best models can show high error for individual species, likely because they are biased by dominant chemical features and fail to learn nuanced biological interactions [26].

The path forward requires a dual focus: 1) developing more sophisticated methods to integrate mechanistic biological knowledge (e.g., from toxicogenomics or pathway analysis) into model architectures, and 2) rigorous external validation and integration with in vitro alternative methods (like fish cell line assays) to build confidence for regulatory application [26]. By providing a common foundation, ADORE not only benchmarks where we are but also clearly illuminates the critical research frontiers for making computational ecotoxicology truly predictive and protective.

The application of machine learning (ML) in ecotoxicology promises a revolution in chemical hazard assessment, offering pathways to reduce costly and ethically challenging animal testing [2]. However, the field's progress has historically been hampered by a lack of standardized data, making direct comparison of model performance across studies difficult and hindering reproducibility [3]. The recent introduction of benchmark datasets, like those common in computer vision (e.g., ImageNet) or hydrology (e.g., CAMELS), provides a critical foundation for objective advancement [1].

Central to this thesis is the ADORE (Aquatic Toxicity Benchmark Dataset). ADORE is an extensive, well-curated dataset focused on acute aquatic toxicity for three ecologically and regulatory-relevant taxonomic groups: fish, crustaceans, and algae [1]. It was created to lower the barrier of entry for ML experts into ecotoxicology by providing a pre-processed, well-described common ground for model training, benchmarking, and comparison [2] [29]. The dataset aggregates ecotoxicological outcomes from the US EPA's ECOTOX database and enriches them with detailed chemical properties (e.g., multiple molecular fingerprints like Morgan and PubChem) and species-specific biological features (e.g., phylogenetic data, life-history traits) [1] [3]. Crucially, ADORE provides predefined data splits to prevent data leakage—a common pitfall where similar experimental results appear in both training and test sets, leading to inflated and non-generalizable performance metrics [2] [3].

This comparison guide is framed within the thesis that benchmark datasets like ADORE are indispensable for rigorously evaluating and advancing modeling paradigms. We objectively compare two advanced paradigms—Pairwise Learning and Graph Neural Networks (GNNs)—by examining their methodological approaches, experimental performance on ecotoxicological tasks, and practical utility for researchers and risk assessors.

Paradigm Comparison: Methodologies and Applications

The following table summarizes the core principles, representative techniques, and primary applications of the two modeling paradigms within ecotoxicology.

Table 1: Comparison of Modeling Paradigms for Ecotoxicology

Aspect	Pairwise Learning	Graph Neural Networks (GNNs)
Core Mathematical Principle	Models the interaction between two entities (e.g., chemical & species) as a matrix completion or factorization problem. Treats the sparse matrix of observed outcomes as a learning target [30].	Operates directly on graph-structured data. Learns node representations by iteratively aggregating features from neighboring nodes, capturing topological relationships [31] [32].
Representative Technique	Bayesian Factorization Machines (Bayesian FM): Decomposes the interaction matrix into latent factor vectors for chemicals and species, learning a global function: y(x) = w₀ + Σwᵢxᵢ + ΣΣxᵢxⱼ〈vᵢ, vⱼ〉 [30].	Heterogeneous GNNs (e.g., R-GCN, HGT): Specialized architectures for knowledge graphs with multiple node/edge types (e.g., Chemical, Gene, Pathway). Use relation-specific weights to aggregate information [31].
Primary Data Structure	Symmetric interaction matrix (Chemicals × Species).	Molecular graphs (atoms as nodes, bonds as edges) or heterogeneous knowledge graphs [31].
Key Advantage	Excels at data gap filling for massively sparse matrices. Naturally captures the unique "lock-and-key" interaction between a specific chemical and a specific species [30].	Integrates multi-scale biological context (e.g., pathway information) beyond chemical structure. Provides a natural framework for mechanistic interpretability [31].
Typical Ecotoxicology Application	Predicting toxicity (LC50/EC50) for millions of untested chemical-species pairs to construct comprehensive hazard heatmaps and species sensitivity distributions [30].	Classifying molecular toxicity (e.g., Tox21 assays) or predicting toxic endpoints by leveraging biological knowledge graphs [31] [33].
Interpretability	Medium. Importance of latent factors for chemicals/species can be analyzed, but the "black-box" interaction term is complex.	Potentially High. Attention mechanisms can highlight important sub-structures or biological pathways relevant to the prediction [31].

Experimental Performance on Benchmark Tasks

Empirical studies demonstrate the strengths of each paradigm on specific tasks defined by benchmark datasets like ADORE and Tox21. The quantitative results below are drawn from published experiments.

Table 2: Experimental Performance Comparison

Study & Paradigm	Dataset & Task	Key Metric & Performance	Comparative Insight
Pairwise Learning (Bayesian FM) [30]	ADORE subset: Predicting LC50 for 3295 chemicals × 1267 species (0.5% data coverage).	R² on test set: ~0.65 – 0.70 (Pairwise Model vs. Mean Model).	The pairwise interaction model significantly outperformed a model using only average chemical and species effects, validating the importance of capturing specific chemical-species interactions [30].
GNN with Knowledge Graph (GPS Model) [31]	Tox21: 12 toxicity classification tasks (e.g., nuclear receptor assays).	Average AUC-ROC: 0.956 (for key tasks like NR-AR).	A heterogeneous GNN (GPS) enriched with a toxicological knowledge graph (ToxKG) outperformed traditional GNNs using only molecular fingerprints, highlighting the value of incorporating biological mechanism data [31].
Advanced GNN Benchmarking [33]	Toxicology molecule classification dataset.	AUC-ROC: 0.816 (using Graph Isomorphic Network with Few-Shot Learning).	This represented an 11.4% improvement over a baseline Graph Convolutional Network (GCN), demonstrating how advanced GNN architectures and training strategies can address data limitations [33].
Stable-GNN (S-GNN) [32]	Various graph datasets under Out-of-Distribution (OOD) shifts.	Performance Drop: Reduced degradation compared to standard GNNs.	Designed to improve generalization to unseen data distributions by decorrelating spurious features, addressing a key challenge in applying models to new chemical spaces [32].

Detailed Experimental Protocols

To ensure reproducibility and clarity, this section outlines the detailed methodologies for two key experiments cited in the performance comparison.

Protocol 1: Pairwise Learning for LC50 Prediction on ADORE

This protocol is based on the work of [30], which applied Bayesian Factorization Machines to the ADORE dataset.

• Objective: To predict missing acute toxicity (LC50) values for all possible combinations of 3295 chemicals and 1267 species, filling a matrix that is 99.5% sparse.
• Data Source & Preprocessing:
  • Core data was extracted from the ecotox_mortality_processed.csv file of the ADORE dataset [30].
  • The input matrix was constructed with chemicals and species as categorical variables. Each unique (chemical, species, exposure duration) triplet defined a data point.
  • LC50 values were log10-transformed (mol/L) to normalize the data. Repeated experiments for the same triplet were kept separate to characterize biological variability.
• Model & Training:
  • Model: A second-order Factorization Machine (FM) implemented using the libfm library with Markov Chain Monte Carlo (MCMC) inference [30].
  • Feature Encoding: One-hot encoding for chemical ID, species ID, and exposure duration (24, 48, 72, 96 hours). The feature vector x is a sparse binary vector with only three active entries.
  • Equation: The predicted log(LC50) is given by: y(x) = w₀ + Σwᵢxᵢ + ΣΣ xᵢxⱼ Σ vᵢ,ₖ vⱼ,ₖ where w₀ is the global bias, wᵢ are weights for main effects, and v are latent factor vectors modeling pairwise interactions [30].
  • Training Setup: The model was run for 2000 epochs with 32 latent factors. Multiple model configurations were compared: a Null model (global mean), a Mean model (chemical/species biases), and the full Pairwise model (including interactions).
• Output & Validation:
  • The model generates over 4 million predicted LC50 values for the previously empty matrix cells.
  • Predictive accuracy was validated using hold-out test sets, with performance measured by R² and root-mean-square error (RMSE) against the observed values.

Protocol 2: GNN with Knowledge Graph for Toxicity Classification

This protocol is based on the study by [31], which integrated a toxicological knowledge graph with GNNs for the Tox21 challenge.

• Objective: To improve the accuracy and interpretability of multi-task toxicity classification by incorporating biological pathway and gene interaction information.
• Data Source & Preprocessing:
  • Tox21 Dataset: Contains assay results for ~8,000 compounds across 12 toxicity targets. Compounds without definitive labels were filtered.
  • Toxicological Knowledge Graph (ToxKG): Constructed by extending the ComptoxAI graph. It integrates entities (Chemical, Gene, Pathway, Assay) and relationships from PubChem, Reactome, and ChEMBL databases [31]. The final graph contained 19,446 chemical and 17,517 gene nodes.
  • Data Integration: Tox21 compounds were mapped to ToxKG via PubChem IDs. Molecular graphs were generated from SMILES strings. Node features for chemicals included both molecular fingerprints (e.g., ECFP4) and knowledge graph embeddings.
• Model & Training:
  • Models Evaluated: Six GNN models were systematically compared: homogeneous (GCN, GAT) and heterogeneous (R-GCN, HRAN, HGT, GPS).
  • Architecture: The GPS (Graph Positioning System) model, which performed best, typically combines local message passing with global attention mechanisms to integrate information from the knowledge graph.
  • Training Setup: Training used a binary cross-entropy loss with class re-weighting to handle imbalanced data. The models were trained to predict activity for each of the 12 Tox21 assays simultaneously (multi-task learning).
• Output & Validation:
  • Predictions are binary classification outputs (active/inactive) for each toxicity assay.
  • Performance was evaluated using area under the receiver operating characteristic curve (AUC-ROC), F1-score, and accuracy. The GPS model's AUC-ROC of 0.956 for the NR-AR assay significantly outperformed baseline models [31].

Visualizing Workflows and Architectures

The following diagrams illustrate the logical workflow of the ADORE benchmark dataset creation and the contrasting architectures of the two modeling paradigms.

ADORE Benchmark Dataset Creation Workflow

Comparison of Pairwise Learning and GNN Modeling Pathways

This table details key software, data, and methodological resources essential for conducting research in machine learning for ecotoxicology, as featured in the discussed studies.

Table 3: Essential Research Toolkit for Ecotoxicology ML

Tool / Resource Name	Type	Primary Function in Research	Key Reference / Source
ADORE Dataset	Benchmark Data	Provides a standardized, curated dataset of aquatic toxicity for fish, crustaceans, and algae with chemical and species features, enabling direct model comparison.	[1] [2]
ECOTOX Database	Primary Data Source	The US EPA's comprehensive knowledgebase for single-chemical toxicity data for aquatic and terrestrial life, serving as the core source for curated benchmarks.	[1]
Tox21 Dataset	Benchmark Data	A public dataset of ~12,000 compounds tested in high-throughput assays against 12 nuclear receptor and stress response targets, standard for computational toxicology.	[31]
libfm	Software Library	A library for learning Factorization Machines, enabling efficient implementation of pairwise learning and matrix factorization models.	[30]
ComptoxAI / ToxKG	Knowledge Graph	A structured toxicological knowledge base integrating chemicals, genes, pathways, and assays. Used to provide biological context to ML models.	[31]
Graph Neural Network Libraries (e.g., PyTorch Geometric, DGL)	Software Framework	Specialized libraries that provide building blocks for implementing and training GNN models on graph-structured data like molecules.	[31] [32]
Molecular Fingerprints (e.g., ECFP4, Morgan)	Chemical Representation	Algorithms to convert molecular structures into fixed-length bit vectors that encode chemical features, usable as input for many ML models.	[1] [31]
Phylogenetic Distance Matrices	Biological Feature	Quantitative representations of evolutionary relationships between species, used as a feature to infer similarity in toxicological sensitivity.	[2] [3]
Predefined Data Splits (Scaffold/Chemical Splitting)	Methodological Protocol	Strategies to split datasets ensuring chemicals in the test set are structurally distinct from those in training. Critical for evaluating real-world generalization and avoiding data leakage.	[1] [2]

The evolution from traditional QSAR models to advanced paradigms like Pairwise Learning and Graph Neural Networks marks significant progress in computational ecotoxicology. As evidenced by their performance on benchmarks like ADORE and Tox21, each paradigm offers distinct advantages: pairwise learning excels at the pragmatic task of filling vast data gaps to enable comprehensive hazard assessment [30], while GNNs, particularly when integrated with knowledge graphs, offer a powerful path toward more accurate and mechanistically informed predictions [31].

The foundational thesis that standardized benchmarks are indispensable is strongly supported. The ADORE dataset has already enabled rigorous comparisons and demonstrated the value of controlled data splitting to prevent over-optimistic results [3]. Future progress in the field hinges on the continued development and adoption of such benchmarks, encouraging models that generalize well to novel chemicals and species. Promising research directions include the development of stable GNNs that are robust to distributional shifts [32], the integration of few-shot learning techniques to tackle data scarcity [33], and the deeper fusion of biologically grounded knowledge graphs with deep learning architectures. For researchers and regulators, the combined use of these paradigms—leveraging pairwise learning for broad-scale hazard screening and GNNs for in-depth mechanistic analysis—presents a powerful toolkit for achieving the goals of safe and sustainable chemical design.

This comparison guide objectively evaluates benchmark datasets and computational tools designed to accelerate ecotoxicological hazard assessment and support Safe and Sustainable by Design (SSbD) frameworks. The analysis is framed within the critical need for standardized, high-quality data to ensure reproducibility and meaningful comparison in machine learning research for ecotoxicology.

Comparative Analysis of Key Ecotoxicology Datasets and Tools

The following tables provide a structured comparison of the scope, design, and utility of major data resources for computational ecotoxicology.

Table 1: Comparison of Core Ecotoxicological Benchmark Datasets

Feature	ADORE (A benchmark dataset for ML in ecotoxicology) [1] [2] [3]	ECOTOX Knowledgebase [1] [34] [35]	EnviroTox Database [1]
Primary Purpose	Serve as a standardized benchmark for comparing ML model performance in predicting aquatic toxicity [1] [2].	A comprehensive, curated knowledgebase of single-chemical toxicity tests for ecological risk assessment [1] [35].	Support ecological Threshold of Toxicological Concern (eco-TTC) analysis and risk assessment [1].
Data Source	Curated subset of the ECOTOX database (September 2022 release), expanded with chemical and species features [1].	Aggregates toxicity data from peer-reviewed literature, government reports, and other sources [1].	A curated, high-quality subset of aquatic toxicity studies traceable to original sources [1].
Taxonomic Focus	Three aquatic groups: Fish, Crustaceans, Algae [1] [3].	Aquatic and terrestrial species [34] [35].	Aquatic species [1].
Key Endpoints	Acute mortality & comparable endpoints (LC50/EC50 for fish, crustaceans, algae) [1].	Wide range of lethal and sublethal effects, endpoints, and exposure durations [1].	Primarily lethal endpoints (LC50/EC50) for eco-TTC derivation [1].
ML-Ready Features	Yes. Includes molecular representations (fingerprints, Mordred descriptors, mol2vec), phylogenetic distances, species life-history traits, and predefined data splits [1] [2].	No. Provides raw experimental data; requires significant processing and feature engineering for ML [1].	Limited. Primarily a curated collection of toxicity values; not packaged with extended ML features [1].
Defined Challenges & Splits	Yes. Provides fixed training/test splits based on chemical scaffolds and species to prevent data leakage and proposes specific prediction challenges [1] [3].	No.	No.

Table 2: Comparison of Predictive Modeling Tools and Data Sources

Tool / Resource	TEST (Toxicity Estimation Software Tool) [36]	EPA CompTox Chemicals Dashboard [34]	ToxCast/Tox21 High-Throughput Screening (HTS) [34] [37]
Type	Standalone QSAR prediction software [36].	Integrative web-based chemistry resource and data hub [34].	In vitro high-throughput screening bioactivity data [34] [37].
Prediction Method	Multiple QSAR methodologies (hierarchical, group contribution, consensus, etc.) [36].	Provides access to data and models; does not make single, unified predictions itself.	Uses assay data to identify bioactivity pathways and potential mechanisms [37].
Key Ecotoxicity Endpoints	Fathead minnow LC50, Daphnia magna LC50 [36].	Provides access to multiple toxicity data sources (e.g., ECOTOX, ToxValDB) [34].	Pathway-based bioactivity for endocrine disruption, hepatotoxicity, etc. [37].
Utility for ML	Serves as a traditional QSAR baseline for comparison with newer ML models [36].	Critical data source. Provides curated chemical identifiers, structures, properties, and linked toxicity data for feature generation [1] [34].	Used as biological feature input for predicting in vivo toxicity or as an alternative data source for data-poor chemicals [37].
Core Strength	Easy-to-use, transparent methodology for estimating toxicity from chemical structure alone [36].	Centralized access to chemistry, exposure, and toxicity data for thousands of chemicals [34].	Provides mechanistic, human-health-relevant bioactivity data at scale, reducing animal testing [34] [37].

Experimental Protocols for Benchmark Dataset Construction

The development of robust ML benchmarks requires meticulous data curation and processing protocols. The methodology for creating the ADORE dataset exemplifies this rigorous approach [1].

Data Sourcing and Core Extraction

The core ecotoxicological data was extracted from the ECOTOX database (September 2022 release) [1]. The initial filter selected entries for three taxonomic groups: fish, crustaceans, and algae, which represent ecologically relevant trophic levels and a significant portion (41%) of available aquatic data [1]. The focus was on acute lethal or analogous effects:

• Fish: Mortality (MOR) after ≤96h exposure (per OECD Test Guideline 203) [1].
• Crustaceans: Mortality (MOR) or immobilization/intoxication (ITX) after ≤48h exposure (per OECD TG 202) [1].
• Algae: Effects on population growth, mortality, or physiology after ≤72-96h exposure (per OECD TG 201) [1]. Only data for the standard median lethal/effect concentration (LC50/EC50) endpoints were retained, expressed in molar units (mol/L) for biological relevance [1]. In vitro tests and assays using early life stages (e.g., fish embryos) were excluded to maintain a focus on whole-organism, apical endpoints [1].

Data Harmonization and Curation

A multi-stage processing pipeline was implemented [1]:

• Identifier Harmonization: Chemical entries were matched and assigned unique identifiers (InChIKey, DTXSID, CAS RN) using the US EPA CompTox Chemicals Dashboard to ensure consistency across data sources [1].
• Species Classification: Taxonomic information was validated and cleaned. Species were filterd based on the ecotox_group field and taxonomic columns to retain only the three target groups [1].
• Experimental Condition Filtering: Entries with non-standard or irrelevant experimental media (e.g., sediment tests) were removed to maintain a focus on water exposure [1].
• Representation Augmentation:
  • Chemical Representations: Six molecular representations were added for each chemical: MACCS, PubChem, Morgan, and ToxPrint fingerprints; Mordred descriptors; and mol2vec embeddings [1] [2].
  • Species Representations: Phylogenetic distance matrices were computed to quantify evolutionary relationships. Available species-level ecological and life-history traits (e.g., trophic level, habitat) were appended [1] [3].

Train-Test Splitting Strategy to Prevent Data Leakage

A critical step was defining rigorous data splits for model validation [2] [3]. Simple random splitting was deemed inappropriate due to the presence of multiple experimental records (replicates) for the same chemical-species pair, which would lead to data leakage and inflated performance metrics [1]. The following split strategies were implemented and provided as part of the dataset [1]:

• Strictly Chemical Scaffold-Based Split: All data points for chemicals sharing a common Bemis-Murcko scaffold are placed entirely in either the training or test set. This tests a model's ability to extrapolate to entirely new chemical structures [1].
• Leave-One-Chemical-Out Splits: For challenges focused on specific species, splits ensure that each chemical appears in only one of the sets, though different species may be exposed to the same chemical [1].
• Taxonomic Extrapolation Splits: Designed for challenges predicting toxicity for one taxonomic group (e.g., fish) when trained only on data from other groups (e.g., crustaceans and algae) [1].

Visualizing Workflows and Pathways

The following diagrams illustrate the dataset construction workflow and the conceptual role of benchmark data within the SSbD paradigm.

Diagram 1: Construction of the ADORE Benchmark Dataset [1] [2].

Diagram 2: Benchmark Data as a Foundation for SSbD.

Table 3: Key Computational Tools and Data Resources for Ecotoxicology ML

Resource	Type	Primary Function in Research	Key Feature for SSbD/HA
ADORE Dataset [1] [2] [3]	Benchmark Dataset	Provides a standardized, ML-ready dataset with curated toxicity data, chemical features, species traits, and validated data splits to ensure fair model comparison and reproducibility.	Enables the development and benchmarking of robust predictive models for acute aquatic toxicity, a core component of ecological hazard assessment.
ECOTOX Knowledgebase [1] [34] [35]	Primary Data Repository	Serves as the foundational source of experimental ecotoxicity results from the literature. Essential for expanding or customizing datasets.	Provides the empirical ground truth data needed to train and validate predictive models for environmental safety.
U.S. EPA CompTox Chemicals Dashboard [34]	Data Integration Hub	Supplies authoritative chemical identifiers, structures, properties, and links to associated toxicity (ToxValDB) and exposure data (CPDat). Critical for feature generation and data linkage.	Connects chemical structure to hazard and use information, enabling the integration of multiple data types for a more comprehensive safety assessment.
ToxCast/Tox21 HTS Data [34] [37]	In Vitro Bioactivity Data	Provides high-throughput screening data on thousands of chemicals across hundreds of biological pathways. Used as features for predicting in vivo outcomes or for mechanistic insight.	Offers a scalable, animal-free source of bioactivity information that can be used to flag potential hazards based on biological pathway perturbation.
TEST Software [36]	QSAR Prediction Tool	Offers well-established, interpretable QSAR models for specific toxicity endpoints. Useful as a performance baseline against which to compare more complex ML models.	Provides a traditional, transparent risk assessment tool for estimating toxicity when experimental data are absent.
Molecular Representations (e.g., Morgan fingerprints, Mordred descriptors) [1] [2]	Data Features	Numerical encodings of chemical structure that serve as the primary input features for ML models predicting toxicity from chemical structure.	Translate molecular design into a computable format, directly linking chemical innovation to predicted safety outcomes.

The application of machine learning (ML) to predict chemical toxicity offers a transformative opportunity to reduce costly and ethically challenging animal testing in ecotoxicology [1]. However, the field's progress is hindered by a fundamental challenge: the inability to directly and fairly compare the performance of different models and algorithms [3]. Model performance is intrinsically linked to the data on which it is trained and tested. Variations in dataset composition, chemical space, and species scope can lead to dramatically different performance metrics, making claims of superiority difficult to validate across studies [2].

This reproducibility crisis underscores the paramount importance of benchmark datasets—standardized, well-curated, and publicly available resources that serve as a common ground for the scientific community [1]. In fields like computer vision (e.g., ImageNet) and hydrology (e.g., CAMELS), such benchmarks have catalyzed progress by enabling objective comparison [3]. For ecotoxicology, the ADORE (Acute Aquatic Toxicity) dataset has been introduced to fulfill this role, focusing on acute mortality data for fish, crustaceans, and algae [1].

A core, yet often underestimated, component of a robust benchmark is the strategy used to split the data into training and testing subsets. A poor splitting method can create data leakage, where information from the test set inadvertently influences model training, leading to optimistically biased and non-generalizable performance estimates [3]. This is particularly perilous in ecotoxicology, where datasets frequently contain multiple experimental results for the same chemical-species pair due to biological variability and repeated studies [2].

This guide provides a comparative analysis of two advanced splitting strategies essential for realistic ecotoxicology ML: scaffold splitting (group-based by chemical structure) and temporal splitting. We frame this discussion within the context of the ADORE benchmark, supported by experimental data, to equip researchers with the knowledge to build models that truly generalize to novel chemicals and future scenarios.

Core Splitting Strategies for Ecotoxicological Data

The choice of how to partition data defines the very question a model is being asked to answer. Moving beyond simple random splits is necessary to assess a model's predictive power in meaningful, real-world contexts.

Figure 1: Compilation of the ADORE Benchmark Dataset [1]

Scaffold Splitting: Generalizing to Novel Chemical Structures

Scaffold splitting is a group-based splitting method where the dataset is partitioned based on the molecular scaffold or core structure of the chemicals [38]. The goal is to ensure that all data points belonging to chemicals with the same underlying scaffold are contained entirely within either the training or the test set, but not both.

• Rationale: This tests a model's ability to extrapolate to genuinely novel chemical classes. In a regulatory context, the toxicity of a new, structurally unique compound is often of greatest interest. A random split might place similar derivatives of the same scaffold in both training and test sets, allowing the model to "interpolate" rather than truly predict, inflating performance [2].
• Implementation: Chemicals are first categorized by their Bemis-Murcko scaffolds or other structural frameworks. These scaffold groups are then treated as immutable units. Splitting algorithms like GroupShuffleSplit or GroupKFold from the scikit-learn library are employed to allocate entire groups to different data subsets [38].
• ADORE Application: The ADORE dataset provides predefined splits based on chemical occurrence, which aligns with the scaffold splitting philosophy. It offers specific challenge datasets like CA2F-diff, where the model is trained on chemicals tested on algae and crustaceans and tested on a different set of chemicals tested on fish. This directly evaluates cross-species, cross-chemical prediction capability [5].

Temporal Splitting: Forecasting Future Outcomes

Temporal splitting orders data chronologically by the date of the experiment or publication and uses past data to train a model that predicts future outcomes [39].

• Rationale: This simulates a real-world deployment scenario where a model trained on historical data is used to predict the toxicity of newly tested or synthesized chemicals. It accounts for potential temporal drift, such as changes in experimental protocols or the emergence of new chemical classes over time. It is the most stringent test of a model's practical utility for prospective hazard assessment.
• Implementation: Data must have a reliable timestamp. The temporal_train_test_split function from libraries like sktime can be used, where a cutoff date is selected. All data before the cutoff is used for training, and all data after is held out for testing [39].
• Challenge in Ecotoxicology: Many public toxicity databases do not consistently provide precise experimental dates, making temporal splits challenging to implement rigorously. However, the principle remains critical for evaluating practical applicability.

Figure 2: Comparison of Train-Test Splitting Strategies

Comparative Performance Analysis on the ADORE Benchmark

Empirical evidence from studies utilizing the ADORE dataset clearly demonstrates how splitting strategy directly impacts perceived model performance and reveals the true challenge of generalization.

Experimental Protocol from a Comparative Study

A comprehensive 2025 study conducted a benchmark evaluation of 161 models using the ADORE dataset [5]. The experimental design is summarized below:

• Datasets: The study used the predefined ADORE splits, including:
  • Within-Species (F2F, A2A, C2C): Training and test data for a single taxonomic group, split by chemical occurrence to avoid leakage.
  • Cross-Species (CA2F-same & CA2F-diff): Training on algae and crustacean data, testing on fish data. The "same" and "diff" suffixes indicate whether the test chemicals are present in the training set.
• Models & Features: Combinations of six ML algorithms (RF, XGBoost, DNN, etc.), five Graph Neural Networks (GCN, GAT, etc.), and three molecular representations (Morgan fingerprint, MACCS, Mol2vec) were evaluated.
• Evaluation Metric: Area Under the Receiver Operating Characteristic Curve (AUC) was used to assess binary classification performance (more toxic vs. less toxic).

Quantitative Performance Comparison

The following table synthesizes key results from the study, highlighting the performance gap driven by the splitting strategy [5].

Table 1: Model Performance (AUC) on ADORE Dataset Splits [5]

Prediction Task	Dataset Split	Best Performing Model	AUC Score	Performance Interpretation
Within-Species	F2F (Fish, split by chemical)	Graph Convolutional Network (GCN)	0.982 - 0.992	Excellent performance when test chemicals are structurally related to training chemicals.
Cross-Species, Seen Chemicals	CA2F-same (Train: Algae/Crustacean; Test: Fish, same chemicals)	Graph Attention Network (GAT)	~0.83	Moderate performance drop. Model transfers knowledge across species but for known chemicals.
Cross-Species, Unseen Chemicals	CA2F-diff (Train: Algae/Crustacean; Test: Fish, different chemicals)	Deep Neural Network (DNN) with MACCS	0.821	Significant challenge. Model must extrapolate across both species and chemical space.
Performance Gap	F2F vs. CA2F-diff	GCN (F2F) vs. DNN (CA2F-diff)	~0.17 decrease	Illustrates the substantial added difficulty of scaffold-based generalization.

Key Findings:

• Models achieve near-perfect AUC (~0.99) on within-species tasks with appropriate chemical splitting, demonstrating the capability of modern ML on well-posed problems [5].
• Performance decreases noticeably (~0.83) for cross-species prediction even when chemicals are the same, highlighting the challenge of biological extrapolation.
• The most realistic and difficult task—predicting toxicity for new chemicals on a different species (CA2F-diff)—shows a further performance decline (AUC ~0.82). The best model for this task was not the best GNN, but a DNN with a simpler fingerprint, suggesting overfitting on the training chemical space can hinder generalization [5].
• This ~0.17 AUC gap between the easiest (F2F) and hardest (CA2F-diff) splits quantitatively underscores the critical importance of the splitting strategy in defining model performance and the current limits of generalizable toxicity prediction.

Implementation Guide: From Theory to Practice

Practical Steps for Implementing Advanced Splits

• Data Preparation: Ensure each data point has a unique chemical identifier (e.g., InChIKey, SMILES) and, if possible, a timestamp. For scaffold splitting, generate molecular scaffolds from SMILES strings using libraries like RDKit.

• Scaffold Splitting with scikit-learn:

• Temporal Splitting with sktime:

• Using Predefined Benchmark Splits: The most reliable method for comparable research is to use the fixed training and test splits provided by benchmark datasets like ADORE [1] or LakeBeD-US [40].

The Scientist's Toolkit: Essential Research Reagents

Table 2: Key Research Reagents and Resources for Ecotoxicology ML

Item	Function in Research	Example/Source
Benchmark Datasets	Provide standardized, curated data for training and, crucially, fixed splits for fair model comparison.	ADORE [1], LakeBeD-US [40]
Toxicity Databases	Source of raw experimental ecotoxicology data.	US EPA ECOTOX database [1]
Molecular Representation Tools	Translate chemical structures into numerical features for ML models.	RDKit (for fingerprints, scaffolds), Mol2Vec [2]
Taxonomic & Phylogenetic Data	Provide features to represent species differences and evolutionary relationships.	Integrated into ADORE from sources like FishBase and phylogenetic trees [1]
Group/Temporal Splitting Algorithms	Implement advanced data partitioning strategies to prevent leakage.	scikit-learn (GroupShuffleSplit) [38], sktime (temporal_train_test_split) [39]
Graph Neural Network Libraries	Implement state-of-the-art models that operate directly on molecular graphs.	PyTorch Geometric, Deep Graph Library

The strategic design of train-test splits is not a mere technical detail but a fundamental determinant of the validity and utility of machine learning in ecotoxicology. As evidenced by performance on the ADORE benchmark, models that excel at interpolating within a known chemical and species space often fail to maintain that performance when tasked with the realistic challenge of extrapolation—predicting toxicity for novel chemical scaffolds in different organisms [5].

The adoption of rigorous, prospectively challenging splitting strategies like scaffold and temporal splits is essential for:

• Accelerating Regulatory Acceptance: Models validated under easy splits create an illusion of competence. Demonstrating robust performance under hard, generalization-focused splits builds trust for use in chemical safety assessment [2].
• Driving Methodological Innovation: Benchmark datasets with fixed, challenging splits (like ADORE's CA2F-diff) direct the research community's effort toward solving the hardest problems, fostering innovation in model architectures and feature engineering [5].
• Establishing Ecological Benchmarks: The principle extends beyond chemistry. Emerging benchmarks like LakeBeD-US for water quality time series employ temporal splitting to test models' forecasting capabilities, ensuring they are evaluated on their ability to predict future ecosystem states [40].

In conclusion, the path toward reliable in silico ecotoxicology is paved with benchmark datasets that enforce rigorous evaluation through careful data splitting. By prioritizing scaffold and temporal strategies, researchers can develop models whose reported performance reflects true predictive power, ultimately contributing to the reduction of animal testing and the protection of environmental health.

Navigating Challenges: Solving Data Leakage, Bias, and Interpretability Issues

The integration of machine learning (ML) into ecotoxicology promises to reduce reliance on costly and ethically challenging animal testing [1]. However, the field faces a significant reproducibility crisis, largely driven by inadequate data splitting practices that lead to data leakage [41]. This occurs when information from the test set inadvertently influences the model training process, yielding overly optimistic performance estimates that fail to reflect a model's true ability to generalize to new chemicals or species [42] [43]. The recent introduction of curated benchmark datasets, such as ADORE for acute aquatic toxicity, provides a common ground for objective model comparison and highlights the critical impact of splitting strategies [1] [2]. This guide compares methodological approaches within this context, demonstrating how proper data handling is paramount for generating reliable, regulatory-relevant predictions.

Experimental Comparison: Splitting Strategies and Model Performance

The performance and apparent reliability of ML models in ecotoxicology are not inherent properties of the algorithms alone but are profoundly influenced by the experimental design, particularly how data is partitioned. The table below summarizes key findings from recent studies on predicting hepatotoxicity and fish acute mortality, illustrating the variable outcomes based on data handling [44] [41].

Table 1: Comparison of Machine Learning Model Performance Across Different Studies and Data Conditions

Study Focus	Best-Performing Model(s)	Key Performance Metric & Result	Critical Data Handling Note
Hepatotoxicity Prediction (Multiple endpoints) [44]	Random Forest, Support Vector Machine (SVM), Ensemble models	Mean CV F1 scores varied from ~0.09 to 0.74, highly dependent on the specific toxicity endpoint and class balancing method.	Performance was heavily influenced by how class imbalance (skewed positives/negatives) was addressed; over-sampling sometimes helped, but results were endpoint-specific.
Fish Acute Mortality (LC50) (ADORE t-F2F challenge) [41]	Tree-based models (Random Forest, XGBoost)	Root Mean Square Error (RMSE) of 0.90 for log10(LC50) (equating to an order of magnitude on the original scale).	Model performance was strongly dependent on data split. Molecular representation had a weak effect, and mass vs. molar concentration did not affect results.

A core insight from the ADORE benchmark work is that the strategy for creating training and test splits is more consequential than the choice of ML algorithm or chemical descriptor [41]. The following table contrasts common splitting methods, evaluating their suitability for ecotoxicological data characterized by repeated experiments on the same chemical-species pairs.

Table 2: Comparison of Data Splitting Strategies for Ecotoxicological Machine Learning

Splitting Strategy	Method Description	Risk of Data Leakage	Simulates Real-World Use Case	Recommended Application
Random Split	Data points are randomly assigned to train and test sets, ignoring underlying structure.	Very High. Repeated measurements for the same chemical-species pair are likely spread across sets, allowing the model to "memorize" [2] [41].	Poorly simulates predicting toxicity for a truly new chemical or species.	Not recommended for benchmark datasets with repeated experiments.
Split by Chemical Scaffold	Chemicals are grouped by molecular backbone; all data for an entire scaffold is placed in either train or test set.	Low. Prevents the model from seeing structurally similar chemicals during both training and testing [1].	Effectively simulates the challenge of predicting toxicity for a novel class of compounds.	Ideal for testing chemical extrapolation.
Leave-Profile-Out / Cluster-Out	All data points belonging to a natural cluster (e.g., a soil profile, repeated experimental series) are kept together in one set [42] [43].	Very Low. Explicitly designed to prevent leakage from correlated observations within clusters.	Simulates prediction for a completely new, unseen experimental unit or condition.	Essential for data with temporal, spatial, or experimental replication structure [42] [43].
Taxon-Based Split	All data for a given taxonomic group (e.g., a specific fish species) is held out for testing.	Low. Prevents the model from leveraging data from the test species during training.	Simulates predicting toxicity for a species with no existing test data, a common regulatory need.	Ideal for testing taxonomic extrapolation [1] [2].

Detailed Experimental Protocols

To ensure reproducibility and fair comparison, studies using benchmark datasets must transparently detail their experimental pipeline. The following protocols are derived from the creation and use of the ADORE dataset [1] [41].

Protocol 1: Dataset Curation and the ADORE Benchmark

The ADORE (Acute DOse REsponse) dataset was constructed to provide a standardized foundation for ML in aquatic ecotoxicology [1].

• Source Data Extraction: Core toxicity data (LC50/EC50 values) for fish, crustaceans, and algae were extracted from the US EPA's ECOTOX database (September 2022 release). Focus was placed on acute mortality and comparable endpoints (e.g., immobilization in crustaceans, growth inhibition in algae) with exposure durations ≤96 hours [1].
• Data Filtering and Harmonization:
  • Entries with missing critical taxonomic information were removed.
  • In vitro tests and tests on early life stages (eggs/embryos) were excluded to focus on traditional in vivo endpoints [1].
  • Chemicals were standardized using unique identifiers (InChIKey, DTXSID, CAS).
• Feature Expansion:
  • Chemical Features: Six molecular representations (MACCS, PubChem, Morgan, ToxPrints fingerprints, Mordred descriptors, mol2vec embeddings) and physicochemical properties were added [2] [3].
  • Species Features: Phylogenetic distance matrices and ecological trait data (e.g., habitat, life history) were incorporated to model interspecies sensitivity [1] [2].
• Predefined Splits and Challenges: The dataset was partitioned using non-random strategies (e.g., by chemical scaffold, by taxonomy) to create specific "challenges." These fixed splits, such as "t-F2F" (all fish data), are publicly available to ensure every researcher evaluates models on identical test data, enabling direct performance comparison [1] [41].

Protocol 2: Model Training and Evaluation with Fixed Splits

A subsequent modeling study on the ADORE fish challenge exemplifies a robust training and evaluation workflow [41].

• Challenge Selection: The researchers selected the predefined "t-F2F" challenge, which contains all fish data with a scaffold-based split ensuring no chemical in the test set is structurally related to those in the training set [41].
• Model and Feature Setup: Four regression models (LASSO, Random Forest, XGBoost, Gaussian Process) were trained separately using each of the six molecular representations. The target variable was the log10-transformed LC50 value.
• Training and Tuning: Models were trained exclusively on the predefined training partition. Hyperparameter tuning was conducted using cross-validation only within the training set.
• Final Evaluation: The final model was evaluated once on the held-out test partition. The primary performance metric was Root Mean Square Error (RMSE). This single test result, obtained without any peeking at the test data during development, provides an unbiased estimate of generalization error [41].

Visualizing Data Leakage and Mitigation Workflows

A key to understanding and preventing data leakage is visualizing how information flows—and where it can spill improperly—within an ML experiment.

The Critical Impact of Data Splitting Strategy

The following diagram details the end-to-end workflow for building a compliant, leakage-free model using a benchmark dataset like ADORE, from data access to final reporting.

Workflow for Leakage-Free Model Benchmarking

The Scientist's Toolkit: Essential Research Reagent Solutions

Building reliable ML models in ecotoxicology requires more than just algorithms; it depends on high-quality, well-curated "research reagents" in the form of data and software.

Table 3: Key Research Reagent Solutions for Ecotoxicology ML

Tool / Resource	Type	Primary Function in Research	Example / Source
Benchmark Datasets	Data	Provide pre-curated, standardized data with defined train/test splits to ensure fair model comparison and prevent data leakage.	ADORE [1], LakeBeD-US [40]
Toxicity Databases	Data	Serve as primary sources of experimental in vivo toxicity data for dataset construction.	US EPA ECOTOX [1], ToxRefDB [44]
Molecular Representation Tools	Software	Convert chemical structures into numerical descriptors or fingerprints that ML models can process.	RDKit (for fingerprints), Mordred [2], mol2vec [3]
Phylogenetic Information	Data	Provide quantitative measures of evolutionary relatedness between species, used as features to inform interspecies sensitivity predictions.	Time-calibrated phylogenetic trees [1] [2]
Structured Splitting Algorithms	Software/Method	Implement splitting strategies that respect the clustered nature of data (e.g., by scaffold, by species) to prevent leakage.	Scikit-learn GroupShuffleSplit, custom clustering scripts [43]
Reporting Checklists	Guideline	Provide structured frameworks to ensure complete and transparent reporting of ML experiments, aiding reproducibility.	REFORMS [41], QSAR best practice guidelines [41]

Adopting these tools and adhering to the experimental protocols centered on rigorous data splitting are fundamental steps toward robust, reproducible ML in ecotoxicology. This approach moves the field beyond isolated studies with inflated performance claims and toward a cumulative science capable of producing reliable tools for regulatory decision-making [2] [45].

Addressing Chemical and Species Bias in Training Data

The integration of machine learning (ML) into ecotoxicology represents a paradigm shift, offering the potential to predict chemical hazards, reduce animal testing, and manage the risks posed by thousands of chemicals in the environment [1]. However, the reliability of these models is fundamentally constrained by the quality and composition of their training data. Biases embedded within datasets—whether from uneven chemical space coverage or disproportionate representation of certain species—can lead to models that perform well only for narrow, well-represented subsets, while failing unpredictably for novel chemicals or ecologically relevant species [46]. This not only limits scientific utility but also raises significant ethical and regulatory concerns, as biased models could lead to inadequate environmental protections or misdirected resources [47] [48].

Addressing these biases is therefore not merely a technical exercise but a prerequisite for building equitable, trustworthy, and generalizable tools for ecological risk assessment [47]. This comparison guide frames the discussion within the critical context of benchmark datasets, which serve as the common ground for developing, testing, and fairly comparing different ML approaches [1]. We objectively evaluate several contemporary methodologies designed to identify, mitigate, or work around chemical and species bias, providing researchers with a clear analysis of their experimental performance, underlying protocols, and practical applications.

Comparative Analysis of Methodologies for Addressing Bias

The following table summarizes the core approaches, their mechanisms for handling bias, key performance outcomes, and inherent strengths and limitations.

Table 1: Comparison of Methodologies Addressing Chemical and Species Bias in Ecotoxicology ML

Methodology	Primary Reference & Core Mechanism	Key Performance Metric (vs. Baseline)	Strengths in Addressing Bias	Limitations & Remaining Challenges
Pairwise Learning via Matrix Factorization	[30]: Treats sparse (chemical, species) data as a matrix completion problem, learning global biases and interaction terms.	RMSE of 0.65 log(mol/L) for predicted LC50s; enabled prediction for 4M missing pairs from 70k experiments [30].	Directly targets data sparsity bias; models species-chemical interactions ("lock-key"); generates full matrices for novel hazard distributions.	Performance depends on initial data density; model is a "black box," limiting mechanistic insight.
Coverage Bias Assessment with MCES	[46]: Uses Maximum Common Edge Subgraph distance to quantify how well a dataset covers the known universe of biomolecular structures.	Identified significant non-uniform coverage in public ML datasets; proposed a diagnostic framework for dataset evaluation [46].	Provides a rigorous, chemistry-intuitive measure to diagnose chemical space bias in any dataset; guides future data curation.	Computationally intensive; does not itself fill data gaps or correct bias.
Autoencoder for Latent Space Representation	[49]: Learns compressed, informative chemical embeddings (latent space) from high-dimensional molecular descriptors.	Achieved R² = 0.668 & MAE = 0.572 for HC50 prediction, outperforming PCA (R²=0.601) and Random Forest (R²=0.663) [49].	Reduces noise and irrelevant features; latent space may better capture biologically relevant chemistry, improving generalization.	Requires substantial data for training; interpretation of latent variables can be difficult.
Specialized SSD Modeling with Expanded Taxonomy	[50]: Builds Species Sensitivity Distribution models using data curated across 14 taxonomic groups and integrates acute/chronic endpoints.	Developed models to predict HC5 for untested chemicals; prioritized 188 high-toxicity compounds from a set of ~8,449 [50].	Explicitly incorporates broader taxonomic diversity to counter species bias; outputs directly applicable to regulatory risk assessment.	Model accuracy is still bounded by the availability and quality of underlying ecotoxicity data.

Detailed Experimental Protocols

Protocol 1: Pairwise Learning for Bridging Data Gaps

This protocol, based on the work of [30], details the process of using machine learning to predict missing ecotoxicity values across vast chemical and species matrices.

Objective: To generate a complete matrix of Predicted LC50 values for all combinations of C chemicals and S species, where the observed data matrix is highly sparse (~0.5% filled).

Input Data Preparation:

• Source: Extract acute mortality data (LC50) from a standardized benchmark dataset (e.g., the ADORE dataset) [1] [30].
• Curation: Filter for tests on defined taxonomic groups (e.g., fish, crustaceans, algae), standard exposure durations (24, 48, 72, 96h), and lethal endpoints. Retain unique identifiers for chemicals (CAS RN) and species (taxonomic name or ID) [30].
• Structuring: Organize data into a list of unique experiments, each defined by a (Chemical, Species, Duration, LC50) quadruple. Express LC50 values in log10(mol/L) format.

Model Training with Bayesian Matrix Factorization:

• Feature Encoding: Represent each experiment as a sparse feature vector x using one-hot encoding for the chemical, species, and duration categories [30].
• Model Definition: Employ a second-order Factorization Machine model. The model learns: a global bias (w₀), first-order bias terms for each chemical and species (wᵢ), and latent factor interactions (vᵢ) between them [30].
• Training: Use Bayesian inference (e.g., Markov Chain Monte Carlo) for optimization. The model is trained to predict the log(LC50) value, learning the average toxicity of each chemical, the average sensitivity of each species, and their specific interaction effects [30].

Output and Validation:

• Prediction: Apply the trained model to all possible (Chemical, Species, Duration) triplets missing from the input data, generating a dense matrix of predicted LC50s [30].
• Validation: Use time-split or scaffold-split cross-validation on the observed data to evaluate prediction accuracy (e.g., RMSE). Compare against null models that only learn mean values [30].
• Derivative Products: Use the full predicted matrix to construct novel tools such as all-species Hazard Heatmaps, Species Sensitivity Distributions (SSDs), and Chemical Hazard Distributions (CHDs) [30].

Diagram 1: Workflow for pairwise learning to bridge data gaps [30].

Protocol 2: Assessing Chemical Coverage Bias with MCES Distance

This protocol, derived from [46], provides a method to evaluate whether a given dataset provides a representative sample of chemical space.

Objective: To quantify the coverage bias of a molecular dataset against a reference universe of biologically relevant small molecules.

Reference "Universe" Construction:

• Aggregate Reference Databases: Compile a large, diverse set of molecular structures from public databases of metabolites, drugs, natural products, and environmentally relevant chemicals [46].
• Standardize and Deduplicate: Process structures to remove duplicates and standardize representation (e.g., using SMILES).

Distance Calculation (Myopic MCES):

• Molecular Representation: Represent molecules as molecular graphs (atoms as nodes, bonds as edges).
• Compute Distance: For a pair of molecules, compute the Maximum Common Edge Subgraph (MCES). The distance is defined as: d(M₁, M₂) = |E₁| + |E₂| - 2 × |MCES(M₁, M₂)|, where |E| is the number of edges [46].
• Efficient Approximation: For large-scale analysis, use a hybrid approach:
  • Compute fast, provable lower bounds for all pairs.
  • Perform exact MCES computation only for pairs with a bound below a set threshold (e.g., 10).
  • For pairs above the threshold, use the threshold value as the "myopic" distance (mMCES) [46].

Visualization and Analysis:

• Dimensionality Reduction: Use UMAP (Uniform Manifold Approximation and Projection) to embed the mMCES distance matrix into 2D for visualization [46].
• Coverage Assessment: Project the molecules from the target dataset onto the same UMAP embedding of the reference universe. Visually and statistically assess whether the dataset clusters in specific regions or uniformly covers the reference space [46].
• Class Analysis: Supplement with an analysis of the distribution of compound classes (e.g., using ClassyFire taxonomy) in the dataset versus the reference universe [46].

Visualization of Bias and Mitigation Pathways

A coherent understanding of bias sources and mitigation strategies is essential. The following diagram synthesizes concepts from the reviewed literature into a unified framework.

Diagram 2: A framework for sources and mitigation of bias in ecotoxicology ML.

The Scientist's Toolkit: Essential Research Reagents & Materials

Table 2: Key Research Reagents and Computational Tools for Bias-Aware Ecotoxicology ML

Item / Resource	Primary Function	Role in Addressing Bias	Example Source/Reference
ADORE Benchmark Dataset	A standardized, curated dataset of acute aquatic toxicity for fish, crustaceans, and algae, with defined train-test splits.	Provides a common, well-characterized baseline for fair model comparison, reducing evaluation bias due to inconsistent data processing [1].	[1]
ECOTOX Knowledgebase	The underlying comprehensive source of empirical ecotoxicity studies from the U.S. EPA.	Serves as the primary data source for building curated datasets and understanding the real-world distribution of tested species and chemicals [1] [50].	U.S. EPA
CompTox Chemicals Dashboard	A hub for chemistry, toxicity, and exposure data for ~900,000 chemicals, providing validated identifiers and properties.	Enables accurate chemical mapping and enrichment of datasets with standardized descriptors, reducing identifier-based noise and bias [1].	U.S. EPA
Maximum Common Edge Subgraph (MCES) Algorithm	A graph-based method for computing the structural similarity between two molecules.	Functions as a bias diagnostic tool to assess how well a training dataset covers chemical space, identifying over- and under-represented regions [46].	[46]
Pairwise Learning / Factorization Machines (libfm)	A machine learning library designed for recommendation systems, adapted for (chemical, species) matrix completion.	Acts as a bias-mitigating model that learns and corrects for global chemical and species biases while capturing their specific interactions [30].	Rendle, S. (libfm)
Autoencoder Neural Networks	A type of neural network that learns efficient, lower-dimensional representations (embeddings) of input data.	Serves as a representation learning tool to derive bias-reduced, task-informed chemical features from high-dimensional descriptors, potentially improving generalization [49].	[49]
Species Sensitivity Distribution (SSD) Models	Statistical models that estimate the concentration of a chemical affecting a given percentage of species.	An application-focused output that uses completed or expanded data matrices to make risk assessments that account for broader taxonomic diversity, countering species bias [30] [50].	[30] [50]

In ecotoxicology and drug development, machine learning (ML) models are increasingly used to predict complex outcomes, from chemical toxicity to a compound's pharmacological activity. However, their predictive power is often accompanied by a lack of transparency, rendering them "black boxes" [51] [52]. This opacity is a significant barrier to trust and adoption in high-stakes scientific fields, where understanding the why behind a prediction is as critical as the prediction itself. Explainable Artificial Intelligence (XAI) addresses this by making model decisions interpretable to researchers and regulators [53].

Two of the most prominent XAI techniques are SHAP (SHapley Additive exPlanations) and LIME (Local Interpretable Model-agnostic Explanations). Both serve as post-hoc explanation tools but are founded on different principles: SHAP is rooted in cooperative game theory to assign a consistent value to each feature's contribution [54], while LIME operates by constructing a simple, interpretable local surrogate model around a single prediction [55] [52]. Within the context of developing robust benchmark datasets for ecotoxicology ML research, these tools are indispensable. They allow scientists to validate model logic against domain knowledge, identify which molecular descriptors or environmental variables are driving predictions, and ensure that models are learning chemically and biologically plausible relationships rather than spurious correlations.

Core Methodologies and Theoretical Comparison

SHAP and LIME provide distinct pathways to interpretability. The following table summarizes their foundational characteristics, which dictate their suitability for different research scenarios.

Table 1: Foundational Comparison of SHAP and LIME

Aspect	SHAP (SHapley Additive exPlanations)	LIME (Local Interpretable Model-agnostic Explanations)
Core Theory	Derived from game theory (Shapley values). Treats each feature as a "player" and the prediction as the "payout," calculating each feature's marginal contribution across all possible feature combinations [52] [54].	A local surrogate model method. Perturbs the input instance and learns a simple (e.g., linear) interpretable model that approximates the complex model's behavior in the local vicinity of the instance [55] [54].
Explanation Scope	Provides both local and global explanations. Can explain single predictions and aggregate explanations across a dataset to show overall feature importance [52] [54].	Provides strictly local explanations. Explains individual predictions but does not natively provide a consistent global feature importance overview [55] [54].
Consistency & Stability	Theoretically more stable and consistent due to its game-theoretic foundation, which guarantees properties like local accuracy and consistency [55] [54].	Can exhibit instability. Explanations may vary for the same instance across different runs due to the random sampling involved in perturbation [55] [54].
Computational Load	Computationally more expensive, especially for exact Shapley value calculation on complex models with many features. KernelSHAP provides an approximation [54].	Generally faster and less computationally intensive for generating a single-instance explanation [54].
Primary Output	A SHAP value for each feature per prediction, indicating that feature's contribution to the deviation from the average model output. Positive/Negative values indicate positive/negative contributions [52].	A set of feature weights for the local surrogate model, showing the magnitude and direction of a feature's influence on the specific prediction [55].

The logical workflow of each method, from the original model to the final explanation, is illustrated in the diagrams below.

Diagram 1: SHAP workflow from model to explanations.

Diagram 2: LIME workflow for local instance explanation.

Performance Comparison on Benchmark Tasks

The utility of SHAP and LIME is best evaluated through their application to real-world scientific problems. A benchmark study on predicting emergency room admissions for cardiorespiratory diseases from environmental factors provides a clear, data-driven comparison [56].

Table 2: Performance of XAI Methods in an Environmental Health Benchmark Study [56]

Component	Model & Performance	SHAP Analysis (Global)	LIME Analysis (Local)
Description	Best Model: XGBoostTask: Regression to predict daily admissions.Performance: R² = 0.901; Mean Absolute Error (MAE) = 0.047. Validated via 10-fold CV.	Identified global feature importance and directional impact.	Identified critical environmental thresholds for high-risk predictions (95th percentile).
Key Features Identified	--	Most influential: Carbon Monoxide (CO), Relative Humidity (RH), Atmospheric Pressure, Average Temperature.	Critical thresholds: CO > 0.84 mg/m³, Atmospheric Pressure ≤ 1006.81 hPa, Avg Temp ≤ 17.19°C, RH > 70.33%.
Interpretation Output	--	Showed that high CO/RH and low pressure/mild temps are associated with increased admissions.	Quantified the precise value of each feature at which the risk of a high-admission prediction increased significantly.

This study demonstrates a synergistic use case: SHAP provided a reliable, global overview of which environmental factors matter most, while LIME drilled down to define actionable, local decision thresholds [56]. This pattern is highly relevant to ecotoxicology, where researchers need to know both the overall most toxicological-relevant molecular features (global) and the specific concentration or property thresholds that trigger a toxicity prediction (local).

Impact of Model Choice and Data Structure

A critical caveat for benchmarking is that explanation outputs are model-dependent. Research on classifying myocardial infarction cases showed that the top features identified by SHAP varied significantly across different model architectures (e.g., Logistic Regression, Decision Tree, LightGBM) [54]. This underscores that explanations are not absolute truths about the data, but reflections of how a specific model understands the data. Furthermore, both SHAP and LIME can produce misleading results when features are highly correlated, as they often assume feature independence [54]. This is a crucial consideration for ecotoxicology datasets, which may contain correlated molecular descriptors or environmental measurements.

Experimental Protocols for XAI Evaluation in Research

Integrating SHAP and LIME into a rigorous experimental pipeline is essential for reproducible and credible results. The following protocols are adapted from benchmark studies.

This protocol is suitable for regression/classification tasks linking environmental or chemical features to an outcome.

• Model Training & Benchmarking: Split data into training/test sets. Train and optimize multiple ML models (e.g., Random Forest, XGBoost, Gradient Boosting). Select the best-performing model based on metrics (R², MAE, AUC-ROC) validated via k-fold cross-validation.
• Global Explanation with SHAP:
  • Compute SHAP values for a representative sample of the test set using the appropriate explainer (e.g., TreeExplainer for tree-based models).
  • Generate a SHAP summary plot (beeswarm plot) to visualize global feature importance and the distribution of each feature's impact.
  • Generate SHAP dependence plots for top features to examine their individual, non-linear relationship with the model output.
• Local Explanation with LIME:
  • Select specific instances of interest (e.g., high-risk predictions).
  • For each instance, instantiate a LIME explainer and generate an explanation.
  • Analyze the feature weights from LIME to understand the local decision logic.
  • Systematically vary feature values to identify critical thresholds that flip the model's prediction.
• Validation: Compare explanations against domain knowledge and established literature. Perform sensitivity analysis to check the stability of LIME explanations upon repeated runs.

This protocol evaluates how explanations vary across models, which is vital for benchmarking.

• Fixed-Dataset Experiment: Hold the dataset (features and target) constant.
• Multiple-Model Training: Train several structurally different ML models (e.g., linear model, tree-based model, support vector machine) to similar performance levels.
• Explanation Generation: Apply SHAP (or LIME) uniformly to all trained models to generate feature importance rankings for the same set of predictions.
• Comparative Analysis: Quantify the rank correlation or similarity (e.g., using Jaccard index) of the top-K important features across models. Document and report the variability.

Applications in Ecotoxicology and Drug Development

The application of SHAP and LIME accelerates discovery and risk assessment by adding a layer of interpretability to complex AI models.

Table 3: Application of SHAP and LIME in Key Domains

Domain	Primary Use Case	Typical Model Type	Utility of SHAP	Utility of LIME
Ecotoxicology & Environmental Chemistry	Predicting toxicity endpoints (e.g., LC50), contaminant fate, and optimal remediation strategies [57].	Gradient Boosting, GNNs, Hybrid AI-Physics models.	Identifies which molecular substructures or environmental variables (e.g., pH, organic carbon) are globally most influential on toxicity or pollutant mobility [57].	Explains why a specific chemical is predicted as highly toxic, highlighting the contributing fragments. Identifies critical environmental condition thresholds for remediation failure.
Drug Discovery & Development	Predicting compound activity, toxicity (ADMET), and protein-ligand binding affinity [51].	Deep Neural Networks, Random Forest, XGBoost.	Provides a global view of chemical features (e.g., presence of certain functional groups, lipophilicity) driving activity across a chemical library [51].	Explains the prediction for a single lead compound, guiding medicinal chemists on which parts of the molecule to modify to improve potency or reduce toxicity.

For example, in a unified AI framework for pollution modeling, SHAP analysis identified natural attenuation processes as the most influential model feature, consistent with physical understanding [57]. In drug research, XAI methods like SHAP are critical for elucidating structure-activity relationships, moving beyond a "black box" prediction to a hypothesis-generating tool for chemists [51].

Table 4: Key Research Reagent Solutions and Resources

Resource Name	Type	Primary Function in XAI Research	Relevance to Ecotoxicology/Drug Development
SHAP Python Library	Software Library	Computes SHAP values for various ML models (TreeExplainer, KernelExplainer, DeepExplainer). Enables generation of summary, dependence, and force plots [52] [54].	Core tool for implementing SHAP-based explanation in custom modeling pipelines for toxicity prediction or compound screening.
LIME Python Library	Software Library	Implements the LIME algorithm for tabular, text, and image data. Creates local surrogate models and visualizes feature contributions for individual instances [55] [52].	Essential for generating case-by-case explanations for specific chemicals or experimental conditions.
EcoTox Benchmark Datasets	Data Resource	Curated datasets linking chemical structures or environmental measurements to toxicological endpoints (e.g., from EPA, NICEATM).	Serves as the foundational data for training and, crucially, explaining models in ecotoxicology. Critical for benchmarking XAI methods.
MoleculeNet/TOX21	Data Resource	Benchmark datasets specifically for molecular machine learning, including toxicity labels [51].	Standard benchmarks for developing and validating interpretable models in computational toxicology and drug safety.
XGBoost/LightGBM	ML Algorithm	High-performance, tree-based ensemble algorithms often offering the best predictive performance on structured scientific data [56].	Frequently the model of choice in applied research. They are natively supported by TreeExplainer for fast and exact SHAP value computation.
Optuna	Software Library	Hyperparameter optimization framework. Used to fairly tune and compare different ML models before XAI analysis [56].	Ensures the model to be explained is in its optimal state, making subsequent explanations more reliable.

The integrated application of these tools within an ecotoxicology research workflow is visualized below.

Diagram 3: Integrated XAI workflow for ecotoxicology research.

SHAP and LIME are complementary pillars of a robust XAI strategy in scientific ML. SHAP excels at providing a consistent, global overview of feature importance, which is invaluable for hypothesis generation, model debugging, and reporting overall findings. LIME offers focused, intuitive local explanations that are particularly useful for diagnosing specific predictions and communicating results to stakeholders [55] [56].

For researchers building benchmark datasets and models in ecotoxicology:

• Use SHAP for global analysis: Employ SHAP to audit your model, ensure it is relying on chemically meaningful features, and to produce global importance rankings for publications.
• Use LIME for case studies: Apply LIME to investigate outliers, understand false positives/negatives, and define actionable thresholds for risk assessment.
• Always validate explanations: Treat XAI outputs as hypotheses. Corroborate them with existing literature, experimental data, or through collaboration with domain experts [54].
• Acknowledge limitations: Be transparent about the model-dependent nature of explanations and the potential confounding effects of feature correlation in your analysis [54].

The integration of these explainability techniques directly enhances the trust, reliability, and scientific utility of ML models. By making the black box transparent, SHAP and LIME transform predictive models from mere statistical artifacts into tools for discovery and insight, accelerating the identification of toxic hazards and the development of safer chemicals.

Optimizing for Cross-Species and Cross-Taxa Predictions

The advancement of machine learning (ML) in ecotoxicology is fundamentally constrained by the lack of standardized data for training and evaluating predictive models. Traditional toxicity assessment relies heavily on animal testing, with millions of fish and crustaceans used annually, creating significant ethical and financial imperatives for developing in silico alternatives [1]. While Quantitative Structure-Activity Relationship (QSAR) models have a long history, they are typically limited to chemical features and simpler algorithms [2]. Modern ML promises to integrate diverse data types—including chemical, phylogenetic, and ecological information—to build more robust predictive models. However, progress has been hampered because model performances are only truly comparable when derived from the same dataset, with identical cleaning and splitting procedures [1].

This comparison guide is framed within the essential thesis that benchmark datasets are the cornerstone of reproducible and progressive ML research in ecotoxicology. The recent introduction of curated, publicly available benchmarks like the ADORE (Acute Aquatic Toxicity) dataset is catalyzing a shift in the field, allowing for objective evaluation of algorithmic approaches [1] [29]. This guide provides a detailed, data-driven comparison of methodological performances on such benchmarks, focusing on the complex challenge of cross-species and cross-taxa prediction, where models trained on data from one set of organisms predict toxicity for another.

Experimental Protocols and Benchmark Data Structure

The ADORE Dataset: A Standardized Benchmark

The ADORE dataset serves as a foundational benchmark for ML in ecotoxicology. Its core consists of acute aquatic toxicity data for three ecologically and regulatory-relevant taxonomic groups: fish, crustaceans, and algae, sourced from the US EPA's ECOTOX knowledgebase [1]. The dataset is explicitly designed to overcome barriers to entry by providing a well-curated resource that combines expert biological knowledge with ML-ready structuring.

Key Experimental Protocol and Curation Steps:

• Source Data Extraction: Data was extracted from the September 2022 release of the ECOTOX database. The focus was on standardized, short-term (≤96 hours) lethal effects: mortality (MOR) for fish, mortality and immobilization (ITX) for crustaceans, and effects on growth and population (GRO, POP, PHY) for algae [1].
• Filtering and Harmonization: Entries with missing taxonomic information were removed. In vitro assays and tests on early life stages (e.g., eggs, embryos) were excluded to maintain a focus on whole-organism, acute toxicity relevant to regulatory animal testing [1].
• Feature Expansion: The core toxicity endpoints (e.g., LC50/EC50 values) were augmented with two critical classes of features to enable advanced modeling:
  • Chemical Representations: Each compound is represented by multiple molecular fingerprints (MACCS, PubChem, Morgan, ToxPrints), the molecular embedding mol2vec, and the comprehensive molecular descriptor set Mordred [2] [3].
  • Species Representations: Species are described by ecological and life-history traits. Crucially, a phylogenetic distance matrix is included, based on the hypothesis that closely related species share similar toxicological sensitivities [2].
• Predefined Data Splits and Challenges: To prevent data leakage and ensure fair comparison, ADORE provides fixed training-test splits. It further proposes specific "challenges" of increasing complexity (see Diagram 1): single-species prediction (e.g., Daphnia magna), within-taxon prediction (e.g., all fish), and cross-taxa prediction (e.g., predicting fish toxicity using data from algae and crustaceans) [2] [5].

Diagram 1: ADORE Dataset Creation and Research Challenges Workflow

Methodology for Model Comparison

A representative large-scale evaluation study trained and compared 161 distinct models to establish performance baselines on the ADORE challenges [5]. The experimental protocol is summarized below and in Diagram 2.

Experimental Protocol for Model Benchmarking [5]:

• Dataset Configuration: Models were evaluated on the predefined ADORE splits, including:
  • Same-Species (F2F, C2C, A2A): Training and testing on data from the same taxonomic group.
  • Cross-Taxa (AC2F): Training on combined algae and crustacean data to predict toxicity in fish. This was further divided into "AC2F-same" (test chemicals appear in training) and the more difficult "AC2F-diff" (test chemicals are novel).
• Feature Encoding: Chemical structures were encoded using three representations: Morgan fingerprints, MACCS keys, and mol2vec embeddings.
• Algorithm Selection: The study encompassed:
  • Classical ML: k-Nearest Neighbors (KNN), Naïve Bayes (NB), Random Forest (RF), Support Vector Machine (SVM), and XGBoost (XGB).
  • Deep Neural Networks (DNN): A standard multilayer perceptron.
  • Graph Neural Networks (GNN): Graph Convolutional Network (GCN), Graph Attention Network (GAT), Message Passing Neural Network (MPNN), Attentive FP, and Fingerprint-Integrated GNN (FPGNN).
• Performance Metrics: The primary metric for this binary classification task (classifying chemicals as more or less toxic) was the Area Under the Receiver Operating Characteristic Curve (AUC). AUC values range from 0 to 1, with 1 representing perfect prediction.

Diagram 2: Model Training and Evaluation Methodology for Cross-Taxa Prediction

Performance Comparison of Modeling Approaches

Quantitative Performance Data

The following tables summarize the key characteristics of the benchmark data and the performance of top-performing models across different prediction challenges.

Table 1: Overview of ADORE Dataset Challenges and Sample Sizes [5]

Challenge Type	Dataset Name	Training Species	Test Species	Number of Samples (Train / Test)	Positive:Negative Ratio
Single Species	Training-F2F / F2F-1	140 fish species	Oncorhynchus mykiss	4,818 / 870	1:2.68 / 1:1.66
	Training-C2C / C2C	17 crustacean species	Daphnia magna	3,062 / 1,472	1:2.15 / 1:2.52
	Training-A2A / A2A	46 algae species	Chlorella vulgaris	321 / 118	1:2.22 / 1:3.91
Cross-Taxa	AC2F-same	Algae & Crustaceans	Fish	2,418 (train+test combined)	1:1.93
	AC2F-diff	Algae & Crustaceans	Fish	2,643 (train+test combined)	1:2.52

Table 2: Comparison of Model Performance (AUC) Across Prediction Tasks [5]

Model Category	Best Specific Model	Same-Species Prediction (AUC Range)	Cross-Taxa Prediction: AC2F-diff (AUC)	Key Advantages	Key Limitations
Classical ML	Random Forest (RF)	0.920 - 0.965	0.796	High interpretability, fast training, robust on smaller datasets.	Performance plateaus; struggles with complex cross-taxa generalization.
Deep Neural Network (DNN)	DNN with MACCS	0.935 - 0.978	0.821	Captures non-linear interactions; best chemical generalization in cross-taxa task.	Requires careful tuning; prone to overfitting with limited data.
Graph Neural Network (GNN)	Graph Convolutional Network (GCN)	0.982 - 0.992	0.803 (GAT best)	Best overall performance on same-species tasks; directly learns from molecular graph.	Highest computational cost; largest performance drop (~17% AUC) in cross-taxa task.

Analysis of Comparative Performance

The experimental data reveals clear trade-offs between model complexity and predictive capability across different tasks:

• Same-Species Prediction: Graph Neural Networks (GCNs) consistently achieve state-of-the-art performance (AUC >0.98), significantly outperforming classical ML and standard DNNs [5]. This superiority stems from their ability to natively process molecular structures as graphs, capturing intricate topological features critical for activity.

• Cross-Taxa Prediction: This task presents a substantially greater challenge, as models must learn a mapping not only from chemical structure to activity but also across different biological systems. All models experience a significant performance decline in the most difficult "AC2F-diff" setting. Notably, the GCN's AUC dropped by approximately 17 percentage points compared to its same-species performance [5]. In this challenging scenario, a DNN using MACCS fingerprints achieved the highest AUC (0.821), suggesting that for extrapolating to novel chemicals across taxa, simpler but robust feature representations coupled with flexible non-linear models can be more effective than highly specialized graph architectures [5].

• The Generalization Gap: The stark contrast between high same-species accuracy and lower cross-taxa accuracy highlights the central challenge of biological extrapolation. Models excelling at interpolation within a taxon may rely on latent features specific to that taxon's biological response, which do not transfer perfectly to others. This underscores the importance of incorporating informative biological features (like phylogeny) and developing algorithms specifically designed for transfer learning across biological domains.

The Scientist's Toolkit: Essential Research Reagent Solutions

Table 3: Key Research Reagents and Resources for Ecotoxicology ML

Item / Resource	Type	Function in Research	Example / Source
Curated Benchmark Datasets	Data	Provides standardized, ML-ready data for fair model comparison and reproducibility.	ADORE dataset [1]; EnviroTox [1]
Molecular Representation Tools	Software Library	Encodes chemical structures into numerical features for ML models.	RDKit (for fingerprints), mol2vec for embeddings [5]
Phylogenetic Information	Data	Provides evolutionary distance metrics between species, used as a feature to model biological similarity in toxicity response.	Phylogenetic distance matrices included in ADORE [2]
Toxicity Knowledgebases	Data	Primary source of experimental ecotoxicity results for curation and expansion.	US EPA ECOTOX database [1]; PubChem [58]
Graph Neural Network Frameworks	Software Library	Enables building models that learn directly from molecular graph structures.	PyTorch Geometric; Deep Graph Library
Model Validation Suites	Software/Methodology	Ensures robust evaluation, prevents data leakage, and assesses applicability domain.	Fixed scaffold splits [1]; OPERA software's AD assessment [58]

The systematic comparison of models on the ADORE benchmark demonstrates that while GNNs represent the current state-of-the-art for predicting toxicity within a species or taxon, the problem of cross-taxa prediction remains a significant hurdle. The performance gap indicates that superior chemical representation alone is insufficient for reliable extrapolation across biological systems.

Future optimization efforts should focus on:

• Enhanced Biological Feature Integration: More sophisticated incorporation of species' physiological, metabolic, and genomic traits to build a predictive bridge between taxa.
• Advanced Transfer Learning Algorithms: Developing ML architectures explicitly designed to learn transferable toxicological principles from source taxa (e.g., algae, crustaceans) and apply them to target taxa (e.g., fish).
• Hierarchical and Meta-Learning Approaches: Exploring models that learn at multiple biological scales, from molecular targets to organismal physiology, potentially leveraging the phylogenetic tree as a structural guide for knowledge transfer.

The establishment and adoption of community benchmarks like ADORE are pivotal. They provide the common ground necessary to objectively measure progress toward the ultimate goal: accurate, reliable in silico models that can reduce our dependency on animal testing in environmental safety assessment [3].

Balancing Dataset Size with Quality and Diversity

Thesis Context: The Imperative for Benchmark Datasets in Ecotoxicology ML

The application of machine learning (ML) in ecotoxicology holds transformative potential for chemical hazard assessment, promising to reduce reliance on animal testing, lower costs, and accelerate the evaluation of environmental risks [2]. However, the field's progress is intrinsically linked to the availability of standardized, high-quality data. Unlike mature ML fields with established benchmarks like ImageNet or CIFAR, ecotoxicology has historically lacked a common ground for model training, benchmarking, and comparison [2] [3]. This absence creates a significant barrier to entry for ML experts and hinders reproducible, comparable research.

The core challenge lies in the fundamental tension between dataset size, quality, and diversity. A large dataset of homogeneous, single-species experiments may train a high-performing but narrowly applicable model. Conversely, a small, incredibly diverse dataset covering many species and chemicals may be statistically inadequate for robust ML. The creation of effective benchmark datasets, therefore, requires a deliberate balancing act—curating data that is sufficiently expansive to train complex models, meticulously quality-controlled to ensure reliability, and biologically diverse enough to yield insights that are generalizable across the environmental contexts ecotoxicology aims to protect [4] [3]. This guide examines current dataset initiatives through this lens, providing researchers with a framework for evaluation and application.

Comparative Analysis of Ecotoxicology Dataset Strategies

The following tables compare key datasets and data resources based on their approaches to balancing size, scope, and quality for ML applications.

Comparison of Dataset Scale and Diversity

This table contrasts the foundational ADORE benchmark with other dataset types, highlighting differences in primary purpose, scale, and compositional diversity.

Table 1: Scale and Compositional Diversity of Featured Datasets

Dataset / Resource Name	Primary Type & Purpose	Key Subjects/Chemicals	Scale (Records/Experiments)	Key Diversity Features
ADORE Benchmark Dataset [4] [2] [3]	Integrated ML Benchmark for predicting acute aquatic toxicity.	600+ chemicals; 140+ species (Fish, Crustaceans, Algae).	~15,000 curated experimental endpoints.	High taxonomic diversity across three groups; includes phylogenetic data & multiple molecular representations for chemicals [2].
Null LC-MS/MS Findings (Brazilian Seafood) [59]	Empirical Environmental Monitoring dataset reporting non-detects.	17 pharmaceuticals; 5 seafood species.	Measurements from multiple tissue samples.	Provides real-world "null" baseline; complemented by in-silico PBT/PMT indicators for priority setting [59].
WFSR Food Safety Mass Spectral Library [60]	Standardized Analytical Reference library for compound identification.	1,001 food toxicants (pesticides, vet drugs, toxins).	6,993 manually curated MS/MS spectra.	Spectral diversity via 7 collision energies; includes ~22% compounds unique among public libraries [60].
U.S. DOI/EPA LC-MS Datasets [61] [62]	Disparate Environmental Surveillance data from monitoring studies.	Varies by project (e.g., pesticides, cyanotoxins, PFAS).	Varies from 1 to 25+ discrete datasets.	Method and matrix diversity (water, sediment, biota); reflects regional and temporal monitoring priorities.

Comparison of Quality and Usability Characteristics

This table evaluates the datasets based on their quality controls, readiness for ML, and inherent limitations.

Table 2: Quality, Readiness for ML, and Limitations

Dataset / Resource Name	Curation & Quality Control	ML Readiness & Features	Primary Limitations & Challenges
ADORE Benchmark Dataset [4] [2]	Compiled from reputable sources; provides fixed train-test splits to prevent data leakage from repeated experiments [2].	High. Includes predefined "challenges," chemical descriptors (fingerprints, Mordred), and species phylogeny. Designed for direct model comparison [3].	Complexity of multi-species prediction. Requires biological knowledge for optimal use of taxonomic features.
Null LC-MS/MS Findings (Brazilian Seafood) [59]	High-quality empirical LC-MS/MS analysis with documented detection limits. PBT/PMT data from standardized in-silico tools (OPERA, EPI Suite) [59].	Low for direct ML. Serves as specialized validation or baseline data. In-silico indicators can be used as supplementary features.	Small scale; focused on non-detects. Useful for contextualizing positive findings elsewhere.
WFSR Food Safety Mass Spectral Library [60]	High manual curation; spectra acquired under standardized conditions on one instrument; quality controls injected [60].	Medium-High. Excellent for developing ML models for spectral matching or compound classification. Adheres to FAIR principles.	Limited to compounds relevant to food safety. Acquired in positive ionization mode only (as of publication).
U.S. DOI/EPA LC-MS Datasets [61] [62]	Quality varies by individual study. Typically follow agency protocols but lack cross-study harmonization.	Generally Low. Heterogeneous in methods and reporting. Requires significant preprocessing, fusion, and curation to be usable for ML.	Fragmented; not designed as a unified ML resource. Missing consistent ontological annotations across studies.

Detailed Experimental Protocols

Protocol: Construction of the ADORE Benchmark Dataset

This protocol outlines the multi-stage process for creating a curated, ML-ready dataset from disparate ecotoxicological sources [2] [3].

• Data Sourcing and Aggregation: Experimental data on acute aquatic toxicity (EC50/LC50 values) for fish, crustaceans, and algae were compiled from existing high-quality databases and the peer-reviewed literature. The core data point is the measured concentration causing 50% effect or lethality for a specific chemical-species pair.
• Curation and Deduplication: Records were examined for completeness and consistency. Critical Step: Repeated experiments for the same chemical-species pair were identified but kept as separate entries to reflect biological variability. This is crucial for subsequent splitting strategies.
• Feature Engineering and Expansion:
  • Chemical Representation: For each chemical, multiple representations were calculated or retrieved: a) 2D molecular descriptors (e.g., via Mordred), b) Multiple structural fingerprints (MACCS, PubChem, Morgan, ToxPrints), c) Molecular embeddings (e.g., mol2vec) [2] [3].
  • Biological Representation: For each species, data were added on: a) Ecological and life-history traits, b) Phylogenetic distance matrices to quantify relatedness between all species [2].
• Design of Train-Test Splits and Challenges: To prevent data leakage and enable benchmarking, fixed data splits were created. Crucially, splits were designed by chemical, ensuring all repeated experiments for a given chemical resided exclusively in either the training or test set [2] [3]. Specific "challenges" were defined (e.g., prediction for all fish, or for a single model species like D. magna) with corresponding splits.
• Documentation and Packaging: The complete dataset, with raw data, added features, and predefined splits, was packaged and shared with comprehensive metadata describing all fields and processing steps [4].

Protocol: Development of a Standardized HRMS Spectral Library

This protocol describes the creation of a high-quality, curated mass spectral library, as exemplified by the WFSR Food Safety Mass Spectral Library [60].

• Standards and Sample Preparation:
  • Chemical Standards: Obtain pure analytical standards for target compounds (e.g., 1,001 food toxicants).
  • Solution Preparation: Prepare stock solutions and subsequent working standard mixtures. For isobaric or enantiomeric compounds, prepare individual injections to ensure spectral purity [60].
  • Quality Control (QC): Prepare separate QC mixtures of representative compounds (e.g., 18 pesticides at 20 ng/mL) for sequence monitoring.
• Standardized LC-HRMS/MS Analysis:
  • Chromatography: Use a defined UHPLC system and column (e.g., Waters BEH C18). Employ a consistent gradient and mobile phase composition (e.g., water/methanol with ammonium formate and formic acid) [60].
  • Mass Spectrometry: Operate in positive (or negative) electrospray ionization mode on a high-resolution instrument (e.g., Orbitrap IQ-X Tribrid).
  • Data-Dependent Acquisition (DDA): Use inclusion lists to trigger MS/MS scans. For each compound, acquire spectra at multiple, defined collision energies (e.g., NCE 15, 30, 45, 60, 75, 90 and stepped energies 25, 38, 59) to capture comprehensive fragmentation patterns [60].
  • System Suitability: Inject solvent blanks between samples to minimize carryover. Analyze QC samples at the start, throughout, and end of the acquisition sequence to monitor system stability.
• Spectral Processing and Curation:
  • Data Extraction: Extract consensus MS/MS spectra for each compound-collision energy pair from the raw data files.
  • Manual Curation: Visually inspect spectra for quality, removing artifacts and ensuring correct precursor ion selection. This manual step is critical for library reliability [60].
  • Metadata Annotation: Annotate each spectrum with mandatory metadata: compound name, CAS number, SMILES, InChIKey, molecular formula, precursor m/z, retention time, and collision energy.
• Library Assembly and Dissemination: Compile curated spectra and metadata into a searchable library format. Distribute the library via public repositories (e.g., GNPS) and a dedicated website, ensuring compliance with FAIR data principles [60].

Protocol: Inter-Laboratory Study for Non-Targeted Analysis (NTA)

This protocol, based on a Norman Network study, is designed to assess the reproducibility of NTA workflows across different laboratories [63].

• Study Design and Sample Preparation:
  • Sample Generation: Deploy passive samplers (e.g., HLB disks) in a standardized manner at a defined environmental site (e.g., river water intake and treated drinking water). Use a common extraction and cleanup protocol for all samplers [63].
  • Sample Pooling and Distribution: Pool extracts from multiple samplers to create homogeneous samples. Ship identical sets of sample vials (extracts, procedural blanks, internal standard mixes) to all participating laboratories (e.g., 21 labs in 11 countries) [63].
• Harmonized and In-House Analysis:
  • Pre-Defined Method: Provide all participants with a detailed, standardized LC-HRMS method specifying column type, gradient, and MS acquisition parameters. Participants should replicate this as closely as possible on their instrumentation [63].
  • In-House Method: Each participant also analyzes the samples using their laboratory's own optimized, routine NTA LC-HRMS method.
• Data Collection and Submission:
  • Participants are required to submit their raw data files, processed feature lists, and a list of identified compounds with associated confidence levels (e.g., following the Schymanski scale) [63].
• Centralized Data Analysis and Benchmarking:
  • The collected dataset, containing raw data from multiple instrument platforms and processing workflows, serves as a benchmark.
  • Analysis focuses on evaluating the consensus and variability in features detected and compounds identified across labs and methods, highlighting the impact of workflow choices on NTA results [63].

Visualizing Dataset Relationships and Workflows

ADORE Dataset Curation and ML Benchmarking Workflow

Diagram 1: ADORE Dataset Curation and ML Benchmarking Workflow

This diagram illustrates the pipeline for constructing the ADORE benchmark dataset, from aggregating raw ecotoxicological data to creating the structured challenges for machine learning model comparison [2] [3].

Spectral Library Cross-Platform Comparison Logic

Diagram 2: Spectral Library Cross-Platform Comparison Logic

This diagram outlines the experimental logic used to evaluate the compatibility and optimal use conditions between mass spectral libraries generated on different instrumental platforms (QqTOF vs. Orbitrap) [64].

The Scientist's Toolkit: Essential Research Reagents & Materials

Table 3: Key Reagents and Materials for Ecotoxicology ML Dataset Development

Item	Primary Function in Dataset Development	Example/Note
Analytical Reference Standards	Provide ground truth for chemical identification and quantification in experimental studies or for building spectral libraries.	Pure compounds for target analytes (e.g., pharmaceuticals, pesticides) [59] [60].
Passive Sampling Devices	Integratively concentrate trace-level contaminants from water for non-targeted analysis, providing time-weighted average concentrations.	Horizon Atlantic HLB-L disks used in inter-laboratory studies [63].
Performance Reference Compounds (PRCs)	Used with passive samplers to calibrate sampling rates and account for environmental conditions (e.g., water flow).	Deuterated or ¹³C-labeled compounds pre-spiked onto silicone sheets before deployment [63].
Chromatography Columns & Buffers	Enable reproducible separation of complex mixtures prior to mass spectrometric analysis.	e.g., Waters BEH C18 column; mobile phases with ammonium formate & formic acid [60].
In-Silico Prediction Suites	Generate consistent digital descriptors for chemicals to augment experimental data with predicted properties.	OPERA, EPI Suite (KOCWIN, BCFBAF), ECOSAR for PBT/PMT and toxicity indicators [59].
Molecular Representation Tools	Translate chemical structures into numerical or binary formats suitable for machine learning algorithms.	Software to generate Mordred descriptors, Morgan fingerprints, or mol2vec embeddings [2].
Phylogenetic Analysis Software	Quantify evolutionary relationships between species to create features that capture biological similarity.	Used to generate phylogenetic distance matrices for species in a dataset [2].
Standardized Spectral Libraries	Serve as authoritative references for compound identification via mass spectral matching in non-targeted analysis.	e.g., NIST, MassBank, or the WFSR Food Safety library [60] [64].

Ensuring Robustness: Validation Frameworks and Comparative Benchmarking

The advancement of machine learning (ML) in ecotoxicology promises to revolutionize chemical hazard assessment, offering a path to reduce extensive and costly animal testing [1] [2]. However, the transition from research prototypes to regulatory-grade tools is hindered by inconsistent validation practices and a narrow focus on simple accuracy metrics [65] [66]. This guide, framed within the broader thesis on benchmark datasets for ecotoxicology, provides a comparative analysis of performance evaluation strategies. It argues that rigorous validation must extend beyond basic metrics to encompass dataset design, bias quantification, and ecological realism to ensure reliable, transparent, and fair ML applications for environmental protection [65] [30].

Performance Comparison of Ecotoxicology Benchmark Datasets and Models

The core challenge in developing reliable ML models for ecotoxicology is the scarcity of standardized, high-quality data. Benchmark datasets are foundational for comparable progress. The table below summarizes key characteristics of two pivotal datasets: the broad ADORE dataset for aquatic toxicity and the specialized ApisTox dataset for pollinator protection.

Table: Comparative Analysis of Ecotoxicology Benchmark Datasets

Dataset	ADORE (Aquatic Ecotoxicology) [1] [2] [3]	ApisTox (Pollinator Ecotoxicology) [7]
Primary Focus	Acute aquatic toxicity (mortality) for fish, crustaceans, algae.	Contact and oral toxicity of pesticides to honey bees (Apis mellifera).
Data Source	Curated from the US EPA ECOTOX database.	Curated from ECOTOX, PPDB, and BPDB databases.
Core Endpoints	LC50/EC50 values (log-transformed molar concentration).	Binary classification (toxic vs. non-toxic).
# of Compounds	~2,800 unique chemicals [30].	1,035 compounds.
# of Species	1,267 species [30].	Single species (honey bee).
Key Features	Chemical descriptors, species traits, phylogenetic data.	Chemical structures, pre-defined challenging train-test splits.
Defined Splits	Yes, based on chemical scaffolds & taxonomy to prevent data leakage.	Yes, including maximum diversity (MaxMin) and time-based splits.
Primary ML Task	Regression (predict continuous LC50).	Binary classification.
Stated Purpose	Provide a standard benchmark for comparing ML model performance across a wide ecological scope.	Evaluate ML performance on a specific, ecologically critical species with challenging generalization tasks.

The choice of validation metric is intrinsically linked to the model's task and real-world application. Accuracy alone is often misleading, especially with imbalanced data [67]. The following table compares the utility of standard and advanced validation metrics, as applied in recent ecotoxicology and broader ML research.

Table: Validation Metrics for Ecotoxicology ML Models

Metric Category	Specific Metric	Definition & Formula	Interpretation & Use Case in Ecotoxicology
Standard Performance [67]	Accuracy	(TP+TN) / Total Predictions	Misleading for imbalanced data. Unsuitable if non-toxic compounds vastly outnumber toxic ones [7].
	Precision	TP / (TP + FP)	Critical for prioritizing testing. High precision minimizes false alarms, saving resources on follow-up testing of safe chemicals.
	Recall (Sensitivity)	TP / (TP + FN)	Essential for risk avoidance. High recall ensures truly toxic chemicals are rarely missed, protecting ecosystems.
	F1-Score	2 * (Precision * Recall) / (Precision + Recall)	Balanced measure for class-imbalanced tasks. Useful for bee toxicity classification where both false positives and negatives are costly [7].
	Mean Absolute Error (MAE) / Root Mean Squared Error (RMSE)	Average/root-squared difference between predicted and true continuous values.	Standard for regression tasks (e.g., LC50 prediction). RMSE penalizes large errors more heavily.
Advanced & Holistic Validation [65] [68]	Expected Calibration Error (ECE)	Σ ( |Accuracy(Binm) - Confidence(Binm)| * |Bin_m|/N )	Measures if a model's predicted confidence matches its actual accuracy. Crucial for reliable risk assessment where confidence matters.
	Region of Practical Equivalence (ROPE) Coverage [65]	Proportion of predictions within a predefined "negligible error" margin.	Evaluates clinical/regulatory utility. For example, what percentage of predicted LC50s are within a 2-fold error margin of the true value?
	Bias Quantification (e.g., Statistical Parity Difference) [65] [68]	Difference in positive prediction rates between subgroups (e.g., chemical classes, taxonomic groups).	Detects if a model is systematically more accurate for certain chemical families (e.g., organophosphates) than others (e.g., neonicotinoids).
	Green Efficiency Weighted Score (GEWS) [69]	Weighted sum of normalized metrics (AUC, Log Loss, Training Time, CO2 Emissions, Latency).	Promotes sustainable AI by benchmarking models on accuracy, speed, and carbon footprint for large-scale deployment.

Experimental Protocols for Rigorous Validation

Adhering to detailed experimental protocols is non-negotiable for reproducibility and meaningful comparison. This section outlines critical methodologies from recent research.

Benchmark Dataset Construction (ADORE Protocol)

The creation of the ADORE dataset established a rigorous protocol for curating ecotoxicology data for ML [1].

• Data Source & Filtering: Core data was extracted from the US EPA's ECOTOX database (Sept 2022 release). Entries were filtered for three taxonomic groups: fish, crustaceans, and algae. Only acute (≤96h) mortality-related endpoints (LC50/EC50) from standardized OECD test guidelines were retained. In vitro tests and embryo life stages were excluded to focus on whole-organism vertebrate and invertebrate data relevant to regulations like REACH [1].
• Feature Engineering: The dataset was enriched with two critical feature types:
  • Chemical Representations: Six molecular representations were provided, including fingerprints (MACCS, PubChem, Morgan, ToxPrints), Mol2Vec embeddings, and Mordred descriptors [2] [3].
  • Biological Representations: Species were described using phylogenetic distance matrices and ecological traits (e.g., habitat, feeding behavior) [2] [3].
• Train-Test Splitting to Prevent Leakage: To avoid inflated performance from memorizing similar data, splits were not random. Instead, "chemical scaffold splits" ensured that chemicals with similar molecular backbones were entirely in either the training or test set. "Taxonomic splits" were used for cross-species prediction challenges [1] [2].

Pairwise Learning for Matrix Completion

A novel approach to predicting toxicity for untested chemical-species pairs uses pairwise learning, treating the problem as a matrix completion task [30].

• Data Matrix Formulation: Toxicity data (e.g., log LC50) is structured into a sparse matrix with rows as chemicals and columns as species. In one study, this resulted in a 3,295 (chemicals) x 1,267 (species) matrix with only 0.5% of cells filled with experimental data [30].
• Model Architecture (Factorization Machine): The model predicts the toxicity y for a chemical-species pair using the equation: y(x) = w₀ + Σ wᵢxᵢ + Σ Σ xᵢxⱼ Σ vᵢ,ₖvⱼ,ₖ where w₀ is a global bias, wᵢ are weights for chemical/species/duration, and v vectors learn latent interactions ("lock-and-key") between chemicals and species [30].
• Validation Strategy: Performance was validated by measuring the Root Mean Square Error (RMSE) on held-out chemical-species pairs. The model's output was used to generate novel hazard assessment tools like full-coverage Hazard Heatmaps and multi-species Species Sensitivity Distributions (SSDs) [30].

Framework for Quantifying Algorithmic Bias

A framework from clinical AI sleep scoring provides a transferable protocol for bias analysis in ecotoxicology [65].

• Performance Distribution Analysis: Instead of reporting only mean accuracy, the framework models the entire distribution of a performance metric (e.g., F1-score) or error across a population. This is done using Generalized Additive Models for Location, Scale, and Shape (GAMLSS) to show how metrics vary [65].
• Bias Dependency on External Factors: The model explicitly tests how predictive errors depend on external factors like chemical properties (e.g., logP, molecular weight) or taxonomic group. This identifies if a model is systematically less accurate for, say, highly hydrophobic compounds or for algae versus fish [65].
• Utility Assessment via ROPE: For continuous predictions (e.g., LC50), the framework calculates the "Region of Practical Equivalence" (ROPE) coverage. This determines the proportion of predictions where the error falls below a predefined, regulatory-relevant threshold (e.g., a 2-fold concentration difference), offering a pragmatic measure of model utility [65].

Workflow and Conceptual Diagrams

Rigorous ML Validation Workflow for Ecotoxicology

Pairwise Learning for Chemical-Species Toxicity Prediction

The Scientist's Toolkit: Research Reagent Solutions

Table: Essential Resources for Ecotoxicology ML Research

Resource Name / Type	Primary Function in Validation	Key Features & Relevance
ADORE Dataset [1] [2] [3]	Standardized Benchmarking	Provides curated acute toxicity data for fish, crustaceans, and algae with fixed splits to prevent data leakage, enabling direct model comparison.
ApisTox Dataset [7]	Specialized Model Validation	Offers a high-quality, curated dataset for bee toxicity with challenging splits, testing model generalization for a critical pollinator species.
US EPA ECOTOX Database [1]	Primary Data Source & External Validation	A comprehensive knowledgebase of ecotoxicity studies. Serves as the source for curated benchmarks and a pool for creating independent external test sets.
OECD Test Guidelines (e.g., TG 203, 202, 201) [1]	Defining Data Quality Standards	Provide the standardized experimental protocols (e.g., 96h fish test) that define the regulatory-relevant data included in benchmarks.
SHAP / LIME [66] [68]	Model Explainability & Mechanistic Insight	Post-hoc explanation tools that help interpret model predictions by quantifying feature contribution, linking predictions to chemical structures or species traits.
LibFM Library [30]	Implementing Pairwise Learning	Software library for factorization machines, enabling the implementation of advanced matrix completion models for predicting toxicity across chemical-species pairs.
GAMLSS Framework [65]	Quantifying Bias & Performance Distributions	A statistical framework used to model not just the mean but the entire distribution of model performance or error as a function of external factors.
Chemical Descriptor Tools (RDKit, Mordred) [2] [3]	Generating Molecular Features	Software for calculating chemical fingerprints and molecular descriptors, which are essential numerical representations for model input.
Phylogenetic Distance Matrices [2] [3]	Incorporating Biological Relatedness	Data structures that encode evolutionary relationships between species, used as features to inform models about expected similarity in toxicological response.

Comparative Model Evaluation on Standardized Benchmark Challenges

The application of machine learning (ML) in ecotoxicology promises to revolutionize chemical hazard assessment by reducing reliance on costly and ethically challenging animal testing [2]. However, meaningful progress depends on the ability to objectively compare the performance of different computational models. Standardized benchmark datasets serve as the essential common ground for this comparison, enabling researchers to evaluate models on identical data with consistent splitting strategies, thereby isolating model architecture and algorithm as the primary variables [1] [3].

This guide provides a comparative evaluation of contemporary ML models using the most current benchmark datasets in ecotoxicology, primarily the ADORE (Aquatic Toxicity) and ApisTox (bee toxicity) datasets. Framed within a broader thesis on benchmark datasets, this analysis highlights how standardized resources are catalyzing a shift from fragmented studies to a cohesive, reproducible, and rapidly advancing field. The comparative data and methodologies presented are intended to inform researchers, scientists, and drug development professionals in selecting and developing models for predictive ecotoxicology.

Benchmark Dataset Landscape and Core Challenges

The effectiveness of any model evaluation is intrinsically linked to the quality and design of the underlying dataset. Modern ecotoxicology benchmarks are curated not merely as data collections but as frameworks that define specific prediction challenges.

The following table summarizes the key characteristics of the two leading benchmark datasets, which cater to different but complementary aspects of ecotoxicological prediction.

Table 1: Characteristics of Primary Ecotoxicology Benchmark Datasets

Dataset	ADORE (Aquatic Toxicity) [1] [29]	ApisTox (Honey Bee Toxicity) [8] [7]
Core Focus	Acute aquatic toxicity (LC50/EC50) for three taxonomic groups.	Contact/oral toxicity (LD50) to the honey bee (Apis mellifera).
Taxonomic Scope	Fish, Crustaceans, Algae (203 total species).	Single species (non-target pollinator).
Data Source	Curated from the US EPA ECOTOX database.	Curated from ECOTOX, PPDB, and BPDB databases.
Key Endpoints	Mortality, immobilization, population growth inhibition.	Lethality (binary classification: toxic/non-toxic).
Number of Compounds	~1,900 (in core mortality dataset).	1,035 compounds.
Unique Value	Integrated chemical, species-phylogenetic, and experimental data; predefined data splits for multiple prediction challenges.	Largest curated public dataset for bee toxicity; includes time-based splits to test model generalizability to newer compounds [7].
Primary ML Task	Regression (predict continuous LC50) & Classification (toxicity brackets).	Binary classification.

Defined Prediction Challenges and Data Splitting Strategies

A critical innovation of modern benchmarks like ADORE is the provision of predefined, non-random data splits designed to prevent data leakage and test specific model capabilities [1] [3]. These splits form the basis of the "challenges" used for model evaluation.

Table 2: Standardized Prediction Challenges in the ADORE Benchmark [5]

Challenge Name	Training Data	Testing Data	Objective	Complexity
F2F, A2A, C2C	Single taxonomic group (Fish, Algae, or Crustaceans).	Same group, unseen chemicals.	Predict toxicity for new chemicals within a known species group.	Intermediate
AC2F-same	Algae + Crustaceans.	Fish (overlapping chemicals with training).	Cross-taxa prediction: Use surrogate species to predict fish toxicity for known chemicals.	High
AC2F-diff	Algae + Crustaceans.	Fish (novel chemicals not in training).	Cross-taxa & chemical prediction: The most rigorous test of generalizability.	Very High

The diagram below illustrates the logical relationships between the core data sources, the curated benchmark datasets, and the specific prediction challenges they enable.

Diagram 1: Ecotoxicology benchmark ecosystem for model evaluation.

Comparative Model Performance on Standardized Challenges

Recent comparative studies provide direct performance metrics for a wide array of ML models on the ADORE benchmark challenges. The results highlight significant differences between traditional methods, deep learning, and specialized graph-based approaches.

Performance Across Algorithm Types and Representations

A comprehensive 2025 study evaluated 161 distinct models on ADORE, combining multiple molecular representations with different algorithms [5].

Table 3: Comparative Performance of ML Models on ADORE Intra-Taxa Challenges (AUC Scores) [5]

Model Category	Specific Algorithm	Fish (F2F)	Algae (A2A)	Crustaceans (C2C)	Notes
Traditional ML	Random Forest (RF)	0.842 - 0.921	0.879	0.868	Performance varies with fingerprint type.
	Support Vector Machine (SVM)	0.810 - 0.903	0.854	0.849	Similar dependency on input representation.
	XGBoost (XGB)	0.848 - 0.924	0.891	0.879	Often top performer among non-graph ML.
Deep Neural Network	DNN	0.825 - 0.909	0.865	0.861	Less sensitive to fingerprint choice than traditional ML.
Graph Neural Networks	Graph Convolutional Network (GCN)	0.982 - 0.992	0.989	0.988	Consistently best performer.
	Graph Attention Network (GAT)	0.974 - 0.987	0.983	0.981	Very close second to GCN.
	AttentiveFP	0.961 - 0.979	0.975	0.973	Strong, but slightly lower than GCN/GAT.

Key Finding: Graph Neural Networks (GNNs), particularly GCN and GAT, decisively outperformed all traditional ML and deep learning models on the intra-taxa classification tasks, achieving Area Under the ROC Curve (AUC) scores above 0.98 [5]. This suggests GNNs' inherent ability to directly process molecular graph structure is superior to using predefined molecular fingerprints as features.

Performance on Advanced Cross-Taxa and Extrapolation Challenges

The more difficult challenges test a model's ability to generalize across biological domains and to novel chemical spaces.

Table 4: Model Performance on ADORE Cross-Taxa Challenges [5]

Model	Representation	AC2F-same (AUC)	AC2F-diff (AUC)	Performance Drop
GAT (Best)	Graph	0.821	0.808	~1.6%
GCN (2nd Best)	Graph	0.819	0.802	~2.1%
DNN (Best Non-Graph)	MACCS Fingerprint	0.785	0.821	-4.6% (Gain)
Random Forest	Morgan Fingerprint	0.762	0.728	~4.5%

Key Findings:

• Overall Performance Drop: Even the best GNN models experienced a significant decrease in performance (AUC drop of ~17 percentage points from intra-species to cross-species prediction), underscoring the difficulty of the cross-taxa extrapolation task [5].
• Generalization to Novel Chemicals: In the most challenging AC2F-diff scenario (predicting fish toxicity for completely new chemicals using algae/crustacean data), a Deep Neural Network (DNN) using MACCS fingerprints unexpectedly achieved the best result (AUC 0.821) [5]. This indicates that for extreme extrapolation, simpler models with robust feature sets can sometimes generalize more effectively than complex GNNs, which may overfit to the training chemical space.

Alternative Paradigm: Pairwise Learning for Data Gap Filling

A separate 2025 study employed a pairwise learning approach (Bayesian matrix factorization) on the ADORE dataset to address the critical problem of sparse data—predicting toxicity for the 99.5% of chemical-species pairs lacking experimental data [30]. This method treats species and chemicals as equally important covariates to learn their interaction.

Table 5: Performance of Pairwise Learning Model for LC50 Prediction [30]

Model Type	Description	Mean Absolute Error (MAE - log mol/L)	Key Capability
Mean Model	Learns average chemical & species effects.	0.93	Baseline for chemical- or species-wise trends.
Pairwise Model	Learns chemical-species-duration interactions ("lock & key").	0.69	Predicts missing interactions in the matrix.
Ideal Model (Theoretical Upper Bound)	Fits each experiment separately.	0.55	Represents inherent noise in biological data.

Key Finding: The pairwise model significantly outperformed the mean model, reducing prediction error by 26%. Its accuracy approached the theoretical limit defined by inter-experimental variability, demonstrating its effectiveness in filling vast data gaps for hazard assessment [30].

Experimental Protocols for Benchmark Evaluation

To ensure reproducibility and fair comparison, studies adhere to detailed experimental protocols defined by the benchmark datasets and the scientific question.

General Workflow for Model Evaluation on ADORE

The standard protocol for comparative studies involves a structured pipeline from data selection to performance validation.

Diagram 2: Standardized workflow for model evaluation on ADORE.

Protocol for Pairwise Learning (Matrix Factorization)

The pairwise learning study followed a distinct protocol tailored to its objective of filling a sparse matrix [30]:

• Data Extraction: LC50 values for 3,295 chemicals and 1,267 species were extracted from the ADORE "mortality filtered" dataset, resulting in a matrix with only 0.5% coverage (70,670 data points).
• Feature Encoding: Chemicals, species, and exposure durations were treated as categorical variables. Each data point (chemical, species, duration triplet) was encoded into a sparse binary feature vector using one-hot encoding.
• Model Formulation: A second-order Factorization Machine model was used, defined by the equation: y(x) = w₀ + Σ wᵢxᵢ + Σ Σ xᵢxⱼ Σ vᵢ,ₖvⱼ,ₖ where w₀ is a global bias, wᵢ are bias terms for chemicals/species/duration, and v terms capture pairwise interaction factors.
• Training & Validation: The model was trained using Markov Chain Monte Carlo (MCMC) optimization for 2000 epochs. Validation was performed to compare against simpler mean and null models.
• Output Generation: The trained model predicted over 4 million missing LC50 values, which were used to generate hazard heatmaps, Species Sensitivity Distributions (SSDs), and Chemical Hazard Distributions (CHDs).

Protocol for Specialized Benchmark Evaluation (ApisTox)

Evaluation on the ApisTox benchmark emphasizes testing model generalizability over time [7]:

• Split Strategy: Use the provided "time split," where models are trained on compounds with earlier literature publication dates and tested on later ones, simulating real-world prediction of new pesticides.
• Model Diversity: Evaluate a wide suite of models, from simple molecular fingerprints with Random Forest to graph kernels and state-of-the-art pretrained graph transformers.
• Chemical Space Analysis: Prior to modeling, analyze the distinctness of ApisTox's agrochemical space compared to medicinal chemistry benchmarks (e.g., from MoleculeNet) using Tanimoto similarity and molecular filter pass rates (e.g., Lipinski, Ghose, Hao pesticide filters).
• Evaluation: Report standard classification metrics (AUC, Precision, Recall, F1) on the held-out test set. A key analysis point is the performance degradation of models that were state-of-the-art on medicinal chemistry benchmarks when applied to ApisTox.

The Researcher's Toolkit for Ecotoxicology ML

Table 6: Essential Research Reagent Solutions for Ecotoxicology ML

Tool / Resource	Type	Primary Function	Key Consideration
ADORE Dataset [1]	Benchmark Data	Provides standardized data & splits for aquatic toxicity prediction.	Use predefined challenges and splits to ensure comparable results.
ApisTox Dataset [8] [7]	Benchmark Data	Provides standardized data for honey bee toxicity prediction.	Utilize the time-split for testing generalizability to new compounds.
ECOTOX Database [1]	Primary Data Source	EPA-curated source of ecotoxicity test results.	Requires extensive curation and processing for ML use.
RDKit	Software Library	Open-source cheminformatics for molecule standardization, descriptor calculation, and fingerprint generation.	Essential for preprocessing chemical structures from SMILES.
scikit-learn	Software Library	Provides implementations of traditional ML algorithms (RF, SVM, etc.) and evaluation metrics.	Foundation for building and evaluating non-graph ML models.
PyTorch Geometric / DGL	Software Library	Frameworks for building and training Graph Neural Networks (GCN, GAT, etc.).	Necessary for implementing state-of-the-art graph-based models.
Mol2Vec / Mordred	Molecular Representation	Provides learned molecular embeddings or a large vector of chemical descriptors.	Alternative to fixed fingerprints; can capture richer chemical information.
LibFM [30]	Software Library	Implementation of Factorization Machines for pairwise learning and recommendation systems.	Key for matrix completion approaches to fill sparse ecotoxicity data.

Assessing Generalizability with External Validation Sets

In ecotoxicological machine learning (ML), the ultimate test of a model's value is its performance on truly novel data—chemicals, species, or experimental conditions it has never encountered during training. This capability, known as generalizability, is paramount for deploying models in regulatory decision-making, prioritizing chemicals for testing, or extrapolating hazards across the tree of life [70] [66]. However, assessing generalizability is complicated by the complex structure of ecotoxicological data, which includes repeated experiments, varying species sensitivities, and a vast, sparsely populated chemical space [1] [30].

The emergence of benchmark datasets like ADORE (Acute Aquatic Toxicity Database) has begun to address this challenge by providing a standardized foundation for model development and, crucially, for comparison [1] [2]. ADORE consolidates data on acute mortality for fish, crustaceans, and algae from the US EPA's ECOTOX database, augmented with chemical properties and species-specific phylogenetic information [4] [3]. Its creation underscores a key principle: model performances can only be fairly compared across studies when the same dataset, cleaning procedures, and data splitting strategies are used [1].

This guide focuses on the pivotal step that follows model training and internal validation: external validation. We objectively compare common validation strategies, analyze their performance implications using data from recent studies, and detail the experimental protocols that ensure rigorous, reproducible assessment of model generalizability within the framework of modern ecotoxicology benchmarks.

Comparative Analysis of Validation Strategies and Outcomes

The choice of how to partition data for training, internal validation, and external testing fundamentally shapes the perceived and actual generalizability of an ML model. The following table summarizes the core strategies, their implementation, and the type of generalizability they purport to test.

Table 1: Comparison of Common Validation Strategies in Ecotoxicology ML

Validation Strategy	Core Methodology	Intended Generalizability Test	Key Advantage	Primary Risk/Challenge
Random Split	Data points randomly assigned to train/test sets (e.g., 80/20).	Performance on a random subset of the overall data.	Simple to implement; maximizes data use.	Severe data leakage if repeated measures for the same chemical-species pair are split across sets, leading to over-optimistic performance [2] [3].
Scaffold Split (Chemical-Wise)	Split is based on molecular scaffolds; all data for chemicals with a given scaffold are in either train or test set.	Predictivity for novel chemical structures not represented in training.	Tests ability to extrapolate to new chemotypes; prevents chemical leakage.	Can be highly challenging; may underestimate performance for regulatory use on similar chemicals.
Time-Based Split	Data is split based on the date of publication or entry into a database.	Performance on newer data, simulating real-world prospective use.	Mimics practical application where future chemicals are unknown.	Requires curated temporal metadata; historical bias in tested chemicals may affect relevance.
Taxonomic/Group Split	All data for a specific taxonomic group (e.g., all algae) or species are held out as the test set.	Predictivity across different taxonomic groups or for a specific untested species.	Directly tests cross-species extrapolation, a major goal in ecological risk assessment [30].	Requires sufficient data for each group; may not reflect chemical diversity within the test group.
Pairwise Learning & Matrix Completion	Treats the chemical-species-toxicity matrix as sparse and aims to predict all missing entries [30].	Predictivity for novel chemical-species combinations.	Maximizes utility of sparse data; explicitly models the "lock and key" interaction.	Model complexity is high; validation requires careful hold-out of entire chemical-species pairs.

The impact of these strategies on model performance metrics is significant. Studies using the ADORE framework demonstrate that scaffold splits consistently yield more conservative and realistic performance estimates compared to random splits. For example, a study using pairwise learning on ADORE data for LC50 prediction reported that a model capturing chemical-species interactions significantly outperformed simpler baselines on scaffold-split data, demonstrating true utility for filling data gaps [30].

External validation on independently sourced datasets provides the strongest evidence of generalizability. A model predicting pesticide phytotoxicity, which integrated molecular and experimental descriptors, achieved an R² of 0.75 on an external validation set, confirming its robustness beyond its training data [28]. Similarly, a model for chemical transfer risk in breast milk maintained an accuracy of 86.36% on an external set, showing strong real-world applicability [71].

Table 2: Performance Outcomes from Different Validation Approaches in Recent Studies

Study Focus	Model Type	Internal Validation Performance	External / Rigorous Split Performance	Validation Strategy
Chemical Hazard Distributions [30]	Bayesian Pairwise Learning (Factorization Machine)	Not explicitly stated for internal split.	Outperformed null and mean models; enabled creation of full chemical-species hazard matrices.	Scaffold-based split on ADORE data; testing on novel chemical structures.
Pesticide Phytotoxicity [28]	XGBoost	R² = 0.69, RMSE = 0.80 (10-fold CV).	R² = 0.75, RMSE = 0.81.	External validation on a temporally/contextually distinct dataset.
Bee Toxicity (ApisTox) [7]	Various ML/DL models	Performance varied widely by model architecture.	Highlighted degradation in performance on scaffold (MaxMin) and time-based splits vs. random splits.	Scaffold (MaxMin) and time-based splits provided with ApisTox benchmark.
Chemical Transfer in Breast Milk [71]	Balanced Random Forest	AUC = 0.8708, Accuracy = 82.67%.	Accuracy = 86.36%.	External validation set from a separate source.

Detailed Experimental Protocols for Key Validation Approaches

To ensure reproducibility and proper comparison, below are detailed methodologies for two critical validation protocols used with benchmark datasets like ADORE.

Protocol 1: Scaffold-Based Splitting for Novel Chemical Generalizability

This protocol tests a model's ability to predict toxicity for chemicals with novel molecular frameworks [1] [7].

• Data Preparation: Start with a curated dataset containing chemical identifiers (e.g., SMILES, InChIKey) and associated toxicity endpoints. Standardize chemical structures (e.g., using RDKit): remove salts, neutralize charges, and generate canonical SMILES.
• Scaffold Generation: For each chemical, generate its Bemis-Murcko scaffold—the core molecular framework retaining ring systems and linkers but removing side-chain atoms.
• Split Assignment: Group all data points (entries) by their unique scaffold. Randomly assign entire scaffolds to either the training set (e.g., 80% of scaffolds) or the test set (20%). Crucially, all entries for a given scaffold reside in the same split. This guarantees no chemical analogue of the test set is seen during training.
• Model Training & Evaluation: Train the model exclusively on the training-set entries. Evaluate its final performance on the held-out test-set entries. Report metrics like RMSE, MAE, or R² for regression tasks, and AUC-ROC, accuracy, or F1-score for classification.

Protocol 2: Pairwise Learning for Chemical-Species Matrix Completion

This protocol, used to fill the vast gaps in the chemical-species toxicity matrix, employs a specialized validation setup [30].

• Matrix Formulation: Structure the data as a sparse matrix where rows represent unique species, columns represent unique chemicals, and cells contain the toxicity value (e.g., log(LC50)). Most cells are empty.
• Pairwise Data Encoding: Encode each observed data point (a chemical-species-toxicity triplet) for ML. Represent the chemical and species as categorical variables using one-hot encoding. The exposure duration can be included as an additional categorical feature.
• Train-Test Split for Pairs: Instead of splitting individual data points, split the observed chemical-species pairs. Hold out a fraction of these unique pairs (e.g., 20%) for testing. All experimental repeats or data points for a held-out pair belong to the test set.
• Model Training with Factorization: Employ a model capable of learning latent representations for chemicals and species, such as a Factorization Machine. The model learns a global bias, individual chemical and species biases, and latent factor vectors whose dot products estimate the interaction effect ("lock and key") for any pair.
• Validation: After training on the training pairs, predict toxicity for the held-out pairs in the test set. Performance on these held-out combinations directly measures the model's ability to generalize to unseen chemical-species interactions.

Visualizing Validation Workflows and Data Structures

Workflow for Validating Ecotoxicology ML Models

The diagram above illustrates the decision points in designing a validation strategy. The path taken after creating a benchmark dataset critically influences the assessment of model generalizability.

Matrix Structure for Pairwise Learning Validation

This diagram depicts the core challenge in ecotoxicology ML: data sparsity. In a matrix of species vs. chemicals, very few cells have experimental data (green). A rigorous validation protocol holds out entire species-chemical pairs (red) for testing. The model's task is to predict these and the millions of missing values (gray) based on learned patterns from the observed data [30].

Table 3: Key Research Reagent Solutions for Ecotoxicology ML Validation

Resource Name	Type	Primary Function in Validation	Source/Availability
ADORE Dataset	Benchmark Dataset	Provides a standardized, multi-feature dataset on aquatic toxicity with predefined, leakage-free splits for fish, crustaceans, and algae to enable direct model comparison [1] [2].	Nature Scientific Data [1] [4]; associated GitHub repositories.
ECOTOX Knowledgebase	Primary Data Source	The US EPA's comprehensive database of ecotoxicological test results; serves as the primary source for curating new benchmark datasets or external validation sets [1] [28].	US EPA website (public access).
ApisTox Dataset	Specialized Benchmark	A benchmark dataset for honey bee (Apis mellifera) toxicity with predefined MaxMin (scaffold) and time-based splits, facilitating validation for pollinator risk assessment [7].	Publication-associated data repositories.
RDKit	Cheminformatics Software	Open-source toolkit used for chemical standardization, scaffold generation, molecular descriptor calculation, and fingerprint generation—essential for preparing and splitting chemical data [7].	Open-source (www.rdkit.org).
OECD QSAR Toolbox	Regulatory Software	Provides methodologies for chemical grouping, read-across, and (Q)SAR model validation, aligning research workflows with regulatory expectations for assessing generalizability.	OECD (subscription).
SHAP (SHapley Additive exPlanations)	Explainable AI (XAI) Library	An XAI method used post-validation to interpret model predictions, identify key chemical or biological features driving toxicity, and build mechanistic understanding, which supports the biological plausibility of generalized predictions [71] [66] [28].	Open-source Python library.

The field of ecotoxicology faces a dual challenge: the ethical and financial burden of traditional animal testing and the pressing need to assess the environmental hazard of tens of thousands of chemicals in use [1]. Machine learning (ML) offers a promising in silico alternative, yet its adoption has been hampered by a lack of standardized datasets, making objective comparison of model performance difficult [2]. In response, the ADORE (A benchmark dataset for machine learning in ecotoxicology) dataset was introduced to provide a common ground for training, benchmarking, and comparing models in a standardized manner [2].

ADORE is a comprehensive, expert-curated dataset focusing on acute aquatic toxicity. Its core comprises experimental results for three ecologically relevant taxonomic groups—fish, crustaceans, and algae—extracted from the US EPA's ECOTOX database [1]. The dataset is richly annotated with chemical information (e.g., molecular fingerprints, descriptors) and species data (e.g., phylogenetic, ecological traits), designed specifically to overcome the barriers to entry for ML research in this domain [2]. This case study uses ADORE as the foundational benchmark to objectively compare the predictive performance of traditional machine learning methods against modern deep graph learning approaches, within the broader thesis that robust, community-adopted benchmarks are essential for advancing computational ecotoxicology.

Comparative Approaches: Traditional ML vs. Deep Graph Learning

The selection of a modeling approach is dictated by the nature of the data and the prediction task. ADORE provides data in both structured tabular form and as molecular graphs, enabling a direct comparison between two paradigms.

Traditional Machine Learning (ML) on Tabular Data

Traditional ML methods operate on fixed-feature, tabular data. For ADORE, this involves using pre-computed feature vectors to represent chemicals and species.

• Chemical Representation: This typically involves engineered molecular descriptors (e.g., Mordred descriptors) or fixed-length fingerprints (e.g., MACCS, Morgan fingerprints) that encode structural or physicochemical properties [2].
• Species Representation: Species are represented by ecological, life-history traits, and crucially, phylogenetic distance matrices, which encode the evolutionary relationships between species under the assumption that closely related species exhibit similar chemical sensitivities [2].
• Algorithms: Common and well-performing algorithms include Random Forest (RF), Support Vector Machines (SVM), Logistic Regression (LR), and gradient-boosting methods like XGBoost [72] [73]. These models are valued for their relative interpretability, computational efficiency, and strong performance on structured data.

Deep Graph Learning (Graph Neural Networks)

Deep graph learning, specifically Graph Neural Networks (GNNs), represents a paradigm shift by directly processing graph-structured data.

• Molecular Representation: A molecule is natively represented as an undirected graph, where atoms are nodes and bonds are edges [72]. This avoids potential information loss from pre-computed fingerprints.
• Model Architecture: Frameworks like Message Passing Neural Networks (MPNNs) operate through an iterative "message-passing" process. In each step, nodes aggregate information from their neighbors, allowing the model to learn complex, hierarchical representations of the molecular structure directly from the graph [72].
• Advantages: This approach automatically learns task-relevant features, which is particularly powerful for predicting biological activity linked to subtle structural motifs. Advanced variants like the Communicative MPNN (CMPNN) incorporate enhanced message-passing mechanisms to capture long-range dependencies within the molecule, further boosting predictive accuracy [72].

The fundamental distinction lies in feature engineering versus feature learning. Traditional ML relies on domain expertise to create informative features, while GNNs learn these representations directly and dynamically from the raw graph data.

ADORE Dataset Compilation and Modeling Pathways

Experimental Protocols & Performance Comparison

Key Experimental Protocols

Robust experimental design is critical for a fair comparison. Key methodological considerations drawn from studies on ADORE and related benchmarks include:

• Task Formulation: The core task is predicting acute toxicity, typically framed as a regression (predicting continuous LC50/EC50 values) or classification (e.g., toxic vs. non-toxic) problem [1].
• Data Splitting: To avoid data leakage and accurately assess generalization, a scaffold split is often used for chemicals, ensuring that molecules with similar core structures are not present in both training and test sets. For tasks involving new species, a species-out split is employed [2] [74].
• Model Training & Validation: Models are trained on the training set, with hyperparameters tuned using a separate validation set or via cross-validation. Performance is rigorously evaluated on a held-out test set that the model never sees during training [72].
• Evaluation Metrics: Common metrics include:
  • Regression: Mean Absolute Error (MAE), Root Mean Squared Error (RMSE), Coefficient of Determination (R²).
  • Classification: Accuracy, Precision, Recall (Sensitivity), F1-score, and Area Under the Receiver Operating Characteristic Curve (AUC-ROC) [74] [72].

The following tables synthesize quantitative findings from recent studies applying traditional ML and deep graph learning to toxicity prediction tasks, including those based on the ADORE principles.

Table 1: Performance Comparison on General Toxicity Prediction Tasks

Model Category	Specific Model	Dataset/Task	Key Performance Metric(s)	Performance Outcome	Reference
Traditional ML	Logistic Regression (LR)	GRAPE (eToxIQ Graph)	Recall	Baseline (Reported as inferior to GNN)	[74]
Traditional ML	Multi-Layer Perceptron (MLP)	GRAPE (eToxIQ Graph)	Recall	Baseline (Reported as inferior to GNN)	[74]
Deep Graph Learning	Graph Neural Network (GRAPE)	GRAPE (eToxIQ Graph)	Recall	Superior, up to 30% increase vs. LR/MLP	[74]
Deep Graph Learning	Graph Neural Network (GRAPE)	Novel Chemical Prediction	Accuracy (Count)	104 correct / 126 total	[74]
Deep Graph Learning	Graph Neural Network (GRAPE)	New Species Prediction	Accuracy (Count)	7 correct / 8 total	[74]

Table 2: Performance on Specific Endpoint Prediction (Reproductive Toxicity)

Model Category	Specific Model	Dataset/Task	Key Performance Metric(s)	Performance Outcome	Reference
Traditional ML	Random Forest (RF)	Reproductive Toxicity (SMILES)	AUC-ROC	Mediocre (Specific value not provided, outperformed by DL)	[72]
Traditional ML	XGBoost	Reproductive Toxicity (SMILES)	AUC-ROC	Mediocre (Specific value not provided, outperformed by DL)	[72]
Deep Graph Learning	Communicative MPNN (CMPNN)	Reproductive Toxicity (SMILES)	AUC-ROC	0.946 (Mean)	[72]
Deep Graph Learning	Communicative MPNN (CMPNN)	Reproductive Toxicity (SMILES)	Accuracy	0.857	[72]
Deep Graph Learning	Communicative MPNN (CMPNN)	Reproductive Toxicity (SMILES)	F1-Score	0.846	[72]

Table 3: Key Research Reagent Solutions for Ecotoxicology ML

Item/Category	Function & Description	Relevance to ADORE/Experiments
Molecular Representations	Convert chemical structure into machine-readable format. Fingerprints (MACCS, Morgan) and descriptors (Mordred) for ML; SMILES strings and molecular graphs for GNNs.	ADORE provides 6 molecular representations to enable research on optimal feature input [2].
Phylogenetic Distance Matrix	Encodes evolutionary relationships between species as pairwise distances, used as a feature to model interspecies sensitivity correlations.	Included in ADORE to leverage the assumption that related species have similar toxicological responses [2].
Toxicity Benchmark Datasets	Curated, standardized data for model training and benchmarking. ADORE (acute aquatic toxicity), eToxIQ (relation prediction), and others for specific endpoints.	Essential for reproducible research. ADORE provides fixed train-test splits to prevent data leakage and ensure fair comparison [2] [74].
Graph Neural Network Frameworks	Software libraries for building and training GNNs (e.g., PyTorch Geometric, Deep Graph Library (DGL)).	Used to implement models like MPNNs and CMPNNs for graph-based toxicity prediction [72].
Chemoinformatics Toolkits	Software for computing molecular features and handling chemical data (e.g., RDKit).	Used to generate molecular descriptors and fingerprints from SMILES strings for traditional ML models [75].
Benchmark Platforms	Platforms like the Open Graph Benchmark (OGB) that provide standardized datasets, data loaders, and evaluators for graph ML.	Exemplifies the benchmark paradigm that ADORE brings to ecotoxicology, ensuring unified evaluation [76].

Analysis of Results and Future Trajectories

Interpretation of Comparative Performance

The experimental data indicates a clear trend: deep graph learning methods, particularly GNNs, consistently match or surpass the performance of traditional ML models on toxicity prediction tasks. The GRAPE model's significant recall improvement and strong performance on novel chemicals/species demonstrate GNNs' superior ability to generalize and capture complex structure-activity relationships [74]. Similarly, the CMPNN's state-of-the-art results on reproductive toxicity highlight the advantage of deep, learnable representations over fixed molecular fingerprints [72].

This superiority can be attributed to the representational advantage of graphs. By learning directly from the atomic connectivity, GNNs can identify toxicophores and structural motifs critical for activity without relying on pre-defined feature sets, which may omit relevant information [77] [72].

Challenges and Future Directions

Despite their promise, deep graph learning approaches face challenges that align with broader issues in computational toxicology:

• Interpretability: The "black-box" nature of deep models is a significant hurdle for regulatory adoption. Future work must integrate explainable AI (XAI) techniques, such as SHAP or attention mechanism analysis, to elucidate which molecular substructures drive predictions [77] [75].
• Data Quality and Integration: Model performance is ultimately bounded by data quality and coverage. The field is moving towards multi-modal integration, combining chemical structure data with toxicogenomics (transcriptomics, proteomics) and high-content phenotypic imaging within frameworks like the Adverse Outcome Pathway (AOP) [77]. ADORE provides a foundation, but future benchmarks may incorporate these additional data layers.
• Causal Inference and Generalization: Moving beyond correlation to causal inference is a key frontier. This involves modeling the mechanistic steps of toxicity to improve predictions for truly novel chemical classes and reduce reliance on chemical similarity [75].

Graph Neural Network (GNN) Architecture for Toxicity Prediction

This case study, framed within the ADORE benchmark initiative, demonstrates that deep graph learning represents a significant advance over traditional machine learning for ecotoxicological prediction. GNNs' native ability to process molecular structure, coupled with their capacity to integrate diverse biological data (like species phylogeny from ADORE), provides a more powerful and generalizable framework.

The establishment of standardized, well-curated benchmarks like ADORE is foundational to this progress. It enables the rigorous, reproducible comparisons necessary to identify best practices and drive the field forward. The future of computational ecotoxicology lies in the development of interpretable, multi-modal, and causally-aware deep learning models, built upon and extending the benchmark principles exemplified by ADORE. This trajectory promises to deliver more reliable tools for chemical safety assessment, ultimately reducing dependence on animal testing and accelerating the identification of environmental hazards [2] [75].

Benchmark Datasets as the Foundation for Predictive Ecotoxicology The application of machine learning (ML) in ecotoxicology promises to revolutionize environmental hazard assessment by offering efficient, ethical alternatives to traditional animal testing [1]. However, the field's progress hinges on the availability of standardized, high-quality data that enables the fair comparison of different algorithmic approaches [2]. Benchmark datasets, such as the ADORE (Acute Aquatic Toxicity) dataset, have been created to provide this common foundation [1] [3]. These datasets are crucial for moving beyond isolated model metrics and toward generating actionable tools like Species Sensitivity Distributions (SSDs) and hazard maps, which directly inform chemical safety and environmental management [78] [79]. This guide compares key methodologies and resources in this translational pipeline, framed within the essential context of benchmark data for ecotoxicological ML research.

Comparison of Benchmark Datasets and Methodological Frameworks

The development of reliable predictive models begins with robust, well-curated data. The table below compares the scope and structure of prominent datasets and modeling frameworks designed for ecotoxicological ML.

Table 1: Comparison of Ecotoxicological Benchmark Datasets and Frameworks

Name / Focus	Core Description & Purpose	Taxonomic & Chemical Scope	Key Features & Provided Splits	Primary Use-Case
ADORE Dataset [1] [2] [41]	A benchmark dataset for predicting acute aquatic mortality (LC50/EC50). Designed to ensure model comparability.	Taxa: Fish, Crustaceans, Algae. Chemicals: ~1,905 organic compounds (fish subset).	Curated from EPA ECOTOX. Includes chemical descriptors (e.g., Mordred, fingerprints) and species traits (phylogeny, ecology). Provides fixed train-test splits to prevent data leakage [41].	Benchmarking ML models for cross-species toxicity prediction; foundational research.
SSD Expansion via ANN [78]	A methodology to generate SSDs for thousands of chemicals using Artificial Neural Networks (ANNs).	Taxa: 8 aquatic species (e.g., P. promelas, D. magna). Chemicals: 8,424 from Tox21 database.	Trains individual ANN models per species on molecular structure. Uses predicted LC50 values to fit SSD curves (log-normal, Weibull) via bootstrapping.	High-throughput screening of chemical hazards; deriving HC5/PNEC values for risk assessment.
Bayesian Network for Nanomaterials [80]	A Bayesian Network (BN) model to predict chronic toxicity of silver nanomaterials (AgNMs) in soils.	Taxa: Terrestrial organisms (various classes). Agents: Silver nanomaterials with varied physicochemical properties.	Incorporates material properties (size, coating), species info, and experimental conditions. Provides interpretable rules for hazard criteria.	Hazard assessment for advanced materials within Safe-and-Sustainable-by-Design (SSbD) frameworks.
Hazard Susceptibility Mapping [79]	A review of ML/DL workflows for creating spatial hazard susceptibility maps (e.g., for floods, pollution).	Hazards: Geospatial (floods, landslides, air pollution, urban heat islands).	Generalizable workflow: data preprocessing → feature selection → modeling → interpretation → map validation. Highlights Random Forest, ANN, SVM as common algorithms.	Spatial planning and risk management; translating model predictions into geospatial visualizations.

Comparison of Machine Learning Methodologies and Performance

Different ML approaches offer varying trade-offs between predictive accuracy, interpretability, and data requirements. The following table summarizes experimental outcomes and protocols from key studies.

Table 2: Comparison of ML Methodologies for Ecotoxicological Predictions

Study & Model	Target & Dataset	Key Experimental Protocol	Reported Performance & Findings	Advantages & Limitations
Gasser et al. (2024) - Tree-Based Models [41]	Target: log10(LC50) for fish.Data: ADORE "t-F2F" challenge (140 species, 1,905 chemicals).	Tested LASSO, RF, XGBoost, Gaussian Process. Used 6 molecular representations (e.g., Morgan fingerprint, Mordred). Implemented chemical split: all tests for a given chemical are in either train or test set to avoid leakage.	Best: RF and XGBoost. RMSE: 0.90 (approx. one order of magnitude on LC50 scale). Performance strongly dependent on data splitting strategy, weakly dependent on molecular representation [41].	Advantage: High predictive performance for regression. Limitation: Poor accuracy for individual chemical predictions; limited capture of taxonomic traits.
SSD via ANN (2021) [78]	Target: LC50 for 8 species.Data: ~2,521 curated data points from ECOTOX and literature.	Trained one ANN per species using selected molecular descriptors. Predicted LC50s for 8,424 Tox21 chemicals. Fitted SSD curves using bootstrapping (1,000 iterations).	Model R²: 0.54–0.75 (median 0.69). Generated SSDs for 8,424 chemicals, greatly expanding coverage. Provided HC5 values (hazardous concentration for 5% of species).	Advantage: Massive scale, directly outputs risk-assessment ready SSDs. Limitation: Performance varies by species; depends on quality of initial experimental data.
BN for AgNMs (2025) [80]	Target: Chronic NOEC for terrestrial species.Data: Literature-derived dataset on AgNM ecotoxicity in soils.	Incorporated features: NM properties (size, shape, coating), species class, exposure media. Network structure refined with expert insight. Model outputs interpretable probabilistic rules.	Average Predictive Accuracy: ~82% across output labels. Identified key influencing factors (e.g., surface treatment, particle size).	Advantage: High interpretability; handles uncertainty well; useful for early-stage material screening. Limitation: Specialized for nanomaterials; requires expert input for structure learning.

From Predictions to Practical Tools: Workflows and Visualizations

Translating model outputs into hazard maps and SSDs involves defined sequential workflows. The diagram below illustrates the general pipeline for creating a geospatial hazard susceptibility map, a common endpoint for environmental risk models [79].

A critical application in ecotoxicology is the generation of a Species Sensitivity Distribution (SSD), which transforms toxicity predictions for multiple species into a comprehensive risk metric for an entire ecosystem [78]. The following diagram details this process.

Experimental Protocols for Key Studies

For reproducibility and comparison, detailed methodologies are essential. Below are condensed protocols from two pivotal studies.

Protocol 1: Implementing the ADORE Fish Challenge with Tree-Based Models [41]

• Objective: To benchmark ML models for predicting fish acute toxicity (LC50) using the standardized ADORE dataset.
• Data Preparation: Use the ADORE "t-F2F" challenge subset. Apply a "chemical split" where all data points for a specific chemical are assigned exclusively to either the training or test set. This prevents data leakage from repeated measurements of the same chemical. The target variable is the log10-transformed LC50 value.
• Feature Engineering: Utilize one of six provided molecular representations (e.g., Morgan fingerprint with radius 2 and 2048 bits) as the chemical feature vector. Taxonomic features are excluded in this baseline protocol.
• Model Training: Train a Random Forest regressor (default scikit-learn parameters, e.g., n_estimators=100). Perform hyperparameter tuning via grid search with cross-validation on the training set only.
• Evaluation: Predict on the held-out test set. Calculate the Root Mean Square Error (RMSE) of the log10(LC50) predictions as the primary metric. An RMSE of 0.90 indicates predictions are, on average, within one order of magnitude of the true LC50 value.

Protocol 2: Generating SSDs with Artificial Neural Networks [78]

• Objective: To create Species Sensitivity Distributions for thousands of chemicals using machine learning-predicted toxicity values.
• Data Curation: Collate high-quality experimental LC50 data for 8 aquatic species from ECOTOX and peer-reviewed literature. Filter for consistent test duration (96-hr for fish, 48-hr for Daphnia), chemical purity (>85%), and pH (5-9). Use the geometric mean for repeated tests.
• Model Development: For each of the 8 species, train a separate Artificial Neural Network (ANN). Input features are molecular descriptors selected via a two-step process (initial screening + correlation filtering). Use a held-out test set to evaluate model performance (R²).
• SSD Construction: For a new chemical, use its molecular structure as input to the 8 trained ANNs to obtain a set of predicted LC50 values. Rank these 8 values from lowest to highest (most to least sensitive). Fit a log-normal distribution to the ranked values using a bootstrapping approach (e.g., 1,000 iterations) to account for prediction uncertainty. From the fitted distribution, calculate the HC5 (the concentration hazardous to 5% of species) and the HC50.

The Scientist's Toolkit: Essential Research Reagent Solutions

Building and applying these models requires a suite of data, software, and conceptual tools. The following table details key components of the modern ecotoxicological ML toolkit.

Table 3: Research Toolkit for Ecotoxicological ML and Hazard Mapping

Tool / Resource	Type	Primary Function in Workflow	Example Source / Implementation
Benchmark Datasets (e.g., ADORE)	Data	Provides a standardized, pre-curated foundation for training and fairly comparing models. Essential for reproducibility [1] [3].	ADORE dataset, hosted on public repositories accompanying [1].
ECOTOX Knowledgebase	Data	A primary source of experimental ecotoxicity results. Serves as the core raw data for curating new models and datasets [1] [78].	United States Environmental Protection Agency (EPA) database.
Molecular Descriptors & Fingerprints	Software/Chemoinformatics	Translates chemical structures into numerical vectors that ML models can process. Critical for QSAR and advanced ML [1] [41].	RDKit (for Morgan fingerprints), Mordred descriptor calculator.
Fixed Data Splits	Protocol	Pre-defined partitions of data into training, validation, and test sets. Prevents data leakage and ensures comparability between studies [2] [41].	Provided as part of the ADORE dataset challenges [1].
SSD Fitting Software	Software/Statistics	Fits statistical distributions (log-normal, Weibull) to toxicity data and calculates hazard concentrations (HCp) [78].	R packages (ssdtools), Python scripts with scipy.stats.
Geographic Information System (GIS)	Software	The platform for creating, managing, analyzing, and visualizing spatial data. Required for generating hazard susceptibility maps [79].	ArcGIS, QGIS (open source).
Model Interpretation Libraries	Software	Helps explain model predictions, identifying which features (e.g., chemical properties) drove a specific outcome. Increases trust and insight [80] [41].	SHAP (SHapley Additive exPlanations), LIME (Local Interpretable Model-agnostic Explanations).

Conclusion

The emergence of curated, publicly available benchmark datasets like ADORE and ApisTox represents a pivotal shift towards robust and reproducible machine learning in ecotoxicology. By providing a common foundation for model development, these resources directly address the ethical and financial imperatives to reduce animal testing. Success hinges on moving beyond simple model performance to embrace rigorous methodological practices—thoughtful data splitting, incorporation of biological context, and application of explainable AI. The future lies in expanding these benchmarks to cover a wider array of species, endpoints, and chronic effects, and in fostering a collaborative culture where model comparisons on shared datasets drive the field forward. This will ultimately empower more reliable chemical safety assessments, support Safe and Sustainable by Design (SSbD) initiatives, and provide critical tools for preserving biodiversity[citation:1][citation:2][citation:3].

References

• 1. A benchmark dataset for machine learning in ecotoxicology [https://www.nature.com/articles/s41597-023-02612-2]
• 2. ADORE | A benchmark dataset in ecotoxicology to foster ... [https://www.datascience.ch/articles/adore-benchmark-dataset-ecotoxicology-adoption-machine-learning]
• 3. Benchmark data for machine learning in ecotoxicology [https://communities.springernature.com/posts/benchmark-data-for-machine-learning-in-ecotoxicology]
• 4. A benchmark dataset for machine learning in ecotoxicology [https://pubmed.ncbi.nlm.nih.gov/37853023/]
• 5. A Comparative Evaluation of Machine Learning and Deep ... [https://pmc.ncbi.nlm.nih.gov/articles/PMC12391983/]
• 6. Combining Deep Learning and Explainable AI for Toxicity ... [https://chatpaper.com/zh-CN/chatpaper/paper/203510]
• 7. Evaluating machine learning models for predicting ... [https://arxiv.org/html/2503.24305v3]
• 8. Towards Rational Pesticide Design with Graph Machine ... [https://arxiv.org/html/2509.18703]
• 9. Comparative Chemical Space Analysis of Pesticides and ... [https://pubmed.ncbi.nlm.nih.gov/41160725/]
• 10. [2503.24305] Evaluating machine learning models for ... [https://arxiv.org/abs/2503.24305]
• 11. ApisTox: a new benchmark dataset for the classification of ... [https://www.nature.com/articles/s41597-024-04232-w]
• 12. ApisTox [https://archive.ics.uci.edu/dataset/995/apistox]
• 13. The ECOTOXicology Knowledgebase: A Curated Database of ... [https://pmc.ncbi.nlm.nih.gov/articles/PMC9408435/]
• 14. cfpub.epa.gov/ ecotox [https://cfpub.epa.gov/ecotox/]
• 15. ApisTox: a new benchmark dataset for the classification of small... [https://www.nature.com/articles/s41597-024-04232-w?error=cookies_not_supported&code=bcc34515-7136-42fd-8859-1a131ae63fc7]
• 16. - Pesticides Properties DataBase PPDB [https://sitem.herts.ac.uk/aeru/ppdb/]
• 17. Standartox: Standardizing Toxicity Data [https://www.mdpi.com/2306-5729/5/2/46]
• 18. Curated mode-of-action data and effect concentrations for ... [https://www.nature.com/articles/s41597-023-02904-7]
• 19. A comparative study of biostatistical pipelines for benchmark ... [https://pubmed.ncbi.nlm.nih.gov/40672902/]
• 20. Recent advances in molecular representation methods and ... [https://www.nature.com/articles/s44386-025-00017-2]
• 21. for machine learning in Benchmark data ecotoxicology [https://communities.springernature.com/posts/benchmark-data-for-machine-learning-in-ecotoxicology?badge_id=scientific-data]
• 22. Evaluating molecular representations in machine learning ... [https://pmc.ncbi.nlm.nih.gov/articles/PMC9521826/]
• 23. Could graph neural networks learn better molecular ... representation [https://jcheminf.biomedcentral.com/articles/10.1186/s13321-020-00479-8]
• 24. A benchmark for machine learning in dataset - PMC ecotoxicology [https://pmc.ncbi.nlm.nih.gov/articles/PMC10584858/]
• 25. Effectiveness of molecular fingerprints for exploring the ... [https://jcheminf.biomedcentral.com/articles/10.1186/s13321-024-00830-3]
• 26. Predicting the toxicity of chemicals with AI [https://www.eawag.ch/en/info/portal/news/news-detail/predicting-the-toxicity-of-chemicals-with-ai/]
• 27. Estimate ecotoxicity characterization factors for chemicals ... [https://www.sciencedirect.com/science/article/pii/S0160412019314412]
• 28. Interpretable Machine Learning Framework for Predicting ... [https://www.sciencedirect.com/science/article/abs/pii/S0013935125026428]
• 29. A benchmark dataset for machine learning in ecotoxicology [https://www.semanticscholar.org/paper/A-benchmark-dataset-for-machine-learning-in-Sch%C3%BCr-Gasser/684621e59b7b864fc56567c196630b42b81a8b12]
• 30. Improving the Ecotoxicological Hazard Assessment of ... [https://pmc.ncbi.nlm.nih.gov/articles/PMC12355940/]
• 31. Graph Neural Network-Based Toxicity Prediction by ... [https://pmc.ncbi.nlm.nih.gov/articles/PMC12656225/]
• 32. Research on GNNs with stable learning | Scientific Reports [https://www.nature.com/articles/s41598-025-12840-8]
• 33. Benchmarking Toxic Molecule Classification using Graph ... [https://arxiv.org/abs/2311.13490]
• 34. Downloadable Computational Toxicology Data [https://www.epa.gov/comptox-tools/downloadable-computational-toxicology-data]
• 35. Ecotoxicology Database (ECOTOX) - Dataset - Catalog [https://catalog.data.gov/dataset/ecotoxicology-database-ecotox13]
• 36. Toxicity Estimation Software Tool (TEST) [https://www.epa.gov/comptox-tools/toxicity-estimation-software-tool-test]
• 37. AI-based toxicity prediction models using ToxCast data [https://www.sciencedirect.com/science/article/pii/S0300483X25001891]
• 38. How to Generate a Train - Test - Split Based on... - GeeksforGeeks [https://www.geeksforgeeks.org/machine-learning/how-to-generate-a-train-test-split-based-on-a-group-id/]
• 39. _ temporal _ train _ test — sktime documentation split [https://www.sktime.net/en/latest/api_reference/auto_generated/sktime.split.temporal_train_test_split.html]
• 40. LakeBeD-US: a benchmark dataset for lake water quality time ... [https://essd.copernicus.org/articles/17/3141/2025/]
• 41. Machine learning-based prediction of fish acute mortality [https://pubs.rsc.org/en/content/articlehtml/2024/va/d4va00072b]
• 42. The problematic case of data leakage: A case for leave- ... [https://www.sciencedirect.com/science/article/pii/S0016706125000618]
• 43. Some combinatorics of data leakage induced by clusters [https://link.springer.com/article/10.1007/s00477-024-02715-1]
• 44. A comparison of machine learning approaches for ... [https://www.sciencedirect.com/science/article/abs/pii/S2468111324000033]
• 45. Machine Learning and Artificial Intelligence in ... [https://pmc.ncbi.nlm.nih.gov/articles/PMC9609874/]
• 46. Coverage bias in small molecule machine learning [https://www.nature.com/articles/s41467-024-55462-w]
• 47. Addressing Bias in Machine Learning for Equitable ... [https://www.helmholtz-munich.de/en/ehc/ehc-news/news-detail/addressing-bias-in-machine-learning-for-equitable-healthcare]
• 48. Ethical and Bias Considerations in Artificial Intelligence ... [https://www.sciencedirect.com/science/article/pii/S0893395224002667]
• 49. Predicting chemical ecotoxicity by learning latent space ... [https://www.sciencedirect.com/science/article/pii/S0160412022001507]
• 50. Species sensitivity distribution modeling for ecotoxicity ... [https://www.sciencedirect.com/science/article/pii/S004896972502515X]
• 51. Explainable Artificial Intelligence in the Field of Drug Research [https://pmc.ncbi.nlm.nih.gov/articles/PMC12129466/]
• 52. Opening the Black Box of Machine Learning Models: SHAP ... [https://medium.com/cmotions/opening-the-black-box-of-machine-learning-models-shap-vs-lime-for-model-explanation-d7bf545ce15f]
• 53. Application of explainable artificial intelligence in medical ... [https://www.sciencedirect.com/science/article/pii/S2352914823001302]
• 54. A Perspective on Explainable Artificial Intelligence Methods [https://arxiv.org/html/2305.02012v3]
• 55. LIME vs SHAP: A Comparative Analysis of Interpretability ... [https://www.markovml.com/blog/lime-vs-shap]
• 56. An Interpretable Machine Learning Framework for Analyzing ... [https://pmc.ncbi.nlm.nih.gov/articles/PMC12349579/]
• 57. Unified artificial intelligence framework for modeling ... [https://www.nature.com/articles/s41598-025-20083-w]
• 58. Comprehensive benchmarking of computational tools for ... [https://jcheminf.biomedcentral.com/articles/10.1186/s13321-024-00931-z]
• 59. Dataset on null LC–MS/MS findings for 17 pharmaceuticals ... [https://www.sciencedirect.com/science/article/pii/S2352340925009850]
• 60. Creation of an Open-Access High-Resolution Tandem ... [https://pmc.ncbi.nlm.nih.gov/articles/PMC12590464/]
• 61. Department of the Interior - 25 - Dataset - Catalog [https://catalog.data.gov/dataset/?tags=liquid-chromatography&organization=doi-gov]
• 62. lc-ms - Dataset - Catalog [https://catalog.data.gov/dataset/?tags=lc-ms]
• 63. Inter-laboratory mass spectrometry dataset based on ... [https://www.nature.com/articles/s41597-021-01002-w]
• 64. Annotating Nontargeted LC-HRMS/MS Data with Two ... [https://pmc.ncbi.nlm.nih.gov/articles/PMC6359582/]
• 65. Beyond accuracy: a framework for evaluating algorithmic ... [https://www.nature.com/articles/s41598-025-06019-4]
• 66. Machine learning in ecotoxicology: Pollutant exposure ... [https://pubmed.ncbi.nlm.nih.gov/41270592/]
• 67. Smart Guide to Validating Your Custom AI Model [https://www.ignatiuz.com/blogs/beyond-accuracy-the-smart-guide-to-validating-your-custom-ai-model/]
• 68. Beyond Accuracy: Exploring Exotic Metrics for Holistic ... [https://towardsdatascience.com/beyond-accuracy-exploring-exotic-metrics-for-holistic-evaluation-of-machine-learning-models-8a093875dcc9/]
• 69. Benchmarking Machine Learning Models for Efficient and ... [https://www.mdpi.com/1999-5903/17/10/467]
• 70. The role of machine learning in predictive toxicology [https://www.sciencedirect.com/science/article/pii/S0024320525004564]
• 71. Assessing chemical exposure risk in breastfeeding infants [https://www.sciencedirect.com/science/article/pii/S0147651325000430]
• 72. A deep-learning approach to predict reproductive toxicity of ... [https://www.frontiersin.org/journals/toxicology/articles/10.3389/ftox.2025.1640612/full]
• 73. Evaluation of machine learning models for the prediction ... [https://pmc.ncbi.nlm.nih.gov/articles/PMC11850780/]
• 74. Graph neural networks-enhanced relation prediction for ... [https://www.sciencedirect.com/science/article/pii/S0304389424010355]
• 75. Computational toxicology in drug discovery: applications of ... [https://pmc.ncbi.nlm.nih.gov/articles/PMC12499773/]
• 76. Open Graph Benchmark | A collection of benchmark datasets ... [https://snap-stanford.github.io/ogb-web/]
• 77. Deep learning advances high-throughput toxicity screening ... [https://www.sciencedirect.com/science/article/abs/pii/S1001074225006205]
• 78. Accelerating the pace of ecotoxicological assessment ... [https://pmc.ncbi.nlm.nih.gov/articles/PMC8800994/]
• 79. Hazard Susceptibility Mapping with Machine and Deep ... [https://www.mdpi.com/2072-4292/16/18/3374]
• 80. Bayesian network modelling for predicting the ... [https://www.sciencedirect.com/science/article/pii/S2452074825000138]