Environmental Toxicity Prediction Challenge

Organizers

This challenge is organized by ICANN'09: International Conference on Artificial Neural Networks, European Neural Network Society (ENNS) and CADASTER project.

Goals of this study

Develop in silico models to predict environmental toxicity of molecules against T. pyriformis using data from [1].

Estimate the prediction intervals for new compounds.

More detailed introduction can be found here, data can be downloaded at this page and results submitted here.

Important key dates

June 1 The submission of results is open.

August 31 The submission of results is closed.

September 10 New submission deadline

September 12 New submission deadline

September 14-17 2009 The winner will be announced at the ICANN'09 conference.

September 15 Final results are available here and experimental values can be downloaded here.

The winner will be identified according to the criteria defined below and (s)he will receive a prize. It is expected that the winner as well as other participants will submit articles describing their methodological approaches for publication in a peer-reviewed journal (under discussion). Information on how you can participate can be found here.
Grand prize for the competition-winners is 1.000 € !
See summary of results.

Criteria for success

Firstly, methods with Root Mean Squared Error (RMSE) non-significantly different to the method with lowest RMSE (according to the bootstrap test, p<0.05) will be identified as the first-pass winners.

Root Mean Squared Error
RMSE = 1/N ∑(Ypred-Yexp)2, where sum id over the test set and Ypred and Yexp are predicted and experimental toxicities, will be used to estimate performance of methods.

Secondly, the method providing the best likelihood criteria between estimated and observed confidences for the blind test set as described in [2] will be identified amid the first-pass winners.

Likelihood criterion
Corresponds to a probability that observed toxicity predictions errors are described by a mixture of Gaussian distributions.

Igor V. Tetko, Terry W. Schultz and Wlodzislaw Duch


[1] Zhu, H.; Tropsha, A.; Fourches, D.; Varnek, A.; Papa, E.; Gramatica, P.; Oberg, T.; Dao, P.; Cherkasov, A.; Tetko, I. V. Combinatorial QSAR Modeling of Chemical Toxicants Tested against Tetrahymena pyriformis J. Chem. Inf. Model. 2008, 48 (4), 766-784.

[2] Tetko, I. V.; Sushko, I.; Pandey, A. K.; Zhu, H.; Tropsha, A.; Papa, E.; Oberg, T.; Todeschini, R.; Fourches, D.; Varnek, A., Critical assessment of QSAR models of environmental toxicity against Tetrahymena pyriformis: focusing on applicability domain and overfitting by variable selection J. Chem. Inf. Model. 2008, 48 (9), 1733-46.