Current Computer Aided-Drug Design

Subhash C. Basak
Departments of Chemistry, Biochemistry & Molecular Biology University of Minnesota Duluth
Duluth, MN 55811
USA

Back

Assessing QSAR Limitations - A Regulatory Perspective

Author(s): Weida Tong, Huixiao Hong, Qian Xie, Leming Shi, Hong Fang and Roger Perkins

Affiliation: Center for Toxicoinformatics, NCTR, 3900 NCTR Road, HFT 020, Jefferson, AR 72079, USA.

Abstract:

Wider acceptance of QSARs would result in a constellation of benefits and savings to both private and public sectors. For this to occur, particularly in regulatory applications, a models limitations need to be identified. We define a models limitations as encompassing assessment of overall prediction accuracy, applicability domain and chance correlation. A general guideline is presented in this review for assessing a models limitations with emphasis on and examples of application with consensus modeling methods. More specifically, we discuss the commonalities and differences between external validation and cross-validation for assessing a models limitations. We illustrate two common ways of assessing overall prediction accuracy, depending on whether or not the intended application domain is predefined. Since even a high quality model will have different confidence in accuracy for predicting different chemicals, we further demonstrate using the novel Decision Forest consensus modeling method a means to determine prediction confidence (i.e., certainty for an individual chemicals prediction) and domain extrapolation (i.e., the prediction accuracy for a chemical that is outside the chemistry space defined by the training chemicals). We show that prediction confidence and domain extrapolation are related measures that together determine the applicability domain of a model, and that prediction confidence is the more important measure. Lastly, the importance of assessing chance correlation is emphasized, and illustrated with several examples of models having a high degree of chance correlations despite cross-validation indicating high prediction accuracy. Generally, a dataset with a skewed distribution, small data size and/or low signal/noise ratio tends to produce a model with high chance correlation. We conclude that it is imperative to assess all three aspects (i.e., overall accuracy, applicability domain and chance correlation) of a model for the regulatory acceptance of QSARs.

Keywords: sar/qsar, model limitation, model uncertainty, applicability domain, model validation, chance correlation, decision forest, consensus modeling

Order Reprints Order Eprints Rights & PermissionsPrintExport

Article Details

VOLUME: 1
ISSUE: 2
Page: [195 - 205]
Pages: 11
DOI: 10.2174/1573409053585663
Price: $58