What Shap Revealed About Molecular Properties: Four Models, Four Different Answers An XGBoost model trained on RDKit descriptors predicted aqueous solubility (logS) with RMSE 0.52 log units and R² 0.89 on held-out compounds, more than twice as accurate as the classic Delaney (2004) formula's RMSE of 1.15, according to a cheminformatics study published on GitHub (cheminformatics-ml). SHAP attributed 63% of the solubility model's feature importance to Crippen LogP, while the top driver flipped to BertzCT for boiling point (XGBoost RMSE 28.5 °C, R² 0.84 versus a molecular-weight-only baseline at RMSE 47.7 °C). For Crippen LogP itself, ElasticNet reached RMSE 0.75 and R² 0.84 but lost to Crippen's atom-contribution method at RMSE 0.65 and R² 0.88, and for Ames mutagenicity logistic regression ranked first on the held-out test set with AUC 0.987 and recall 0.909. Disclosure: I drafted this article with AI assistance from my own notebooks and results. I reviewed and edited it, and checked every number against my code. Most chemistry notebooks on Kaggle treat molecules as just another tabular dataset. Compute some descriptors, fit XGBoost, report R², done. I wanted to know something else: when a model predicts a molecular property well, is it learning chemistry, or a shortcut? So I built four models from the same family of RDKit descriptors, for four different properties, and asked SHAP what drove each one. The answer was different every time, and each time it matched the physical chemistry. Two of the four also ended with a result I didn’t expect: the fancy model lost. All notebooks and data are on GitHub cheminformatics-ml https://github.com/shehanmakani/cheminformatics-ml . Every notebook follows the same recipe, so the differences come from the chemistry, not the pipeline. The baselines matter most. A model that beats LightGBM but loses to a 20-year-old formula hasn’t learned anything useful. XGBoost predicted aqueous solubility logS with RMSE 0.52 log units and R² 0.89 on held-out compounds. The classic Delaney 2004 formula 1 , applied across the dataset, scored RMSE 1.15, more than twice the error. SHAP put Crippen LogP far ahead, at 63% of XGBoost’s feature importance. That is the thermodynamics of water exclusion: hydrophobic molecules don’t dissolve. Polycyclic aromatics such as pyrene logS −5.18 set the insolubility floor, with high LogP and no heteroatoms to bond with water. Tree models beat linear ones because the effect isn’t additive: LogP and ring count interact. XGBoost reached RMSE 28.5 °C and R² 0.84 across compounds boiling from −162 to +365 °C. A molecular-weight-only baseline managed RMSE 47.7 °C. The top SHAP driver flipped. It was BertzCT, a measure of structural complexity, not LogP. More complex molecules have more surface for van der Waals contact, so they hold together longer as liquids. H-bond donors push boiling points well above non-polar molecules of the same weight. Here I removed RDKit’s own Crippen LogP from the features and asked whether ML could rebuild it from structure. The best model, ElasticNet, reached RMSE 0.75 and R² 0.84. Crippen’s atom-contribution method 2 still won, at RMSE 0.65 and R² 0.88. That isn’t a failure; it’s the point. Crippen encodes a measured contribution for each atom type, which 119 molecules can’t relearn. SHAP’s top driver was molar refractivity MolMR , a proxy for electron polarizability, which is the physical reason a molecule prefers octanol to water. ML was competitive in the LogP 1–3 range, where most drug molecules sit. For Ames mutagenicity, a binary yes/no, logistic regression ranked first on the held-out test set: AUC 0.987 and recall 0.909. In 5-fold cross-validation, though, SVM and XGBoost scored higher AUC 0.98 vs. 0.90 . With 122 compounds, that ranking is fragile. The structural alerts were not fragile. Every compound with a nitro or nitroso group in the set was mutagenic 4, 5 , because these groups become electrophiles that attack DNA. Aromatic amines were mutagenic 30% of the time, and alkyl halides 43%. I optimized for recall, because in toxicology a missed mutagen costs far more than a false alarm. The whole loop fits on one screen. This is the boiling-point version, trimmed from the notebook: python import numpy as np, pandas as pd, shap, xgboost as xgbfrom rdkit import Chemfrom rdkit.Chem import Descriptors, Crippen, rdMolDescriptorsfrom sklearn.model selection import train test split python def featurize smiles : mol = Chem.MolFromSmiles smiles return { "MW": Descriptors.MolWt mol , "LogP": Crippen.MolLogP mol , "HBD": rdMolDescriptors.CalcNumHBD mol , "TPSA": Descriptors.TPSA mol , "MolMR": Crippen.MolMR mol , "BertzCT": Descriptors.BertzCT mol , ...the full notebooks use more descriptors } X = pd.DataFrame featurize s for s in df "smiles" y = df "bp celsius" 70/15/15 train/validation/test splitX train, X tmp, y train, y tmp = train test split X, y, test size=0.30, random state=42 X val, X test, y val, y test = train test split X tmp, y tmp, test size=0.50, random state=42 model = xgb.XGBRegressor n estimators=300, learning rate=0.05, max depth=5, subsample=0.8, colsample bytree=0.8, random state=42 model.fit X train, y train Which descriptor drives the prediction?shap vals = shap.TreeExplainer model .shap values X test importance = pd.Series np.abs shap vals .mean axis=0 , index=X.columns print importance.sort values ascending=False .head Swap the target column and the same code answers a different chemistry question. That is what makes the comparison fair. The top SHAP driver for each property: I run a specialty chemical formulation company, and these are the questions I ask before choosing a surfactant or a solvent. ML earned its place in that work only once I could check that it was reasoning like a chemist. SHAP is how I check. The four notebooks, data and figures are on GitHub https://github.com/shehanmakani/cheminformatics-ml and Kaggle https://www.kaggle.com/shehanmakani . 1 J. S. Delaney, “ESOL: Estimating aqueous solubility directly from molecular structure,” J. Chem. Inf. Comput. Sci. , vol. 44, no. 3, pp. 1000–1005, 2004. 2 S. A. Wildman and G. M. Crippen, “Prediction of physicochemical parameters by atomic contributions,” J. Chem. Inf. Comput. Sci. , vol. 39, no. 5, pp. 868–873, 1999. 3 S. M. Lundberg and S.-I. Lee, “A unified approach to interpreting model predictions,” Advances in Neural Information Processing Systems 30 , 2017. 4 K. Hansen et al., “Benchmark data set for in silico prediction of Ames mutagenicity,” J. Chem. Inf. Model. , vol. 49, no. 9, pp. 2077–2081, 2009. 5 J. Kazius, R. McGuire and R. Bursi, “Derivation and validation of toxicophores for mutagenicity prediction,” J. Med. Chem. , vol. 48, no. 1, pp. 312–320, 2005. What Shap Revealed About Molecular Properties: Four Models, Four Different Answers https://pub.towardsai.net/what-shap-revealed-about-molecular-properties-four-models-four-different-answers-32b185e30300 was originally published in Towards AI https://pub.towardsai.net on Medium, where people are continuing the conversation by highlighting and responding to this story.