A couple of weeks ago, we came across a preprint [https://arxiv.org/abs/1805.10970] tackling the problem of forward prediction using a stepwise approach. The main premise is that reactions can be described as a linear sequence of electron pushing events, which a neural model can be trained to select/predict. The authors aim to develop a method for predicting reaction mechanisms without requiring annotated mechanistic data. These mechanistic explanations are of particular interest to us because they can be provide insights and may be easier to interact with than traditional neural models designed in “black box” manner. Not surprisingly, this topic has been studied in the past research. However, the primary drawback of the existing mechanistic predictors (e.g., work by the Baldi group [https://www.ncbi.nlm.nih.gov/pubmed/22978639]) is the need to manually encode rigid mechanistic rules to generate training and testing data. Being able to make accurate mechanistic predictions with only reactant and final product information would be a significant contribution.
The paper aims to address this ambitious goal proposing an interesting model with some empirical support. However, as we argue below what is predicted is not a reaction mechanism and has limited utility beyond reproducing the authors’ hard-coded expert heuristics:
- The premise of reactions as sequential motions of electrons is a little too restrictive, at least how the authors interpret it. In the USPTO dataset that has been used previously by us [https://arxiv.org/abs/1709.04555] and Schwaller et al. [https://arxiv.org/abs/1711.04810], 27% of the reactions do not fit that premise. It’s not obvious (without digging through the data set in greater detail) what exactly is lost in the 27% of these USPTO reactions. We pulled the first five test reactions that arenot in their subset and found a reductive amination, a total deprotection of a tertiary amine to a primary amine, a thioether oxidation to a sulfoxide, thiourea addition to an alkyl iodide, and an alkene ozonolysis to an aldehyde. So maybe not reactions you use every day, but these are certainly reactions one would want these systems to be able to predict. Still – if one is content to work under this premise, then the problem becomes much more structured. One can structure the model’s ability to make decisions to only allow it to make predictions that obey these assumptions.
I also must point out that this makes the comparison to our WLDN and IBM’s Seq2seq model highly flawed: neither model was designed specifically for this data subset, which narrows the prediction task significantly.
- This is worth emphasizing:what is being learned is not a mechanism. The model is trained to reproduce sequences of pseudo-mechanistic steps that strictly follow pseudo-mechanisms generated by expert heuristics in a predetermined fashion. If you look at Figure 3 in the preprint, it will be immediately obvious what I mean. The ability to draw reactions with arrows indicating bond changes is not the same as being interpretable, nor does it mean what is being predicted has any physical significance. If the model is accurate – as defined by the authors – then it will simply reproduce the sequence of steps defined by their heuristic pseudo-mechanism generator. The same “interpretation” would be possible if one makes a black box prediction of the major product and then applies those same heuristics.
In summary, the paper explores an important and challenging problem of learning mechanistic explanations. The model is very nicely designed and executed. However, given the points above, we think that the problem is still open.