In a recent preprint (arXiv:1305.4904) entitled “Classical signature of quantum annealing” Smolin and Smith point out that a bimodal distribution presented in (arXiv:1304.4595) for the success probability in the D-Wave device does not in itself provide sufficient evidence for quantum annealing, by presenting a classical model that also exhibits bimodality. Here we analyze their model and in addition present a similar model derived from the semi-classical limit of quantum spin dynamics, which also exhibits a bimodal distribution. We find that in both cases the correlations between the success probabilities of these classical models and the D-Wave device are weak compared to the correlations between a simulated quantum annealer and the D-Wave device. Indeed, the evidence for quantum annealing presented in arXiv:1304.4595 is not limited to the bimodality, but relies in addition on the success probability correlations between the D-Wave device and the simulated quantum annealer. The Smolin-Smith model and our semi-classical spin model both fail this correlation test.