This study reports on a recent evaluation of the similarity model used by Recommendation Explorer, an automatic recommender system. In particular, we consider the role of several system-internal factors in determining the quality of recommendation. More generally, we discuss factors in the recommendation task itself that complicate the construction and evaluation of recommender systems, and reflect on the implications of our findings for research in this area.