I basically think of the ratings as follows:
**** = A recording which redefines the way an entire spectrum of
music can be heard & appreciated.
*** = A landmark recording in its field.
** = A solid contribution to the overall repertory.
* = A recording with definite merit, but perhaps lacking in
some aspect.
Note that I was previously giving 5- & 6-star ratings, but decided that they were not used often enough and that I would simplify to a 4-star system.
The comments found by clicking on the rating for a specific recording can vary a lot in depth & detail. Sometimes I feel like the rating itself says most of what needs to be said, and sometimes I feel like there is more to say. Please keep in mind that these comments are not really a full review, but rather an explanation of a rating (once given that rating) within the context of the full set of lists. The "closed" nature of these lists (meaning they are intended to be complete for their repertories) is, I think, valuable boundary information which can be used to judge the usefulness of the rating itself.
I will also re-evaluate recordings continuously, sometimes changing the rating at whim, or after an infatuation or misunderstanding wears off. Also, new releases can sometimes have major ripple effects on my opinion of other recordings. As mentioned above, a new 4-star recording would quite probably change the rating of every recording near it in the list. Generally, 2 stars is the most "stable" rating.
Just to be clear, I consider all of these things to be advantages of the medium. If I say something is a must-hear for a specific repertory, I like to know I can easily retract that if an even better recording appears the next month. While such dynamism is very rare in core classical music (my favorite Brahms recordings have all been the same for years, for instance), there are definitely sub-repertories of early music where I have experienced this. Of course, this dynamism also means that writing a "real" review seems like a waste of time, since the overall context in which a specific recording exists is going to vary in the short term, and these contexts are crucial to writing a review of any depth.
Well, I'm sure this obfuscation has all been very fascinating.... I hope you find my picks informative.
T. M. McComb