Any comments on what CR says and what aspects are considered unreliable ?
I noted that in my comment: the gas engine, body integrity, and audio systems got the worst scores. (New cars are so reliable on average, though, that it's sometimes hard to pin down problem areas for new models; there are so few problems in specific areas, even for cars that are bad overall, that statistical significance is lacking.) For details, see their web site. (You'll need to subscribe to get every detail.)
Qinsp said:
I have a hunch CR doesn't vet their sources.
In survey research (as their reliability ratings are), "vetting" of sources is a bad thing -- good survey research takes a random sample and does not exclude ("vet") data, unless there's a strong reason to think the survey's being manipulated. Where CR can be criticized on this is that their sample is arguably non-random, since it consists of CR subscribers. I have yet to see a good argument for why that particular variety of non-randomness might invalidate their results, though. Even if it did, the fact that their results are derived from their readers' experiences may make it those results
more valuable for their readers, even if not for the general population.
Qinsp said:
They are not exactly car savvy either. Nor consistent. In a short interval, they say the Model S Tesla is so damn good, they need to make a new scoring scale. No car in history has ever been better. Then they put them on the THIS SUCKS list, then they just put them back on the OMFG list.
Like many criticisms of CR, this one is based on lack of understanding. A couple years ago, the Tesla Model S produced the highest
road test score of any car. (The road test includes things like acceleration, fuel economy, cargo capacity, seating comfort, etc. -- but not reliability, customer satisfaction, and some other measures.) At that time, its reliability was average. In order to be recommended by CR, a car had to score highly on the road test and get at least average reliability. The Model S met those criteria and so was recommended -- and got a lot of positive press because of its record-setting road-test score. The next year, though, the Model S's reliability dropped just enough for it to be disqualified from recommendation. Its road test results were still excellent, though. Now the reliability has increased again to average, so it's back on the recommended list. There's nothing weird, conspiratorial, or incompetent about this; CR simply has a hard cutoff of "average" reliability to recommend a car.
Coincidentally, CR also changed the way it presented its results about a year ago. If you looked at lists of cars before then, they were ranked according to their road test scores, which meant you could see something at the top of the list that was not recommended because it was unreliable. They've now created a new overall score that includes the road test score, reliability, and other measures, and they present cars ordered by this new overall score. The intent was to reduce confusion in readers, but of course if you didn't understand the nature of this change and compared auto rankings from two different times, you'd see rather significant changes to the order and score, which might be confusing.