re: #670 Obdicut

No, there really is such a thing as differences in how well people take tests. Some people in those particular environments freeze up— this is easily demonstrable by the same person getting different grades depending on how a test is administered.

This is just the No True Scotsman of ‘good test item’. How many tests of that sort, out of the number of tests given, actually exist?

And what do tests test? Do MCATs test an ability to gain medical knowledge, or an ability to pass through medical school— do they predict success on other tests, or in the real world?

As one of my wife professor’s said to her— “Evidence-based medicine does not, unfortunately, mean evidence-based learning.”

MCATS do correlate with success in medical school. Students with very low MCAT scores, who are admitted anyhow, are much more likely to have to repeat a year of medical school, and much more likely to fail their state licensing exams. When they go into practice, if they ever qualify at all, they’re more likely to have bad results, and they’re more likely to have their licenses revoked.

Tests do predict real world results.

There are some bad tests, but one reason for that is that items that require reasoning as well as factual knowledge tend to be purged from the instrument, because they drag scores down or something. In other words, the authorities who commission the test don’t want it to be a good test. They want it to give results that make them look good.