I've mentioned this before in context of drug screening in high school students, but Bayes' theorem holds when analyzing tests that assess teacher performance as well.

Suppose your district buys into Edumegacorp's teacher rating system, the Assessment of Holistic Learning Environments (the AssHoLE test). The test is 95% sensitive--it will detect bad teacher 95% of the time. It is also 95% specific--"good" teachers will pass 95% of the time. Sounds like a great tool, no?

Let's further suppose it's used in a district that uses the best teachers in the universe--99% of them are competent. 1000 of them are tested. Hey, a good teacher has nothing to fear, eh?

Do the math. The ten incompetents will be correctly identified--it's a sensitive test. 5% of the remaining 990, however, will also be identified as incompetent. About 50 teachers will fail their assessment. Only 10 of the 60 who fail the AssHoLE test are truly incompetent.

It gets better though. The tests being used to judge teacher performance were not designed to test teacher performance, they were designed to assess student performance in very limited tasks.

This is like biopsying your brain to see how your liver is doing.

Just saying....

The brain biopsy is via Wikimedia commons, with a share alike CC license--
I hope they asked the patient who's brain has been splayed open to the internet universe.