from Applying the Science of Learning to the University and Beyond; Teaching for Long-Term Retention and Transfer
Halpern, Diane F., Hakel, Milton D.. Change. New Rochelle:Jul/Aug 2003. Vol. 35, Iss. 4, p. 36
Experience alone is a poor teacher. There are countless examples that illustrate that what people learn from experience can be systematically wrong. For example, physicians often believe that an intervention has worked when a patient improves after a particular treatment regime. But most patients will improve no matter what intervention occurs. If the patient does not improve, then physicians may reason that he or she was "too sick" to have benefited from effective treatment. There are countless examples of this sort of erroneous thinking in both professional practice and everyday life, where current beliefs about the world and how it works are maintained and strengthened, despite the fact that they are wrong.
People, therefore, frequently end up with great confidence in their erroneous beliefs. Confidence is not a reliable indicator of depth or quality of learning. In fact, research in metacognition has shown that most people are poor judges of how well they comprehend a complex topic.
The fact that most people don't know much about the quality of their comprehension is important, because there is a popular belief that all learning and assessment should be "authentic"--that is, nearly identical in content and context to the situation in which the information to be learned will be used. But what is missing from most authentic situations--and from most real-life situations as well--is systematic and corrective feedback about the consequences of various actions.
To return to the example of physicians, many medical schools have now adopted simulated patients as a teaching and testing tool--actors trained to present a variety of symptoms for novice practitioners to diagnose--because unplanned clinical encounters with real patients can't provide the necessary variety and feedback.