Dunning Shmunning, shaddup!

[Sean Hayden]

Dunning-Kruger Isn't Real
The least knowledgeable people are not the most overconfident.

…

The controversy stirred up around Dunning-Kruger by the recent blog post was based on McKnight finding that he could create something that looked a lot like the Dunning-Kruger effect from a model where the worst performing people weren’t any more or less wrong about their skill level. People were just wrong randomly, and the pattern looked similar to the one originally published by Dunning and Kruger. A refinement of this was then posted by Benjamin Vincent of the University of Dundee in Scotland. In Vincent’s version, people were biased, but there was no difference between those who know the most and those who know the least. Everyone was just a bit overconfident in their abilities, no matter what level they were at. This matched the observed data beautifully.* (I reproduced the figure below, adjusting the data slightly to suggest that it's harder to measure people's real skill than their personal judgments. Code is publicly available here.)

…

https://www.psychologytoday.com/us/blog ... -isnt-real

[pErvinalia]

I prefer it to be real.

[Sean Hayden]

It’s definitely real by now, we’ve manifested it.

[Svartalf]

I don't know what you folk are yammering about.

[rasetsu]

I don't understand his explanation.

[rasetsu]

Summary:

• A noise only model, with no bias, is capable of generating systematic over- and under-estimation of one’s abilities even though there is no systematic bias present in the model.
• This alone may make one believe that the Dunning-Kruger effect is artifactual, a result of measurement error alone. However this is incorrect.
• The empirical observations are better accounted for by a noise + bias model as I presented above. But serious readers should check out Burson et al (2006) for a much more in-depth treatment of this topic.
• Take home message — The Dunning-Krugger effect probably is real, but you probably want to base your interpretations of psychological explanations upon the noise + bias model.

https://drbenvincent.medium.com/the-dun ... 778ffd9d1b

[Sean Hayden]

Thank you, Dr Vincent’s take away is only briefly summarized at the end of the link I posted.

Incidentally, Vincent argues that this shows that there is a Dunning-Kruger effect, because people are biased, but that’s it’s just a different effect from the one in the literature. Knowing more doesn’t make people less biased: Everyone’s equally biased. I’m saying this means we have a different effect, but the argument is just about whether we shift the meaning of Dunning-Kruger or use a different label.

—//—

The bit above the summary from your link:

After building this model, I became aware that I’d touched on an argument made by Burson et al (2006). They proposed a noise+bias model akin to the one I have presented. They go further in a number of ways however. Firstly, they also explore the notion that the estimation noise of one’s subjective ability may increase as one’s true ability decreases. This amounts to suggesting that people with low ability have higher uncertainty about their own true ability. Secondly, Burson et al (2006) manipulate task difficulty in an attempt to manipulate people’s estimation biases. They do in fact empirically find that harder tasks shift the curve downwards, such that the majority of people now underestimate their abilities. This is powerful evidence in favour of their noise + bias explanation of the Dunning-Kruger effect.

[rasetsu]

I'm still not understanding how noise with no bias generates that plot. Maybe after I read Nuhfer.

[aufbahrung]

AI generated word salad. Dunning-kruger is still bogus though. Read a article somewhere, before AI made articles machine generative dubious. Seemed to make sense. Need some over confidence or you'd never try anything difficult and novel. Dunning-Kruger is a parasite on the basics of bootstrapping failures, that continue till you succeed.

[Sean Hayden]

That’s an interesting take. This article just tries to point out that the noise only model is close to what was observed by Dunning-Kruger, which raised some suspicions. Dr Vincent’s noise + bias model supposedly matches DK’s very closely, suggesting that the bias is similar across skill levels, which of course is nearly, if not exactly opposite to the usage of DK’s findings you regularly encounter.

[Brian Peacock]

I'm still not understanding how noise with no bias generates that plot. Maybe after I read Nuhfer.

Is he saying that the prevalence of Dunning-Kruger in any subject sample is indistinguishable from any random sampling - 'noise' having no discernable underlying structure or pattern when re-sampled?

[rasetsu]

I'm still not understanding how noise with no bias generates that plot. Maybe after I read Nuhfer.

Is he saying that the prevalence of Dunning-Kruger in any subject sample is indistinguishable from any random sampling - 'noise' having no discernable underlying structure or pattern when re-sampled?

I don't see how that explains the graph. The blue line is the estimated performance.

[Sean Hayden]

I'm still not understanding how noise with no bias generates that plot. Maybe after I read Nuhfer.

Is he saying that the prevalence of Dunning-Kruger in any subject sample is indistinguishable from any random sampling - 'noise' having no discernable underlying structure or pattern when re-sampled?

That does seem to be close to what he’s saying. But according to the link provided by rasetsu that’s not accurate. To more closely match the results some bias is necessary, it’s just not restricted to the low skilled.

—//—

Others have used real world data to test the effect as well.

To prove the last point – that the least skilled can adequately judge their own skill – required a different approach.

My colleague Ed Nuhfer and his team gave students a 25-question scientific literacy test. After answering each question, the students would rate their own performance on each question as either “nailed it,” “not sure” or “no idea.”

Working with Nuhfer, we found that unskilled students are pretty good at estimating their own competence. In this study of unskilled students who scored in the bottom quarter, only 16.5% significantly overestimated their abilities. And, it turns out, 3.9% significantly underestimated their score. That means nearly 80% of unskilled students were fairly good at estimating their real ability – a far cry from the idea put forth by Dunning and Kruger that the unskilled consistently overestimate their skills.

https://www.scientificamerican.com/arti ... ink-it-is/

[L'Emmerdeur]

In everyday life situations, I can see this being accurate. Online blowhards are a different case--self-selecting to produce monotonously consistent manifestations of what is now known as the Dunning-Kruger effect.

[JimC]

I am totally confident that my political stance on all and any subjects is the most sensible viewpoint ever!