In The Narcissism Epidemic the third chapter challenges several myth’s about the narcissist, specifically the idea that some narcissism is good. Twenge and Campbell write:
Our culture tells us it pays to believe in yourself as long as you aren't arrogant or narcissistic. However, this isn't really true [...]. A major review of the research on self-esteem and achievement found that high self-esteem does not cause better grades, test scores, or job performance. It's a problem of correlation not equaling causation. There is a small correlation between self-esteem and better achievement, but it is almost entirely explained by better performance causing higher self-esteem. Self-esteem comes after success, not before, because self-esteem is based on success (whether that's academic success or simply being a good friend to someone). Much of the rest of the already small link is due to confounding variables- rich kids, for example, have higher self-esteem and make better grades. Some children with low self-esteem do poorly, but it's because they were abused or had parents who did drugs- things that cause both low self-esteem and poor outcomes. On its own, self-esteem does not lead to success.
Think about it this way: if self-admiration caused success, American children, who have the highest self-esteem of children anywhere in the world, would also be the most successful. This simple prediction, however, doesn't match the data. In a recent study, 39% of American eighth-graders were confident of their math skills, compared to only 6% of Korean eighth-graders. The Koreans, however, far exceeded the U.S. students' actual performance on math tests. We're not number one, but we're number one in thinking we are number one.
Wikipedia describes the Dunning–Kruger effect as a cognitive bias where people of low ability suffer from illusory superiority, mistakenly assessing their cognitive ability as greater than it is. The cognitive bias of illusory superiority derives from the metacognitive inability of low-ability persons to recognize their own ineptitude; without the self-awareness of metacognition, low-ability people cannot objectively evaluate their actual competence or incompetence. This isn't necessarily a problem of an overactive ego, or excessively high self-esteem, but rather it is our own inability to estimate how good we are at doing things that creates the problem.
There is a good video on YouTube titled: Why incompetent people think they're amazing. It describes several studies, one done at two separate computer companies where the programmers were asked to provide a rating for their own performance. At the first company 32% rated themselves among the top 5%, while at the second company over 60% of the programmers rated themselves in the top 5%. In another study 88% of American drivers described themselves as above average and more competent behind the wheel than most others. This same effect, where individuals overestimate their ability can be demonstrated for all sorts of skills, activities, and attributes. The vast majority of people simply believe that they are above average at most things. Numbers like these violate the simple law of averages which states that the odds of you being a little worse than average at some task are about equal to the odds that you are better than average at some other task. Those with the least ability are most likely to overrate themselves by the highest degree. Poor performers lack the very expertise needed to understand the problems with what they are doing.
People are exceptionally bad at estimating how good they are at something, and not only does this apply to the incompetent who wildly overestimate their abilities, but it also applies to experts who tend to do the opposite and underrate their abilities.
Graduate students, professors, and other high achievers often suffer from a phenomena called the imposter syndrome. Wikipedia provides a good overview:
Individuals who suffer from the impostor syndrome have a marked inability to internalize their accomplishments and a persistent fear of being exposed as a "fraud". The term was coined in 1978 by clinical psychologists Pauline R. Clance and Suzanne A. Imes. Despite external evidence of their competence, those exhibiting the syndrome remain convinced that they are frauds and do not deserve the success they have achieved. Proof of success is dismissed as luck, timing, or as a result of deceiving others into thinking they are more intelligent and competent than they believe themselves to be.
[Several behaviours are common to those that suffer from imposter syndrome]:
Diligence: Gifted people often work hard in order to prevent people from discovering that they are "impostors". This hard work often leads to more praise and success, which perpetuates the impostor feelings and fears of being "found out". The "impostor" person may feel they need to work two or three times as hard, so over-prepare, tinker and obsess over details. This can lead to burn-out and sleep deprivation.
Feeling of being phony: Those with impostor feelings often attempt to give supervisors and professors the answers that they believe they want, which often leads to an increase in feeling like they are "being a fake". If shown evidence of their competence or that they may suffer from a case of impostor syndrome, they tend to doubt themselves even more.
Avoiding display of confidence: Another way that a person can perpetuate their impostor feelings is to avoid showing any confidence in their abilities. A person dealing with impostor feelings may believe that if they actually believe in their intelligence and abilities they may be rejected by others. Therefore, they may convince themselves that they are not intelligent or do not deserve success to avoid this.
As described by social psychologists David Dunning and Justin Kruger, the cognitive bias of illusory superiority results from an internal illusion in people of low ability and from an external misperception in people of high ability; that is, "the miscalibration of the incompetent stems from an error about the self, whereas the miscalibration of the highly competent stems from an error about others." Hence, a corollary to the Dunning–Kruger effect is that persons of high ability tend to underestimate their relative competence and erroneously presume that tasks that are easy for them to perform are also easy for other people to perform.
While the imposter syndrome is not described in the standard manual of psychiatric disorders (DSM-IV or DSM-5), it is a risky state of mind. Underrating your physical abilities might deter you from joining an exercise group which would otherwise welcome a new member, and this might represent a missed opportunity for socialization. Such decisions, while not catastrophic, are unfortunate. Genuine mental health issues often result from similar distorted perceptions of reality, and these can take many forms. In extreme cases, a person may begin to doubt their competency at relatively basic tasks. When feelings of incompetency and being an imposter make their way into your day-to-day life, like being unable to ride the bus because you believe you won’t understand the route or the schedule, or feeling rejected by others in simple situations like going to a grocery store or a bank, then a false self-evaluation can wreck real havoc in your life.
In MHTWT there is a chapter titled: “The Passion for Self-Distrust”, Low writes:
My patients have gone through months or years of torture and in the process developed sustained tenseness and symptoms attending it. Their weariness, their pains, fatigues, pressures and spasms have made them self-conscious in the extreme. Hence, they lack the feeling of vitality and accomplishment; they have lost their self-confidence, are unable to relax or enjoy things. Required to formulate plans and intentions they are instantly gripped with the fear that their muscles will fail them, that they will not be ready to carry out what they are asked to do. Being the victims of an unrelenting self-consciousness they question their capacities, watch and check every one of their moves and perform with hesitation and anxiety. Their attitude is that of an abiding pessimism; they feel whipped and defeated; their guiding philosophy of defeatism has hardened into a settled conviction. They are "sure" and "certain" and "positive” that acting is impossible, that their muscles will defy orders, that their power to get things done is lost, that their personality functions are doomed. Their philosophy of "I can't" has assumed the status of a dogma; it is implicitly believed, hotly defended and fondly sheltered. The calamity is that the relatives and friends do not share the patient's defeatism and refuse to subscribe to the cult of "I can't." They look at the sufferer and notice a blooming complexion, a strong voice, a lively facial expression. They observe the patient in a fit of his frequent tantrums and witness a display of force and energy which belies the claim to invalidism. Their conclusion is that the patient could but would not do the things which are to be done. The idea is forced on them that he is unwilling instead of unable to perform his function. They upbraid him, urge him to make an honest effort and with this they accuse him of shamming disease, of playing a game, of practicing deception. They indict his character, his honesty; they charge him with deliberate neglect of duties and obligations and fasten the label of irresponsibility on him. This strikes at the root of his self-respect, of his personal value and social position. This savage assault must be repelled. The patient feels he must bend every ounce of his energy to the vital task of convincing the others that he "really" can't, that he is "truly" incapable of acting, that he is "positively" helpless. The patient is now a crusader for the philosophy of "I can't." He concentrates on the effort to win over the others to his dogma of defeatism, to make converts, to spread the gospel of his incurability. In order to convince those about him, including the physician, he must engage in a veritable campaign of complaining, wailing, lamenting. In his interminable moaning and groaning he is compelled to overemphasize the utter unreliability of his organs and functions. His body is forever about to crumble, his mind is constantly ready to disintegrate. As he continues on this career of self-denunciation he fairly gorges himself with the idea of distrust and in the end develops the PASSION FOR SELF-DISTRUST. His untiring crusading for the philosophy of "I can't" has netted him one faithful and unswerving convert: himself.
Rather than thinking about this information in the context of judging or criticising someone you know who claims that they are either awesome or incompetent, when you are sure that they are perfectly average and neither an expert nor a total failure, it’s more valuable to apply this information to yourself. In Recovery we say “Expectations can lead to disappointments”, and while it’s true that when you apply for a job (ask someone on a date, enter a marathon race etc.), you have a certain expectation that you are probably qualified and might get what you are hoping for, it is important to remain less focused on the outcome and more focused on putting in a good effort. Keep in mind that you will probably not have a good understanding of exactly what the other person is looking for, or who you are competing against, and that you may misjudge your own abilities. This confusion is average. It is average for people to need to experiment to understand their own abilities, and it is average to never get a full picture of how you rate against everyone else. In Recovery we say “Mistakes are healthy, wholesome and necessary” and that without mistakes we can’t learn.