Ego as a term came into being when Freud was translated into English in 1923. Freud himself used the German 'ich' which would normally be translated simply as 'I'. The use of Latin words to translate Freud made it seem that ego had the same sort of status as say metatarsus. Ego, however, is just an idea, a way of talking about a function of the psyche rather than any kind of actually existing entity.
The function Freud was referring to is rather complex. But simplistically it refers to our sense of selfhood. This sense of selfhood forms the basis for how we interact with the world. All human beings pass through a series of reasonably well defined developmental stages. Freud is often, perhaps unfairly, associated with a particular phase of development which he identified - the anal phase. We may disagree over the specifics of the stages, and there have been many models, but it is quite apparent that human beings develop over time, and that infants develop into mature individuals. These developments affect every aspect of the individual, although some may be perceived as being primary mental, and some primarily physical.
Our sense of selfhood appears quite early - around two years old. We become self-aware to some degree long before our bodies are fully developed. A little bit later, around age 3 or 4, comes our sense of other people as separate individuals. This is normal human development. If we fail to develop a sense of selfhood, then none of the rest of our development can proceed. Similarly if we do not develop a sense of people as separate individuals, then we cannot relate to them as people - they are either treated as extensions of our self, or not as people at all. Again all subsequent development is impeded by this lack. Without an ego, without the self-awareness function of the psyche we do not develop into fully functioning human beings.
It's common to hear someone who is bragging, or insistent on getting their own way, described as having a "big ego". However when I started thinking about this I realised that it’s not a matter of having too much ego. The reason people act to reinforce their sense of self is because they are insecure, they doubt their own existence as a self. What they need is not less ego, but more! R.D. Laing is rather out of fashion these days as the trends of treating emotional and behavioural oddities have moved towards chemical approaches. However he coined a very useful phrase: ontological security. By which he means a well defined sense of oursleves as concretely existing. In his book, The Divided Self, he argues that schizophrenia can be seen, in part at least, as an adaptive response from someone who lacks a sufficiently strong sense of ontological security. I find much to recommend this view.
But Buddhism says that the idea of self is false, and that self is at the root of craving and therefore the whole problem of evil in the world. Well sort of. The arguments over what Buddhist texts say about 'self', or indeed what the word itself means, fill many books. Scholars have come to a range of conclusions over what various schools of Buddhism have said about the existence of a self. The Pali texts, from my reading, seem to focus more on the preoccupations of selfhood (sakkāya-ditthi), rather than selfhood per se. And this makes sense to me, because if the absence of ego is a debilitating developmental problem, and the goal of Buddhism is the absence of ego, then Buddhism is creating a lot of vegetables! It is true that the Buddha said that phenomena lack an unchanging essence, but this statement is attacking the Upanishadic idea of a soul (ātta) which exists independently of mind and body, and is permanent and unchanging. I don't think that this is the same thing as a sense of selfhood in the sense that I am talking about it.
The approach of some Buddhists to feelings of insecurity regards selfhood, is to attack the ego with even more vigour. After all this is what the masters in the old stories do. But I think we are different in crucial ways from many of the great spiritual heros of the past. I think many of us have been held in arrested states of development. Our societies tend to encourage infantile behaviour, reward it even. Most of us have some way to go before our ego’s are strong enough to withstand the rigours of all-out spiritual practice. Paradoxically we must have a strong sense of self in order to contemplate a world in which we are not the most important being. If you tell someone with poor self-esteem that they have to kill off their ego, then you are asking for trouble.
I said earlier that our sense of other people as people is dependent on our sense of selfhood. That the early Buddhists knew this is shown by a verse in the Pāli Canon which suggests that by reflecting that all beings regard their sense of self just as preciously as we regard our own, we can develop the empathy which stops us harming them. [Samyutta Nikāya 3,8 (8) ] It is a rare passage to be sure, but it neither denies selfhood, nor demonises it.
There is no doubt that if we are self-preoccupied then it will be hard to be really happy. Getting caught up in the preoccupations of self does tend to be painful. However it doesn't seem practical to me to treat self-preoccupation by trying to annihilate any sense of self. It's interesting that in English we have the adjective selfless, which doesn’t mean 'lacking a self', but 'concerned for the welfare of others', or 'not being self-preoccupied'. Selflessness then need not say anything about whether or not we have a self, or an ego, but it does point to an attitude which seeks the benefit of others. Self-preoccupation, as I argued in my essays on the six perfections, is best tackled by becoming aware of other people as people. And for that we need to have a sense of selfhood.
So: ego is absolutely essential in the spiritual life!
See also these other Raves on the subject of Ego.