Anatta is the third of the tilakkhaṇā or three marks. In the Dhammapada 279 it says that sabbe dhammā anatta - All dhammas are non-self. The order of presentation of the lakkhanas is significant. In fact it is helpful to work through them backwards. We might ask for instance why are all dhammas anatta? They are anatta because of the second lakkhana - dukkha. Dhp 278 says in fact that sabbe saṅkhārā dukkha - all compounds are suffering.
Backtracking a little we need to look at what atta or ātman is. Ātman, using Sanskrit because it fits the context, is a concept introduced by the philosophers associated with the Upaniṣads. It was introduced not that long before the Buddha and was a distinct move away from the Vedic religion which had revolved around sacrifices to gods, and bonds between this world and the cosmos known as bandhu. It was also associated with a new idea about reincarnation - Joanna Jurevich has shown that reincarnation in a nascent form is, contrary to popular opinion, present in the Ṛgveda. However the Upaniṣads made reincarnation dependent on the actions of the person, on their carrying out of their religious duties and ceremonies. Ātman here was the immanent aspect of godhood - brahman. Not to be confused with the masculine personification of godhead Brahmā. Brahman was an abstract absolute transcendental principle. However the Upaniṣads equate ātman and brahman. The latter idea became highly influential in the popular form of Hinduism known as Avaita-Vedanta. The immmanent and transcendent aspects of godhead were not two. Brahman was said to have only three attributes (trilakṣaṇa) : satcitānanda - being, consciousness, and bliss. Ātman seems to have been the most influential religious idea in India at the time the Buddha was born. One's attitude to ātman - to the nature of selfhood as immanent godhood - was what defined many religious discussions, just as the existence and influence of the Christian God define religious discourse in the present.
Returning to the Buddhist anatta idea we can see that where there is an experience of dukkha - suffering, misery, diappointment, grief, etc, then that is not blissful. What is not blissful is not, ipso facto, ātman. Now the Buddha says that all compounded experiences are disappointing. The Buddha seems to have considered all experiences associated with the senses or the mind, which he considered as being synonymous with all unenlightened experience, as being disappointing (dukkha). Hence his constant refrain that the senses and the cognitive apparatus are anatta - not the ātman.
Note also that the Buddha taught that cittā - consciousness - arises in dependence on contact between a sense organ and a sense object. Because of this we must consider all sense experience as compounded or complex. More crucially cittā ceases when the contact ceases. Now if consciousness (cit) is a dependent product of contact, then brahman in it's cit aspect is conditioned! This is a major blow against the Upaniṣadic philosophy that doesn't get much attention these days because Buddhists are largely ignorant of that philosophy and fail to see the relevance of it.
We need to briefly mention that the reason that the Buddha said sabbe saṅkhārā dukkha, was because he had already observed in Dhp 277 that sabbe saṅkhārā anicca. Compounds are compounded of dhammas - and these are the objects of mano, the mind, and therefore saṅkhārā is more or less synonymous with cittā when used in this sense. Because we fail to properly see dhammas as ephemeral and fleeting (see also Language and Discrimination) we find all of our experiences disappointing. (The argument for unpleasent dhammas is more complex, but it also amounts to disappointment).
So in forward order: experiences are fleeting; because we don't get this at a fundamental level we find experiences disappointing; and because experience is not blissful it cannot be ātman. So nothing related to the body, senses, or mind - the apparatus of experience - can be the ātman. This is the proper context for the idea, and is the only context where it really makes sense.
Now for a variety of reasons, most of which relate to later Buddhist failure to take interest in the context the Buddha was operating it, the doctrine became decontextualized. Buddhists began to make new explanations for what the Buddha meant by anatta. One of the most prominent became that the Buddha taught that we have no self. There is apparently, and here I rely on Sue Hamilton, no explicit denial of self per se in the Pali Canon. What the Buddha denies is that any aspect of our experience is ātman in the sense of immanent godhood. The Buddha is trying to reframe the religious discourse away from ātman and towards a consideration of the existential experiential situation - he repeatedly refused to answer metaphysical questions and responded that he taught "suffering, the cause, the end and the way to end suffering".
A popular version of this corruption is that the Buddha taught something called "egolessness". Now this is problematic in several ways. The term ego is introduced by Freud's English translators - he called the psychic function in question "ich". Using Latin led to a reification of the term in popular usage - it moves from being an abstract function, to being a concrete part of the person. One can now speak of "having an ego", for instance, as though ego is a "thing". One can have too much ego, or perhaps too little. This is a dismal error that flies in the face of Buddhist approaches too being as process as well as what is intended in psychological jargon.
Buddhists take this one step further by making the ego wholeheartedly bad, and proposing that all people should be egoless. A person with no ego would be incapable of communication or learning, or any kind of interaction. Egolessness would be disastrous for the individual. I've expounded this at length in the past. Ātman as the immanent godhood is nothing at all to do with the ordinary sense of self. The Buddha even at one point suggests that a sense of self is essential for the development of empathy! I've suggested that the English word "selfless" is much more in keeping with the Buddhist concept - it means not, someone with no self, but someone who is altruistic! A final irony is that Buddhists who promote egolessness are often the same ones who are proponents of the doctrine of tathāgatagarbha (literally "the matrix of one who is like that") - or Buddha nature. Now some of the tathāgatagarbha literature equates the tathāgatagarbha with ātman (see for instance Williams, p.98-9). So while treating anatta as egolessness, they promote the idea of an intrinsic immanent Buddhahood which is like the ātman. So we're basically back to Vedantic eternalism at this point, the very kind of idea which anatta was designed to critique.
The idea of anatta is often elevated to being "the doctrine of anatta". I don't think it was ever intended as a stand alone doctrine. It seems more likely that it required not only a Buddhist context, but the Vedantic context against which it was being offered as a polemic, in order to make sense. So on the whole it does not make sense in the present. Anatta was part, and only a part, of a Buddhist demolition Vedantic arguments which are not relevant in the modern west, though it may still be relevant in India. What we need at present is a Buddhist critique of the Christian idea of creation, and the scientific idea of evolution. Both tend to draw attention away from the existential situation and from the problems associated with the apparatus of experience - and therefore neither are likely to be helpful in the Buddhist Enlightenment project. Perhaps a subject for a future rave...
- Hamilton, Sue. 2000. Early Buddhism : a new approach. The I of the beholder. Richmond, Surrey : Curzon.
- Williams, P. 1989. Mahāyāna Buddhism : the doctrinal foundations. 1st ed. London : Routledge.