oṃ in the
For the ancient Indians grammar was one of the major paradigms for understanding how the universe functioned. One product of this is in the understanding of the seed syllable (bījākṣara) oṃ in the Vedic and then the Hindu traditions. The earliest references to oṃ are in the Yajur Veda. This Veda was composed sometime after 1000 BC but before the Buddha. In some rituals the hotṛ Brahmin shouts oṃ at the end of the invocation to the god being sacrificed to (anuvākya) as an invitation to partake of the sacrifice.
The analysis of oṃ as being made up of three parts (a + u + ṃ) originates in the Sanskrit grammarian tradition but is given ritual or religious significance in the post-Buddhist early Upaniṣads, especially the Māṇḍūkya and Praśna Upaniṣads. Let's look at how this works.
Vowels may be monophthong or diphthong - made up on a single sound (short or long), or made up of two sounds (short or long). Linguists might describe a diphthong as beginning on one vowel sound and ending on another. The vowels of Sanskrit can divided up like this:
|monophthongs||a i u ṛ ḷ|
अ इ उ ऋ ऌ
|ā ī ū ṝ|
आ ई ऊ ॠ
Note that the anusvāra (ṃ) and visarga (ḥ) are often counted as vowels, but practically they are modifications of existing vowels: nasalisation and aspiration respectively. They can be applied to any of the vowels. The long vowel ḹ (ॡ) is a theoretical possibility but in practice is never used.
The vowel o is a diphthong which is made up of two sounds: a + u. However note that the vowel au is a long diphthong which is analysed as ā + u. The two vowels o and au sound quite different: o sounds like o in hope; au sounds like ou in sound. Similarly e can be thought of as a + i: and ai as ā + i. Technically (and metrically) e and o are long vowels. In Sanskrit the Proto-Indo-European short vowels e and o converged with a (which helps to explain why a is far more common than other vowels). Since there is no short e or short o in Sanskrit there is no need to write the long vowels as ē and ō, though this would be more consistent.
It is necessary to understand these distinctions in order to understand some sandhi phenomena, because in some cases o actually behaves as a+u. The conjugations of the verbal root √bhū 'to be' offer a good example. This is a class 1 verb and forms a stem in -a with guṇā (strengthening) of the root vowel: so bhū (with guṇā) > bho; and when we add the stem vowel -a we get the stem form bhava; and the 3rd person singular is bhavati. What happens here is that the o in bho is treated as a + u, and the addition of -a invokes sandhi rules governing when two vowels meet - in this case u + a > va: ie bha+u+a > bhava. (This kind of thing is what makes learning Sanskrit difficult).
auṃ in the
From a purely technical point of view we can see that the analysis of o as a + u does not justify writing oṃ as auṃ using the long diphthong. Note that the syllable auṃ written in Siddhaṃ (left) looks like the modern Hindu ॐ which is frequently transliterated as auṃ, and suggests that some confusion about this crept into Hindu discourse. Buddhist texts did not adopt the practice of writing oṃ as auṃ as far as I have been able to discover.
Another purely technical point is that the notation oṃ indicates a nasalised o vowel. This should rhyme with the French 'bon', not with the English 'bomb'. In fact this distinction seems to have been lost for some time, and oṃ (ओं) is regularly pronounced as om (ओम् ) even in India (i.e. with the bilabial rather than the pure nasal). Note also that au is the vowel sound in the English word 'sound'. So auṃ should not sound like oṃ and vice verse.
These jejune distinctions were important to the Indian grammarians because it was thought that the Vedas were divinely inspired, eternally unchanging and true, texts. They were transmitted orally, and after some centuries the vernacular Sanskrit language was significantly different from Vedic  which lead to scholars making a thorough investigation of the language - both canonical and vernacular at around the time of the Buddha. It was important to get the pronunciation right if the meaning was to be preserved. Changing the pronunciation was unthinkable.
By the time the early Upaniṣads were being composed (beginning ca 800 BCE) there was quite a lot of interest in the relationship between words and reality. The existence of eternal, true words gave this a particular flavour. Also note that the word for 'true' and 'real' was the same: sat. It was the authors of the Upaniṣads, especially the Chāndogya (CU), who began to make the connections between syllables and aspects of the cosmos, though this seems to have been a natural development of the idea of correspondences (bandhu) between the macrocosm and the microcosm which was also a preoccupation in the Vedas. Oṃ in CU text is seen as a single syllable and equated with the udgītha, that is with the chanting of the sāman or hymns of the Sāma Veda. In other texts oṃ is associated with brahman. Then later, in the last century BCE, the technical breakdown of o into a+u was given esoteric significance. A key passage from the Māṇḍūkya reads:
so 'yamātmādhyakṣaramoṅkāraḥ | adhimātra pādā mātrā mātrāś ca pādā akāra ukāro makāra iti || ManU 1.8
On the subject of syllables, this syllable 'oṃ' is the ātman; on the subject of metre, the feet are the metre, and the feet are the syllables a, u, and ma. (my translation)
Here oṃ, the ātman, is likened to 'śloka' the poetic metre (mātra ) consisting of four lines or 'feet' (pādā) of eight syllables, with each of the lines likened to a constituent phoneme. The Māṇḍūkya then spells out the esoteric correspondences of the constituent phonemes. The fourth foot (pādā) is said to be without a phoneme (amātra) and ineffable (avyavahāraya).
I'm not aware of any canonical Buddhist text which restates the Vedic breakdown of oṃ into a+u+ṃ, though Kūkai does break hūṃ into ha+a+ū+m suggesting that the technique was not unknown to him.  For Buddhists the esoteric significance is typically based on the Arapacana acrostic which was originally a mnemonic for remembering aspects of an extended reflection on śūnyatā, for example: akāra (the syllable a) is the first syllable; which reminds us of the key word anutpanna (non-arising); and the full reflection subject is akāro mukhaḥ sarvadharmāṇāṃ ādyanutpannavāt (the syllable 'a' is an opening because of the primal quality of non-arising of all mental phenomena). Various versions of the Arapacana exist, the earliest date from around the 1st or 2nd century CE.
This method of analysing mantras is far more significant in understanding the function of a mantra than the words in the mantra. For instance Kūkai always seems to have broken down words (even sūtra titles) into syllables in order to understand their esoteric significant. In Tibetan Buddhism the fact that the Avalokiteśvara mantra has six syllables which enables it to match up with the six realms of conditioned existence is probably more important than the understanding of the word maṇipadme (which has been central to Western exegesis of the mantra).
- This distinction is a bit vague. I call the religion Vedic which is primarily based directly on the three Vedas (Ṛg, Sāma, Yajur) excluding the Atharva (which is a distinct tradition I think) and which is focussed on the sacrifice: it's main gods were Indra, Agni, and Soma - though several dozen deities were propitiated. Hinduism is a complex of various religious ideas and practices where the Vedas have faded into the background and practice is focussed on devotion (bhakti) or with Tantric rites (śakti): prominent Gods are Śiva, Viṣṇu, Brahmā and the mother goddess in many forms especially Lakṣmi and Kāli. This is of course a massive over-simplification. What seems important is to mark that there have been tectonic shifts in India religions over the millennia.
- Vedic is the most common name for the language of the Vedas. It has a number of differences from Classical Sanskrit which was codified by Pānini in the 5th or 4th century BCE.
- Note the phonetic similarity with the English word - both come from the same Indo-European root meaning 'to measure'.
- see Ungi gi in Hakeda Major Works p.246ff
- Hakeda, Yoshito S. 1972. Kūkai: Major Works. Columbia University Press.
- Jayarava. The Arapacana Alphabet. Visible Mantra Website.
- Jayarava. Arapacana on Jayarava's Raves.
- Padoux, André . 1990. Vāc : the concept of the word in selected Hindu Tantras. (Trans. 1992 by Jacques Gontier) Delhi : Sri Satguru Publications.