Scholars are still at odds with each other, and with traditional Buddhist narratives, on the issue of when tantric Buddhism came into being. This essay is an overview of an emerging narrative which relocates Tantra in history, away from representing it as the death throes of Buddhism, but without accepting traditional stories which trace Tantra back to the time of the Buddha in 5th century BCE Indian.From probably the 2nd century sutras began to appear which contained and were focussed upon a form of protective magic. These "dharani sutras" were to prove very popular in China. A little later, perhaps the 4th century, dharani style mantras began to be interpolated into well known Mahayana Sutras, particularly the Lotus, Lankavatara, and Golden Light Sutras. The Golden Light Sutra contains an obvious predecessor of the five Buddha mandala. Indeed there is an alternative version of the Golden Light, which was translated into Chinese earlier than the version which formed the basis of the English translation by Emmerick, and in that alternate and possibly older version, it is clear that a visualisation meditation is intended. By the early 5th century a more or less clear version of a tantric initiation appears in the Karandavyuha Sutra, and mantras were aimed at gaining rebirth in the Pure Land. However scholars agree that a fully formed Tantra definitely can be found in the mid 7th century - this being marked especially by the composition of the first systematic tantric text the Mahavairocana Abhisambodhi Tantra in which mantras are properly a tool for Awakening for the first time.
The early presence of various elements, such as mandalas and mantras (of a sort) which were later adopted by tantric Buddhists, has given rise to the misleading nomenclature "proto-tantra". If for instance the use of mantras constitutes "proto-tantra" then the entire Vedic tradition is proto-tantric. The term is meaningless. It makes more sense to just say that the mantra is Vedic. Similarly Tantra incorporated aspects of Shaiva practice - which is not proto-tantric, it is Shaivite. Ronald Davidson argues in his book Indian Esoteric Buddhism that despite the presence of some elements of tantric Buddhism in earlier periods, that a fully formed tantric movement came into being, quite suddenly, in the mid 7th century. This, he argues, was a response of Buddhism to the political and social chaos resulting from the destruction by the invading Huns of the Gupta Empire with it's extensive trade networks and many wealthy lay merchants.
Early Western scholars struggled to understand the history of Buddhism in India and came to some conclusions, that in retrospect look quite suspect. Some contemporary scholars have argued that this is because those 19th and early 20th century people were applying ready made historical narratives to India rather than observing what was there. The argument is that protestant critiques of the Catholic Church, which were in turn based on the understanding of the story of the Roman Empire were in operation. English and German scholars especially were expecting to find a three act narrative: an original Buddhism based on the founders own words and preserved in a canon of texts; a mature period of consolidation and missionary activity; and a period of decline and descent into idolatry and moral turpitude. Theravada Buddhism was shoe-horned into the first category largely, perhaps, because they had an intact canon and celibate clergy. Mahayana Buddhism was seen as a degeneration, for introducing elements of devotion, but did at least coincide with the spread of Buddhism to Central and East Asia. Tantra however was not even Buddhism, it was a distortion of the rational message of the Buddha, and the "pure" conduct of the Theravada monks.
However as research has filled in the gaps in history, and as methods especially in anthropology have become more sophisticated another picture emerges. There have been streams of Theravada Buddhism which have remained vital, and these fortunately have flourished in the west. However generally speaking the Buddhism of traditional Theravadin countries is moribund: the bhikkhus do not seek Awakening or even meditate; and when not lost in the abstruse categories of abhidhamma they are meddling in politics. This is not to say that early Buddhist methods (don't mention the 'H' word) were not effective if practised, only that the keepers of the texts ended up preferring to chant them as protective spells rather than put the contents into practice (a tendency that can be seen in every type of Buddhism in every county). Where they are practised the most ancient methods are as effective as any that came later, and some of the most inspiring Buddhists in contemporary times have roots in Theravadin reform movements.
The Mahayana is said to have emerged in part as a response to the formalism of early schools of Buddhism which emphasised scholasticism and had drifted into thinking of dhammas as actually existing (a subtle form of eternalism). A closer look tells us that there were many often competing Mahayanas. Arguments continue on exactly when and where Mahayana ideas began to emerge, but Gandhara with influences from Greek, Persian and Central Asian invaders must be at the top of the list of contenders - Indian writing kicked off here, as did making images of the Buddha, both of which had enormous effect on the Mahayana. Many Mahayana ideas - especially regards the Perfection of Wisdom - display links with this part of the world. Ironically some scholars have begun to refer to early, non-Mahayana, Buddhism as "mainstream", when that name more properly belongs to the Mahayana as it was amongst these communities that the Buddha's message was kept alive and constantly renewed, whilst conservative early schools preserved their canon, they did nothing with it.
Finally, and in full contrast to the Western view of it, tantric Buddhism was a reinvigoration of a waning Indian Buddhism which was reeling from incursions from the Huns - whose cousins were heading west to wreck Rome. Trade networks broke down and with it large scale support for monasteries where Buddhism was centred. Social chaos meant a change in priorities and required something new from the religious communities of India. The result was a brilliant synthesis of many existing elements providing both an invigorated search for Awakening, and a powerful protective magic - the two essential elements of (Indian) religion, which despite rationalist views had been present in Buddhism right from the beginning (to judge by our scriptures anyway). Buddhism, let alone Tantric Buddhism, did not survive the subsequent invasion of the Islamic Arabs and Turks that followed the collapse of the Guptas, and when Nalanda was sacked in the 12th century Indian Buddhism was already dead on it's feet. But Buddhism continues to thrive in adjoining areas such as Ladak, Bhutan, and Tibet right down to the present. The Chinese eventually ousted Tantric Buddhism, but it did survive in Japan in part by syncretising with Pure Land Buddhism and creating a strong power base in the laity.
Contra the views of scholars in the 19th and early 20th centuries then, a new view is emerging which looks at Buddhist history in a different light. Theravada far from being pure "original" Buddhism is a quiet, stagnant backwater; Mahayana carried the spiritual impetus of Buddhism forward and out of India into Central and Eastern Asia, and west as far as Persia as well! However it foundered in the breakdown of post Gupta Indian society. Tantra in this view, far from being a degeneration, was a vitally needed renewal in times of maximum upheaval and uncertainty - a positive revitalising response to changing times, rather than a fall into decay. While Buddhism in for example Sri Lanka mouldered (and the ordination lineage died out twice!), Buddhism in Tibet churned out saints and seers by the dozen.
The "original" spirit, the living heart of Buddhism was never situated in formalised rituals and preserved texts, though this view exists in Buddhism and helped to reinforce Protestant presumptions: it was and is in the hearts and minds of Buddhist practitioners striving for Awakening. Some of that spirit has been transplanted into "Western" soils now and we will see whether it can continue to grow.
