November 1861. Sunlight glitters on the lances and pikes of hundreds of attendants and guards, as a procession winds slowly along a mountain road in central Japan. In all, there are 20,000 people – lords and ladies in palanquins, warriors on horseback and on foot, officials, ladies-in-waiting, maids, and maids of maids. Then come shoe-bearers, parasol-bearers, bearers of imperial bath water and the imperial bath, chefs, bearers of food and tea-making equipment, porters humping boxes and dragging huge, wheeled trunks, and grooms leading packhorses laden with luggage – so many that it takes four days for the multitude to pass through each village along the way. In the Victorian West, the railway has been invented, but in Japan, under the rule of the shoguns, there is no wheeled transport other than for goods.

At the heart of the procession, in a luxurious palanquin borne on the shoulders of eight men, is a 15-year-old girl. Her name is Princess Kazu and she is the half-sister of the emperor. She is being escorted from her home in the imperial capital, Kyoto, to the great city of Edo, to marry the shogun, also 15 and who she has never met. The marriage is a last-ditch attempt to bring together the rival clans who are ripping Japan apart in civil war – those who support the shogun and those who fight in the name of the emperor.

The shoguns are military rulers; the word “shogun” means “generalissimo”. For the past 258 years, the same family of shoguns has ruled Japan and given it peace and prosperity. The emperor, meanwhile, is a figurehead with prestige but no power, who never leaves his vast palace in Kyoto.

Now, however, the shogun’s enemies have taken up arms under the pretext that the emperor should be “restored to power”.

Princess Kazu’s is the most splendid procession that has ever passed through these mountains. In the villages along the way there are detailed records of where she stopped to rest, look at the view, dine or spend the night, and of how many thousands of pillows, plates, cups, chopsticks, porters and horses each village had to provide. The imperial chefs left recipes of the dishes they prepared for her – soup, tofu, rice and fish brought from Kyoto. There is even a photograph of her, taken years later. She looks thin and miserable, engulfed in the stiff folds of her kimonos.

Lady Tsuguko, her chief lady-in-waiting, wrote of the princess and the shogun: “Neither of them wanted the marriage.” Astonishingly for a woman of her time, the princess made her feelings known. On the road she wrote poems describing her sadness and resignation to her fate. One reads: “Without regret, If it is for you, my lord, And your people, I will vanish with the dew, On the plain of Musashi.” (“My lord” is her brother, the emperor, and the “plain of Musashi” is the flat lands on which Edo stands, quite different from the beautiful valley of Kyoto.)

After three long weeks on the road, the procession arrives in Edo. Soon to be known as Tokyo, it is the largest city in the world. The travellers wind through the streets past gawking crowds, cross several moats and finally reach the vast white battlements of Edo Castle. They file across the innermost moat, and the double gates swing shut as the last retainers disappear inside.

Edo Castle was Japan’s Versailles. An enormous complex of buildings a mile across and four miles in circumference, it was like Buckingham Palace combined with the Houses of Parliament. It was where the shogun lived and, with the help of an army of government officials, ruled the country. But unlike Versailles, there were no women mingling with the crowds of noblemen and peering flirtatiously over the tops of their fans. As in the Forbidden City in Beijing and the seraglios of the Ottoman sultans, the ladies of Edo Castle lived in seclusion. Visitors were admitted no further than the edge of the omote, the outer palace, where the bureaucrats dealt with affairs of state. Beyond that lay the naka-oku, or middle palace, the residence of the shogun and his personal servants. On the far side of that was a solid wall slicing through the complex of buildings, pierced with a single opening. Only one man could pass through the door: the shogun.

The o-oku, the “great interior” or women’s palace, was larger than the omote and the naka-oku put together. So much mystery surrounds it that it is not even clear how many women lived there, though the most commonly accepted figure is 3,000. All the women, from the grandest lady to the lowliest maid, had to swear never to reveal its secrets, even to their closest family. Most never did. The closest ordinary people came to them was on the rare occasions when a lady went out and the crowd might catch a glimpse of her getting out of her palanquin.

Not long after Princess Kazu arrived there, Edo Castle was surrendered to the imperial forces. The palace was disbanded and the women left to support themselves as best they could. But they still kept silence. It was only many years later, when they were old, that a few revealed a little about what life in the palace had been like.

Some time in the 1890s, an ex-lady-in-waiting spoke to her son, Wakasaburo Ono, who wrote a book called Mother’s Stories of the Castle. In 1892, a group of scholars at Tokyo University interviewed a few ladies-in-waiting and maids who had served the 12th and 14th shoguns. They asked them about life at the palace and the stories passed down about past shoguns and concubines. The women remembered the rules that encircled every aspect of life in the palace and the strict attention to rank. The different positions ranged from the seven elders who ran the palace with a rod of iron, to the nuns who performed official duties, the ladies in charge of hand water and tobacco, those who specialised in ritual music and ritual events and, right at the bottom, the “honourable pups” or “honourable dogs”, who ran errands and lived on leftovers. There was one line that could never be crossed: between those of noble birth who were allowed to enter the presence of the shogun and those below, who throughout their entire careers would never see him.

In 1920 a scholar called Kaneyoshi Takayanagi published a book called Life in the Women’s Palace of Edo Castle, in which he put together material from these accounts together with histories of the shoguns. For many years, that was all there was. Now a recent television series in Japan, set in the palace, has led to a surge in interest and some picture books have come out. But there are very few photographs of the women and none of the palace. Pictures by male artists who, of course, could never have seen it, are carefully researched imaginings.

Plans of the palace show that it was divided into three sections: the wing where the shogun’s wife had her apartments, an area of offices where lady officials carried out the day-to-day administration, and – by far the largest area – the private chambers of the ladies and their maids. In all there were well over 400 rooms and corridors. Only the highest-ranking ladies and those who succeeded in bearing the shogun’s child had their own rooms. The rest had to share.

There were also kitchens, dining rooms, huge baths with areas for reclining, storerooms, shrine rooms, rooms for receiving guests, and pleasure gardens dotted with pavilions. At the opposite end of the palace from the shogun’s entrance was a heavily-guarded gate that led to a bridge across the moat. Here the women came and went on the few occasions when they were allowed to go out, and female merchants brought silks, make-up and other goods to sell.

The palace was a world in itself, with woods and gardens, streams and red-lacquered pleasure barges. There the women amused themselves with poetry-writing competitions, tea ceremonies, the incense-guessing game and the shell-matching game. They performed plays and masques and observed the annual festivals, feasting under the cherry blossoms in spring, performing graceful dances at the height of summer, and picking mushrooms in autumn.

On New Year’s Eve they performed the bean-throwing ceremony to ensure good luck in the following year. One of the senior officials from the men’s palace, usually the elderly keeper of the castle, would cross into the women’s palace along with the shogun. Once he had thrown the beans, 20 or 30 women would bundle him up in a futon and throw him in the air, making sure he ended up sprawling on the mats. For the women there was a certain frisson in being with a man, even if he was old. It was the only time in the entire year that a man other than the shogun set foot in the palace.

With no male guards, the women had to be responsible for protecting the shogun. There were units of women guards skilled with the naginata – the long-handled spear, akin to a halberd. Consisting of a long curved blade as sharp as a razor, fitted to the end of a staff considerably longer than a sword, it gave a woman the chance to get in a good swipe at a man’s legs before he could reach her. Most women of the warrior class were adept with the halberd, but the women of the inner palace were particularly formidable. They studied the art from childhood and prided themselves on their fearlessness and their skill at striking, thrusting, slashing, parrying and blocking. Every woman had a uniform – a thick black broadcloth jacket, stiff black pleated trousers and a black silk cap bound with a white band – and there was a training hall in the palace where they practised.

In fact, the shogun was never attacked in the women’s palace. Several shoguns were assassinated, but their enemies always found other ways to reach them. One was cut down in the outer palace and a couple were poisoned, probably by treacherous ladies-in-waiting. Perhaps the killers knew that if they tried to break into the women’s palace, they would be beaten off.

Daily life in the palace was a picture of elegance. The ladies spent their mornings at their toilette, preparing for the shogun’s three daily visits. Each had maids to help her. The first job was to shave her eyebrows and freshen up the blacking on her teeth. At that time all adult women blackened their teeth with a dye of sumac-leaf gall, sake and iron. A woman with unpainted teeth would have looked barbarous.

Next, the maid painted the lady’s face with white make-up. She outlined her eyes in black, rouged her cheeks and dabbed red safflower paste on her lips, then oiled and combed her long glossy hair and coiled it into a knot.

There were different hairstyles for each rank and also different styles of kimono.

As the clock moved towards 10 in the morning, and again at two in the afternoon and eight in the evening, there was a great flurry and bustle throughout the palace. The highest-ranking women made their way to the Upper Bell Corridor, which led to the double door between the men’s and women’s palaces. As the castle drums sounded the hour, the cluster of bells that hung by the door jangled.

One maid remembered how fearful she was of cleaning them. The bells were only supposed to ring when the shogun entered the women’s palace, and anyone who rang them at any other time was severely punished. But the trouble was that the slightest move set them jangling. Dusting and polishing them were terrifying tasks.

When the bells sounded, shaven-headed nuns who acted as officials unfastened the locks, drew aside the bolts and slid open the door. Men waited on the other side, but only one stepped through: the shogun.

The women’s palace was his home. In the outer and middle palaces he was surrounded by plot and intrigue, by rival factions of courtiers jostling for position. In the inner palace, among his womenfolk, he could relax.

As they knelt waiting to greet the shogun, the words that all the younger women hoped to hear were “What is her name?” This was the code that meant they had caught the shogun’s eye and he wished to spend the night with them.

For the shogun’s primary job and the purpose of the palace was to ensure that he had an heir. The reason no other men could enter was to make sure that any child born there was the son of the shogun and nobody else. Any woman who bore the shogun’s heir was guaranteed a leap in salary and rank. As the mother of the heir, and later, when he grew up, of the shogun, she could advise him as to what decisions to make and which courtiers and petitioners to favour.

People would shower her with gifts, hoping she would speak to the shogun on their behalf.

In the 250 years that the shoguns ruled Japan, only two of the 15 shoguns were sons of the shogun’s wife. The rest were all born to concubines. The wife was usually a Kyoto lady of noble family, married to the shogun in order to bind the shogun’s court to the emperor’s court in Kyoto. Kyoto aristocrats tended to be rather inbred and not very strong, and if the wife had any children, they usually died in infancy.

Officially the concubines were from noble families. All the girls were chosen on the basis of their rank and physical beauty. Only nobility could enter the presence of the shogun and thus in theory they were the only ones he ever encountered. But in practice quite often the shogun would spot a particularly lovely girl among the lower-ranking maids or even out on the street. When a commoner became a concubine, she was said to have “jumped into the jewelled palanquin”. Her family were always richly rewarded. Behind their backs they were derided as “firefly lords” – upstarts who had flitted after the fire in their daughter’s tail.

When the shogun slept with a concubine, the whole procedure was tightly regulated. First the girl was stripped and searched to make sure there were no weapons or notes on her body or in her hair. No hairpins were allowed and combs had to be checked to make sure there was no cutting edge. If it was the girl’s first time, one of the old ladies would check she was a virgin. Once she and the shogun were in bed, there would be two ladies-in-waiting lying wide awake in the room, one on each side. Two more listened behind screens not far way, to ensure that the girl did not make any requests for herself or her family.

No doubt all this formality was as oppressive for the shogun as for the concubine. The early shoguns took to spending a lot of time in the bath, where there was just one cheerful low-class girl to scrub his back. Quite a few children were born to bath attendants; they were known as the “children of the bath”. Eventually the elders put an end to this practice and thereafter the shogun had to bath in the men’s palace.

Both wives and concubines retired from connubial duties at the age of 30, added to which a lot of children died. So there was a constant need for fresh young concubines.

One of the shoguns outdid all the others: the 11th, Ienari, grandfather of the 15-year-old youth whom Princess Kazu travelled from Kyoto to marry. During his lifetime, 1773 to 1841, he fathered 53 children by 27 concubines, though many died as babies and others were mentally or physically disabled. There were also many other women who did not bear children and whose names have not been recorded.

Going to pray – for the shogun’s health, for a son, or at the tombs of the shogun’s ancestors – was the one time when women were officially allowed to leave the palace. Over the years many women, chafing at their enforced chastity, found ways to take advantage of this. The temptation to stray was sometimes irresistible – though the penalties were very severe. The long trunks used to bring kimonos and other goods into the palace were long enough to hold a man – and sometimes they did. Kabuki actors – most of whom moonlighted as male prostitutes in those days – were smuggled in on a regular basis and women also smuggled themselves out. But the best opportunity was after the temple trips, when ladies often treated their maids to a play at the kabuki theatre on their way back to the castle.

A famous case took place in 1714, when Ejima, one of the most senior women in the palace, took her maids to the theatre. There she was entertained by a handsome kabuki actor called Shingoro Ikushima. Legend has it they met again and again and Ikushima was even taken into the women’s palace in a trunk. When Ejima was discovered, both she and Ikushima were sent separately into exile. Ejima left the palace by the Fujomon, the Unclean Gate, a small side gate used only for those who died in the palace or left in disgrace. She walked through barefoot, in a single white shift. Her family were held responsible for her bad behaviour and her brother was sentenced to death by hara-kiri. The 300 maids who had accompanied her to the theatre and enjoyed the company of actors themselves were reprimanded.

Princess Kazu, arriving at Edo Castle at the age of 15 to be the young shogun’s bride, had problems of another sort. She found herself pitted against the widow of the previous shogun, Lady Tensho’in, a beautiful and imperious woman who was officially her mother-in-law even though she was only 10 years her senior.

The trouble began as soon as Princess Kazu arrived, when Tensho’in greeted her seated in the position of honour and on a cushion. Although Kazu was royalty and Tensho’in just a commoner, Kazu was expected to take the humble position, with no cushion. Her lady-in-waiting recorded that she could “find no words to express her anger and frustration”.

Then Lady Tensho’in refused to move out of the wing where the shogun’s wife officially lived. The princess and her retinue of 280 maids had to be housed in the servants’ quarters. Her ladies complained that the rooms were so dark they couldn’t read even in the daytime and had to write letters in the garden.

Princess Kazu’s relationship with the teenage shogun was also complicated. One lady-in-waiting wrote a letter home, reporting loyally: “When the shogun comes into the inner palace, the princess looks after him very well. She is the perfect wife and will soon be granted a son.” In reality Kazu never had children and even refused to see the shogun off when he went to war. The day before he left, however, she gave him a gift of a concubine, to ensure he had a son and heir.

Nobody would have imagined the shogun would never return. Not long after he left, the castle was surrendered, the palace closed down and the women were thrown out into the cold. The girl whom Kazu gave to her husband was destined to be the last concubine of the shoguns s

Lesley Downer’s latest novel, The Last Concubine (Bantam Press, £12.99), is available at the BooksFirst price of £11.69 including p&p. Tel: 0870 165 8585

Explore Women

• Fashion

• Beauty

• Diet & Fitness

• Relationships

• Families

• Celebrity

• Body & Soul

• The Way We Live

Times recommends

• The dirty weekend guide to Europe
• Sharing the McCanns’ hell
• Steve Coogan, weathering the storms