Lady Antonia Fraser adjusts her pearls, gazes out of the french windows opening out to the garden, and tells me to f*** awf. This, five minutes into our interview, comes straight after her waving a two-fingered salute at Private Eye.

I had inadvertently mentioned the satirical magazine, so thought I might as well ask her whether she had forgiven the chaps yet for nicknaming her Lady Magnesia Freelove – ooooh, about four decades ago, when London was swinging in every sense of the word. Her first response was as measured and dignified as her demeanour: “I’ll tell you what, Ginny, I decided that as I was campaigning for a free press, I couldn’t object. But I, too, was free and I never read Private Eye again – because I have the freedom not to read it.”

She went on to say that she does read all her reviews: “I take the criticism, you know. I’m interested by it. Of course, I’d much rather have a favourable than an unfavourable review and I mind what the public thinks of my books and I mind what the critics think, you know, historians, but as to what Private Eye thinks, well…” and then came the surprising V-sign.

Did she do that before she met Harold Pinter? “No, he’s been a very bad influence on me.” I tell her about an interview I did with the late Alan Clark when, on a tour of Saltwood Castle, he greeted a magisterial portrait of his father, Kenneth “Civilisation” Clark, with the same disrespectful gesture. “How frightfully funny!” Lady Antonia, 75, says. Does she often use the F-word, I ask. “No. That’s why I put my fingers up.” Has she ever used it? “Yes.” Can I hear you say it? “Well, I don’t want to look at you. Erm…” and then she gamely obliges. But why did she feel that she had to avert her gaze? “Well, I thought it would be so rude to look at someone and say it,” she says, and offers me another cup of coffee.

We are sitting in the living room of the house in Holland Park that has been home to Fraser for most of her adulthood. Like her rich and varied life, there is an impression of colour and profusion: walls covered in paintings, flowers tumbling out of vases, every inch of a coffee table layered with handsome books on opera, which she describes as her passion. She is wearing a smart navy dress and has debutante deportment, knees clamped tight at right angles to her feet, which are clad in black patent leather court shoes. This is where she lived with her first husband, Sir Hugh Fraser, the Catholic Conservative MP whom she married in 1956 at the age of 23, and, six children later, divorced in 1977. Two years earlier, the Frasers and their guest Caroline Kennedy narrowly escaped being blown up by an IRA bomb which had been secreted under the MP’s Jaguar. Their neighbour, Gordon Hamilton-Fairley, was killed when he spotted something suspicious under the car while walking his dogs.

This was the same year, 1975, that Lady A had her coup de foudre with the playwright Harold Pinter while he was still married to the actress Vivien Merchant. The next year, her anthology Love Letters was published with its dedication “for Harold”. In her introduction she wrote: ‘It is obvious… that I am on the side of love letters… Anyone can write a love letter and almost everybody has – one should beware those who boast of never having fallen in love, there is either something missing somewhere or else the boaster is subtly begging to be roused from his or her frozen state of inanition.”

This reads like a clarion call to lovers. During her research, she wrote: “My friends were not slow to suggest the great love letters of fiction, whereas I should have much preferred them to turn out their own.” Fraser has always maintained that her intimate approach to historical biography – did such and such a king visit his mistress’s bed or vice versa – revealed a great deal about the character of her subjects as well as the period.

I had rather hoped that this might mean she would be relaxed about talking about her own ancient history in this respect, the list of admirers detailed in the Daily Mail all those years ago, but she says: “I am making no comment on that. I have never confirmed or denied.” But why have they (Jonathan Aitken, ex-King Constantine of Greece, Rupert Lycett-Green, Lord Lambton and Robert Stephens, who confirmed an affair in his autobiography) been written about with such authority? “You tell me. But what I would point out is you will not find one statement from me on the subject.” Does she think it is unseemly to talk about it, even at this remove, or that married women shouldn’t take lovers… “None of your business,” she says, firmly but without a trace of froideur.

In my research, I came across a gem of an article written by Aitken in 1969, the year of Fraser’s first biography, Mary Queen of Scots, which was a publishing phenomenon. He sounds mildly irritated: “Antonia Fraser rather defensively likes to mention the interviews she has turned down. Some cynical observers might think she has turned them down only because she had difficulty fitting them into her schedule.” But then beguiled: “Lady Antonia turns out to be a sort of Lady Madonna of the tennis courts. Clad in a plain white miniskirt, with a glory of golden hair tumbling over her shoulders, and beautiful Botticelli-like features, she looks about half the 36 years she claims on the book’s dust jacket.”

Wherever this attraction may or may not have led, the two have remained close in the intervening decades. She describes him as “a very kind person who takes a lot of trouble… I’m sure there are lots of people in the world who nobody knows about who’ve been helped by Jonathan.” She talks about her grandson – one of an incredible 17 grandchildren – Thomas, son of Benji, who is at Harrow where Aitken gave a talk about literacy in prisons and prison reforms: “Thomas went up to him and introduced himself and Jonathan took infinite trouble to talk to him about his grandfather, Hugh, whom of course he never knew.”

I wonder whether she found her old friend much changed after his seven-month spell in prison. “He came to lunch after he came out and he was incredibly thin, of course. Very, very thin,” Fraser recalls. “Yes, I think he has changed. He would say that he’s seen the light. I don’t know what language he uses but…” He’s embraced religion? “Really embraced it, believes strongly. And this is what saved him in adversity. I think it’s wonderful to be saved by something spiritual.”

This talk of prisons and spiritual succour takes us into Fraser’s own fascinating family and, in particular, her father Lord Longford, who died in 2001 at the age of 95; 14 months later, in October, her mother, the writer Elizabeth Longford, died at 96. In November, the next month, Myra Hindley – the child murderer on whose behalf Lord Longford had campaigned – also died, at 60, of a chest infection.

Explore Women

• Fashion

• Beauty

• Diet & Fitness

• Relationships

• Families

• Celebrity

• Body & Soul

• The Way We Live

Times recommends

• Dumped by my best friend
• I froze my eggs
• Just give them grit