Charlotte Gainsbourg, in a ginger wig and full Edwardian costume, is swimming around in a pool of milk, using a giant carrot as a float. Few actresses could carry off such a scene without fatally damaging their cool. But, in her latest film, the rather strange Golden Door, Gainsbourg manages it – just.

She has, after all, been effortlessly hip from the cradle. The daughter of the upper-crust, gap-toothed British actress Jane Birkin, who scandalously flashed her pubic hair in Blowup, and France’s enfant terrible of pop, the late Serge Gainsbourg, she was making headlines before she was even born.

And, while most 15-year-olds are delighted to star in the school play, by that age Gainsbourg had three well-regarded films under her belt and a César award – the French equivalent of a Bafta – on her mantelpiece, and had caused an international controversy with the release of Lemon Incest, a song she performed with Papa, lisping, “The love we will never make is the purest, the most tender,” to the tune of Chopin’s Etude Op 10 No 3 in E.

We meet, of course, on Paris’s Left Bank, the spiritual home of the Gallic avant-garde, where Gainsbourg lives. She strides in, tall and slight, flicking her straight brown hair away from her face. She has a big nose, sticking-out ears and bushy eyebrows, and is wearing a giant khaki shirt, rolled up over her arms, jeans and battered cowboy boots. Most 35-year-old mothers of two would look ludicrous like this; she, naturally, is casually glam and teenager skinny.

Doubtless hoping that some of this cool rubs off, the French designer Gérard Darel has hired her to be his model and muse and has had a bag created for her, the Drape, which is ubiquitous in Paris (although it cannot compare to the Hermès Birkin, named after her mother).

Unlike most muses, however, she doesn’t discuss the label, and when I ask about her wardrobe, she tells me that she’s “not really interested” in fashion. “I love Nicolas Ghesquière, who does Balenciaga,” she says. “I don’t really pay attention to other things, because I just like what he does.”

So, Charlotte, I say breathlessly, are you the coolest woman in Paris? The sphinx face cracks into a grin. “No,” she says (even her accent is perfect, Birkin posh with a hint of Gainsbourg sexiness). “I don’t go out; I stay at home with my children. Sometimes, parties are nice, but I don’t enjoy myself, really.” That is precisely what she would say. No cool person would ever admit to enjoying parties, hanging out with pop stars and getting free clothes. The hip-to-the-bone will insist that they barely leave the house, have no friends and spend their time crocheting rugs. So you must judge her by her actions, not her words.

For instance, Gainsbourg is married to the dark, handsome, César-winning director Yvan Attal, with whom she has two children, Ben, 9, and Alice, 4. She has barely taken a false step in two decades of bilingual acting, from Zeffirelli’s acclaimed Jane Eyre to 21 Grams, with Sean Penn. She has just finished filming I’m Not There, a Bob Dylan biopic also starring Cate Blanchett and Christian Bale, in which she plays Dylan’s wife, Sara. As if all that weren’t enough, last year, as a little project to entertain herself, she recruited the groovy electro-pop duo Air and Jarvis Cocker, now a Paris resident, and recorded an album, 5:55, that received rave reviews. I can just imagine her, Cocker and his wife, the super-cool stylist Camille Bidault-Waddington, hanging out together – but apparently not. “Jarvis is not a close friend, but I love him. I admire him very much,” she says. “He’s quite secretive, a strange character.”

Golden Door, Gainsbourg’s latest film, follows the fortunes of a Sicilian widower and an Englishwoman (played by Gainsbourg) who meet on a steamship to America, where they believe money grows on trees and hens are the size of horses – hence the giant-carrot sequence. “We were in a swimming pool for five hours,” she says. “It was lovely, but there were so many in the pool, and not knowing who was peeing...”

Golden Door was mostly filmed in Buenos Aires, which meant Gainsbourg spent four months away from her family. “I thought of not doing it,” she says, “but the adventure was so exciting. It was a big deal not having my children, though. I’d never do that again.”

Much of her own childhood was spent hanging around film sets and recording studios. “It’s special for a child, but we never missed school,” she says. “My parents weren’t workaholics. It was a normal life.”

Normal is, of course, a relative term: Gainsbourg père was an alcoholic, and his house, which she hopes to turn into a museum, was painted black from floor to ceiling. He considered himself so hideous that he refused to own a mirror, but women found him irresistible. First, there was Brigitte Bardot, but his apparent phobia of breasts led him to fall for the elfin charms of Birkin. They separated when their daughter was nine. Birkin, despite having another daughter (Lou Doillon, the actress and model), with the film director Jacques Doillon, continued to say that Serge was her one true love, which must have been rather confusing for all concerned.

It seems that Gainsbourg was a lonely child; she was teased by her classmates and, as a result, changed school every year. “I didn’t make many friends,” she says. “I protected myself.” I bet they’re all kicking themselves now.

She launched her acting career at 12, disappearing off to Canada to film Paroles et Musique with Catherine Deneuve and Christophe Lambert. “When I see my own children, 12 seems so small to let me go off abroad. I don’t know how my parents felt about it,” she says, lighting a cigarette. “It was only for a week. Having my own room and nobody to look out for me was great, but for them... I don’t know if they were scared.”

This was all happening around the time that Gainsbourg recorded Lemon Incest. “My mother felt it would be nice for me not only to have this thing with my father, but also something of my own,” she explains. “She felt acting would make me stronger.”

These days, although she’s uncomfortable selling herself as the heir to her father’s musical talents – “Because I admire him and put him on a pedestal, it would be arrogant for me to say I am like him” – she wants to continue making music, ideally with Cocker and Air. “It was the perfect thing, with them,” she says.

But, however cool the result, isn’t there something a little bit try-hard about middle-aged mothers making pop albums? “You can think that music is for young people,” she says, “but I hope it’s not true – otherwise I’ll have to stop.”

Well, if anyone can carry it off, it has to be Gainsbourg.

Golden Door is released on June 29

Times recommends

• Are eldest children really a cut above?
• Peaches Geldof: censored
• Top 10 style-setting films of all time