I want you to do something for me. Think of it as a game, a quiz, a trick. Go and find an old person —one who’s not related to you or a neighbour. Just a random, strange old person, a lurking crusty. It doesn’t matter what sex — sex really doesn’t matter to old people. They don’t do or have or belong to sex any more, they’re just old. Old is the third sex: girls, boys and the aged. So, look at this old person, stare at them, get really close. Don’t be frightened — they won’t hurt you. They’re not contagious; they’re more frightened of you than you are of them. Right, here’s the game bit. Can you tell me how old they actually are? Look carefully at that face, at the wrinkles, the crepey, sunken cheeks, the frail, eroded jaw, the thin folds of wattle. Count the archipelagos of age spots, examine the wind-coloured hair patted into the habit of a lifetime. Look into the fretted, damp eyes, their lids sagging like ragged bedroom curtains, and add up the years. Pick a number, like guessing the weight of a cake or the height of a steeple. You’ll see it’s much more difficult than it looks.

You can discern the years between 16 and 20. You know a 21-year-old from a 28-year-old, but I bet you can’t mark a decade between 60 and 90. You can’t read the gradations and patinas. Not that old people hide them; you can’t tell because you don’t look. And you don’t look because you don’t care. Really, who cares how old the old are? Old is a destination. There is nothing after old. Just nothing. Now, just one more thing: take another look at your old person and tell me, what was it that determined that they were old? What made you think they weren’t just young with a lived-in face and a hangover? If you can’t tell what age old is, how do you know when they’ve got there? Do you think they just wake up one morning to discover they’re past everything but care and caring? Old is not a number, it’s not a date. It’s simply the absence of youth, the absence of attraction, interest, new friends, society. The absence of conviviality, warmth, choice, or surprise, or life.

We have a problem with old age, a huge problem. If we arbitrarily cut the birthday cake at 65, then that makes the old 16% of the population, which will rise to 22% by 2031. The old use up more than 40% of the national health budget. But the old aren’t the problem — it’s the rest of us. It’s you and I that have the problem. It’s our collective refusal to look at the old, to be in a room with them, to ask them into our lives. The great terror of our age is age. We would rather consign the old to a netherworld, a waiting room where they are out of mind and out of sight. The fear is plainly not of the old: it is that we will become them. The old are the zombies at the end of our own home horror movies.

St Leonards-on-Sea was built by an old man. James Burton bought a lump of farmland on the coast and conceived a new town, a town of bracing gentility. This collection of gleaming, po-faced streets, the promenades of probity, the municipal gardens for civic reflection, became journey’s end for the retired civil servants of empire and the imperial military, a final move for the widowed spouses of industrial engineers and provincial department-store magnates, the rheumatic and the consumptive, the dun-achieving tricked down here to the sea’s reflected glister that bounced off the white cliff of stucco guesthouses and sedate residential hotels.

They came to play vicious bridge and smiley bowls, formed exclusive societies, had tea dances and charabanc outings, and filled stuffy rooms with Benares brass, Burmese teak, Turkey rugs and careful china. They hung gilt-framed views of Table Mountain and dead boys in khaki, and dusted the parsimonious riches of adventurous lives lived with a gingerish prudence.

St Leonards was, from the start, mildly risible, a crepuscular community twinned with the letters page, the Conservative party and the crematorium. But it was also a reward, a just desert, a symbol of a life lived with standards, with napkins and polished shoes. And if you had to be old, it was a good place to be old in at a time when being old was an achievement. But those old black-and-white granny ghosts should see it now.

The steely Channel still dowses the front with a squinting brightness, but the streets are gap-toothed, the shops boarded up or given over to charity. In the pub, the motes of wasted time dance in the light over yellow-eyed men in tracksuits, who measure the day in toothpick roll-ups. There is nothing genteel about St Leonards. Like the rest of the south coast, it has been given over to the long-term useless, the invalidity-addicted, the flotsam of refugees and carelessly relocated. But they haven’t displaced the old. They’re still here, the indigenous community, but hiding. St Leonards has one of the highest populations of aged in the country.

On the sixth floor of a caretakered block, an old man sits quietly in his front room surrounded by pottery, china, wood, cats, none living. He has an untouched leatherette-bound edition of Dickens, and above the mantel a print of 18th-century huntsmen quaffing in front of a roaring fire. Their conviviality mocks his stifling solitude. The room has the smell of exhausted air. A clock strikes a cacophonous quarter that would infuriate anyone who had something else to listen to. Here the time doesn’t go quietly. Everything harks to an absent woman. Her knick-knacks and mementos, the holiday souvenirs, the jolly vanities now fight for space and memory with the detritus of communal care: crutches, bottles of pills, easy-grip utensils. This man has diabetes, a heart condition, swollen ankles, but that’s not what ails him. He’s old and alone. His tracksuit bottoms are stained. Food is delivered every two days by a nice agency worker from the Philippines who does a bit of shopping for him.

I am here with the district nurse, who wants to check his blood is not too sweet. She asks how he is; the question is rhetorical. He has a bottle of gin and the telly. Once he lived in Malta, once he was a private detective, once he kept real cats, once he had a wife and friends. He’s not uncared for; he has the tablets and the tinctures, and a string in the corner that, if pulled, will summon a man in a call centre, who will phone to ask what the matter is and, if nobody answers, will send an ambulance. We as a caring society will fend off his creeping death, insulate his awful loneliness for as long as possible. We just can’t supply him with anything worth living for.

In another small flat along the seafront, another man sits in another chair that seems to have grown around him like a fungus. He sits with his back to the bright sea view. The room has hardly any other furniture. There is nought for comfort. In the corner is a glass cabinet containing a collection of china princesses. “They’re nice,” I say. “Did you collect them?”

“Do you want to buy them?” he replies.

This man also has diabetes, and depression, and an ulcer on his leg that won’t heal. He’s had it for years. They think the best cure is to cut off the limb. He doesn’t use it much anyway. He worked on the railways at Paddington, met all sorts — royalty, stars. Every day was different. He had a wife. She died. He had a son and a daughter. They don’t see him or call. He wants to die. The loneliness, the sadness, has made living a mortal sickness. I ask him how old he is: 69. Only 69. He could reasonably expect to live like this, minus a limb or two, for another 15 or 20 years.

Times recommends

• How I lost the gift of joy for a while
• How to get Coleen Rooney's post-baby body
• The A-Z of winter style rules