It began with a chance encounter at a New Year's party in 2004. I was trying to leave, but a woman said she had a message for me. She knew I had been a journalist and she had a friend who wanted my help to solve a family mystery. I agreed to a meeting, and found myself embarking on a five-year quest for a man I had never met.

The woman's friend was called Jane, a financial administrator from St Albans. She was in her late 30s and had been through an emotional experience. Just before Christmas, her mother, Philomena, tipsy on festive sherry, had revealed a secret she had kept for 50 years – she had a son she had never mentioned to anyone.

Jane said her lost brother would be in his early 50s and probably living in America. The reason for the secrecy was that he had been born outside of marriage in Ireland at a time when such things were considered shameful.

A little later I met Philomena herself. She told me she had given birth in a country convent at Roscrea in County Tipperary on 5 July 1952. She was 18 when she met a young man who bought her a toffee apple on a warm autumn evening at the county fair. "I had just left convent school," she said with an air of wistful regret. "I went in there when my mother died, when I was six and a half, and I left at 18 not knowing a thing about the facts of life. I didn't know where babies came from ... "

When her pregnancy became obvious, her family had Philomena "put away" with the nuns. After her baby, Anthony, was born, the mother superior threatened Philomena with damnation if ever she breathed a word about her "guilty secret". Terrified, she kept it quiet for more than half a century. "All my life I couldn't tell anyone. We were so browbeaten, it was such a sin. It was an awful thing to have a baby out of wedlock ... Over the years I would say 'I will tell them, I will tell them' but it was so ingrained deep down in my heart that I mustn't tell anybody, that I never did."

I was intrigued to know why the nuns had been so insistent on the importance of silence and secrecy. The answer, almost certainly, lay in what had happened next.

Philomena was one of thousands of Irish women sent to convents in the 1950s and 60s, taken away from their homes and families because the Catholic church said single mothers were moral degenerates who could not be allowed to keep their children.

Such was the power of the church, and of Archbishop John Charles McQuaid, that the state bowed before its demands, ceding responsibility for the mothers and babies to the nuns. For them it was not only a matter of sin and morality, but one of pounds, shillings and pence. At the time young Anthony Lee was born, I discovered that the Irish government was paying the Catholic church a pound a week for every woman in its care, and two shillings and sixpence for every baby. And that was not all.

After giving birth, the girls were allowed to leave the convent only if they or their family could pay the nuns £100. It was a substantial sum, and those who couldn't afford it – the vast majority – were kept in the convent for three years, working in kitchens, greenhouses and laundries or making rosary beads and religious artefacts, while the church kept the profits from their labour.

Even crueller than the work was the fact that mothers had to care for their children, developing maternal ties and affection that were to be torn asunder at the end of their three-year sentence. Like all the other girls, Philomena Lee was made to sign a renunciation document agreeing to give up her three-year-old son and swearing on oath: "I relinquish full claim for ever to my child and surrender him to Sister Barbara, Superioress of Sean Ross Abbey. The purpose is to enable Sister Barbara to make my child available for adoption to any person she considers fit and proper, inside or outside the state. I further undertake never to attempt to see, interfere with or make any claim to the said child at any future time."

Philomena says she fought against signing the terrible undertaking. "Oh God, my heart. I didn't want him to go. I just craved and begged them to please let me keep him. None of us wanted to give our babies up, none of us. But what else could we do? They just said, 'You have to sign these papers.'

"I remember it was a Sunday evening ... I'm so sorry, I'm crying now when I think about it ... "

Philomena cried when Anthony was taken from her at Christmas, 1955. She was not told he was going or allowed to say goodbye, but she spotted him being bundled into the back of a black car. When she shouted to him, the noise of the engine drowned out her voice, but as the car pulled away she is convinced that he stood up and peered through the rear windscreen looking for her.

Afterwards, her father would not take her back because of the shame: he had told friends, neighbours and Philomena's sisters that she had gone away and no one knew where she was. So in the end the church dispatched her to work at one of its homes for delinquent boys in Liverpool.

Philomena trained as a nurse, got married in 1959 and had two more children. She longed to tell them about their lost brother, but couldn't. She kept her secret but never forgot her son. "Oh he was gorgeous," she told me. "He was a lovely, gentle, quiet lad. All my life I have never forgotten him. I would so often say, 'I wonder what he is doing? Has he gone to Vietnam? Is he on skid row?' I just didn't know what had happened to him ... "

Finally, without telling anyone, Philomena embarked on a lonely, desperate search to find him. She went back to the convent in Roscrea several times between 1956 and 1989 and asked the nuns to help her. Each time they refused, brandishing her sworn undertaking that she would "never attempt to see" her child.

When I agreed to help look for Anthony in 2004, we had little to go on. We knew his date and place of birth, but his name would certainly have been changed by his adoptive parents. Philomena had been told her son would be taken to the US, but little else.

Early on in the search I realised that the Irish Catholic hierarchy had been engaged in what amounted to an illicit baby trade. From the end of the second world war until the 1970s, it considered the thousands of souls born in its care to be the church's own property. With or without the agreement of their mothers, it sold them to the highest bidder. Every year, hundreds were shipped off to American couples who paid "donations" (in reality, fees) to the nuns. Few if any checks were made on the suitability of the adopting families – the only condition laid down by Archbishop McQuaid was that they should be practising Catholics.

When rumours of the church's role began to emerge decades later, much of the incriminating paperwork disappeared in unexplained circumstances, and even today the church guards its adoption archives fiercely. It took a painstaking trawl through passport records and the piecing together of fleeting references in old newspaper articles to discover what had become of Anthony Lee ...

Doc and Marge Hess from St Louis, Missouri fulfilled the McQuaid criteria – they were good Catholics, a professional couple in their early 40s, and Marge's brother was a bishop. The Hesses already had three sons, but they wanted a daughter. In the course of my research, I came into possession of Marge Hess's diaries and was able to trace her innermost thoughts as she flew to Ireland in August 1955 to scour the church's mother and baby homes for a little girl. I read her first impressions of the shy three-year-old, Mary McDonald, who was offered to her by the mother superior of the Roscrea convent. And I discovered the twist of fate that led her to adopt Anthony Lee.

When Marge leaned down to pick up her new daughter in the convent nursery, she was charmed to see Mary's best friend, a little boy in baggy trousers, come running to give her a kiss. She fell for him at once. That evening she called her husband in St Louis and asked if it would be OK to bring two children back instead of one.

Anthony's spontaneous show of affection for Marge changed his life. By the end of 1955, he and Mary had been transported from rural Ireland to a new existence and new identities. He was renamed Michael Hess and grew up to be an A student. He was physically attractive and gifted, ran cross-country and sang in school musical productions. But he was haunted by half-remembered visions of his first three years in Ireland and by a lifelong yearning to find his mother.

Separated by fate, mother and child spent decades looking for each other, repeatedly thwarted by the refusal of the nuns to reveal information, each of them unaware that the other was also yearning and searching.

Michael became a successful lawyer. As a rising star of the Republican National Committee, he masterminded the party's electoral strategy, brokering the redistricting (gerrymandering) reforms that kept them in power for more than a decade. When George Bush Sr became president, he made Mike his chief legal counsel.

But Michael Hess was gay. He was obliged to conceal his sexuality in a party that was rabidly homophobic. He was tormented by the double life he was forced to lead and by the fact that his work was entrenching in power a party that victimised his friends and lovers.

He was tormented, too, by the absence of his mother and by the orphan's sense of helplessness: he didn't know where he came from, didn't know who he was or how he should live. He felt unloved by his adoptive father and brothers; he felt guilt over his sexuality and he had a series of stormy relationships. A spurned lover burned himself to death because Mike rejected him.

But he was loved by his adoptive mother and by the little girl who was plucked with him from the Roscrea convent who became his lifelong friend and sister. He found some happiness in a long-term relationship with a caring, loving partner. But he could never be at peace. He went back to Roscrea, first in 1977 and again in 1993, to plead with the nuns to tell him how to find his mother. They turned him away.

On his return to the US, he plunged into alcohol, drugs and unbridled sexual indulgence. His behaviour brought with it the terrible fear of exposure that would destroy him as a senior Republican official, but he could not stop himself. On one of his lost weekends he became infected with HIV.

He and Pete, his long-term partner, agonised over their future. Pete stood by him, but Michael's health began to deteriorate. Fearing the worst, they flew to Roscrea in 1993 to make an emotional appeal to the nuns ... but still they refused to tell him where he could find his mother, or indeed that her sisters and brother – his aunts and uncle – were living just a few miles down the road.

In desperation, Mike asked the mother superior if he could at least be buried in the convent if he were to die: he would put enough information on his gravestone to help his mother find out about his life "if ever she comes looking for me". As we know – but Mike did not – Philomena was looking for him, returning to Roscrea, seeking traces of her son ...

Obituaries in US newspapers after Michael's death in August 1995 provided vital clues in my search for him. The hunt for Michael took me through state and church archives, through adoption agencies, American university records and Republican party sources before it led to the end of the trail and the story's poignant, unexpected conclusion. It threw up a Hardyesque tale of coincidences and missed connections, and a powerful indictment of two historical eras: 1950s Ireland and 1980s America.

In addition to Mike and Philomena's quest, I discovered the thousands of other lost "orphans" whose lives were changed for ever by the greed and hypocrisy of church and state. Like Michael, many of them are still looking for their parents and, through them, for their identity.

Now in her 70s, and five years after visiting her son's grave for the first time, Philomena is remarkably devoid of bitterness. She has started to go to mass again. But she blames herself for everything, for giving her son away and for not speaking out about him earlier, when things could have been different: "If only, if only. I curse myself every time I think of it. If only I'd mentioned it all those years ago, maybe he wouldn't ... Oh Lord, it makes my heart ache! I'm sure there are lots of women to this very day – they're the same as me; they haven't said anything.

"It is the biggest regret of my life and I have to bear that. It is my own fault and now it is my woe."

The Lost Child of Philomena Lee by Martin Sixsmith is published by Macmillan, £12.99. To order a copy for £11.99 with free UK p&p, go to guardian.co.uk/bookshop or call 0330 333 6846

SOURCE