John Prescott has agreed to give up his country estate at Dorneywood, saying the row about his continued presence there was getting in the way of his job.
The deputy prime minister has been under fire for holding on to the official home in Buckinghamshire after losing his departmental responsibilities in the reshuffle last month.
The publication of photographs of Mr Prescott - who continues to draw his £134,000 ministerial salary - playing croquet in the grounds of the house during office hours last week further stoked the controversy.
At least two Labour MPs have called for him to resign, saying that in the wake of his affair with his secretary, Tracey Temple, he had become too much of a joke to remain.
In a statement issued last night Mr Prescott said: "I have accepted that my continued use of Dorneywood is getting in the way of my job in government."
He said he had used the house as a place to relax, as deputy prime ministers had before him, but could no longer do that, fearing photographers would record his every move.
"I am well aware that my use of [Dorneywood] is now a subject of public controversy and criticism, and a matter of concern among some MPs and the Labour party," he said.
"I have told the prime minister that it is my personal decision that I no longer want to be the official resident. He has accepted this decision.
"I will continue to focus on the role the prime minister has given me."
In an interview with The Guardian, Mr Prescott said Tony Blair warned him that keeping Dorneywood would make him a target for critics, but he had insisted on staying.
But he implied that much of the hostility directed at him in the media - which has been reflected among the Labour backbenches - was an attempt to get at the prime minister.
"At the moment I am the shield they are battering," he said, but added: "Tony and I are a good pair."
His announcement has been welcomed by some Labour backbenchers - Stephen Pound said it was a "dignified statement", although warned it could have come earlier.
"John will be haunted until the day he leaves parliament by the picture of the man playing croquet on a Thursday afternoon," he added.
Ian Gibson MP added: "I guess this is a reaction to the pressure that has been building up and I think this is a gesture towards recognising that this kind of lavish behaviour is unnecessary."
However, Conservative party chairman Francis Maude was sceptical that it would help diffuse criticism of the government, given the ongoing rows about the NHS, immigration and law and order, to which Mr Prescott's antics had simply added.
"At last John Prescott has done the right thing. It is just a pity he has taken so long to do it. But John Prescott's absurd antics are ultimately not what matters to people - they are not really interested in what minister moves in and out of what house," he said.
"What they are concerned about is the health service being run properly, tax credits being administered in the right way and having foreign prisoners deported. John Prescott is merely a symptom of a government that is falling apart at the seams."