The bedroom tax is forcing nearly nine out of every ten people affected to worry about meeting their living costs, new research suggests.
Polling by Ipsos Mori for the National Housing Federation found over three-quarters are worried about falling behind on their rent, compared to 46% among unaffected social housing tenants. Seven in ten are worried about conviction.
The research found a third of people hit by the change in rules - which since April 2013 has removed half of housing benefits in homes with two unoccupied rooms - had been forced to cut back on food.
A further 46% had been obliged to borrow money in order to meet their rental payments.
"People stung by the bedroom tax are being forced to make difficult choices on which bills to pay and which essentials to go without," NHF chief executive David Orr said.
"They are living in fear that they will lose their homes and have resorted to borrowing from friends and family to try and get by.
"As we feared and warned, the bedroom tax is having a disastrous impact. The only solution is to abolish this policy which fails on every level."
Housing associations have complained that the introduction of the policy reversed years of instructions from Whitehall to build two- or three-bedroom homes in order to provide their tenants with flexibility.
Many registered social landlords responded by cutting back on the number of single-bedroom properties in their portfolio.
One housing association in the north-west, Contour Homes, has warned the change of government policy has left "an awful lot of people in a difficult position", managing director Judith Winterbourne said.
She has said providing all those who wish to downside into a smaller home will take a minimum of five years.
Figures from the Department for Work and Pensions (DWP) showed nearly half a million housing benefit claimants had been forced to accept a cutback in their payout.
"Tenants are taking action in a number of ways from moving and using home swap services to finding work or increasing earnings," a spokesperson said.
The government rubbished the NHF's report, saying it was based on a sample of cases from just 19 self-selected housing associations out of well over 1,000.
"It was simply wrong that the taxpayer was paying for social housing tenants to have spare bedrooms when around 300,000 people were living in overcrowded homes and around 1.7 million were on social housing waiting lists in England alone when the policy came into force," the spokesperson added.