Public outrage at Benefits Street is the result of Britain's worst welfare claimants becoming "ghettoised", Iain Duncan Smith has said.
The documentary has been widely criticised for painting a caricatured picture of those on benefits, but its narrative was tacitly accepted by a speech from the work and pensions secretary.
Duncan Smith used his address marking the tenth anniversary of the Centre for Social Justice thinktank to suggest that the controversy generated by the Channel 4 series is the result of middle-class ignorance.
"With income inequality under Labour the worst for a generation, whilst the middle-class majority were aware of the problems in poor communities, they remained largely unaware of the true nature of life on some of our estates," he said.
"We let these problems be ghettoised as though they were a different country. Even now, for the most part they remain out of sight – meaning people are shocked when they are confronted with a TV programme such as Benefits Street."
Duncan Smith's attempt to make work pay with the introduction of universal credit has been blighted by delays, but he will go on the offensive against Labour by accusing the opposition of "campaigns set to whip up unnecessary fear".
He added: "Our real success has been to reframe the argument – challenging a narrative beloved of the left … which focuses so exclusively on how much is being spent on welfare that it risks overlooking the real question: it is not about how much goes into the benefit system, but what difference it makes to people at the other end."
Earlier this week shadow work and pensions secretary Rachel Reeves raised eyebrows by announcing proposals to strip claimants of their benefits if they fail basic tests in English and maths.
The move, in her first major speech in the job, was seen as an attempt to demonstrate Labour can also be hardnosed on welfare issues.
"Only Labour, the party of work, can deliver the reforms needed to tackle the causes of rising benefit bills, so we can keep the costs of social security under control, while recognising responsibility and contribution," she said.