Today's report by the justice select committee makes for despairing reading. They paint a picture of a prison system lost in crisis and a leadership unable to grasp what is happening or come up with plans which offer the slightest reassurance for the future. The authorities say it's the fault of legal highs and prison campaigners say it's the fault of over-stuffed prisons. They’re both right. But only a pragmatic liberal response to both issues will help.
You can almost hear the dismay in MPs' voices as they look at the stats. Assaults are soaring – up 20% in the second half of 2015 from where they were in the first half. Assaults against staff are up nearly as much. A prison officer working for Serco, Lorraine Barwell, died last July after being attacked by a prisoner.The number of suicides in prison rose from 79 in 2014 to 100 in 2015. National tactical response teams sent out to deal with serious disorder, including riots and hostage taking, are now being called out an "unprecedented" 30 to 40 times a month.
We've been watching this get worse and worse for years now. Under Chris Grayling, the Ministry of Justice put its hands over its ears and said there was no prison crisis. Under Michael Gove they have admitted it, but reform proposals have been disappointingly minor and slow in coming.
In March last year, the justice committee said "all available indicators" pointed to rapid deterioration.
Michael Spurr, chief executive of the National Offender Management Service (Noms) blamed drugs. "The issue of NPS [new psychoactive substances or legal highs] escalated in an unprecedented fashion over the period from early 2014 through this year," he told the committee. "I think we are getting on top of it now, but we are not yet at that position."
The Prison Officers' Association blames a low staff-to-inmate ratio. Ministers promised a new recruitment process was going "full throttle" after the unwise decision to start cutting them back at the formation of the coalition. But it seems impossible to retain staff in the depressing, dangerous world of the prison estate. They took on 2,250 extra prison officers but lost almost as many in the same period. The net increase was just 440.
So who is right? They both are. Many of the legal highs taken in prison are incredibly dangerous. And there are simply too many prisoners, in cramped conditions, without staff to take them on activities. Lock up men for 23-hours a day without structure or anything to aspire to and they will fall into violence and despair.
The problem is that the solution to both problems is liberal, and prison policy is rarely open to liberal solutions.
The legal highs problem in prison is self-created. It began when mandatory drug tests were introduced in 1996. The most popular drug in prison is cannabis, because it is very good at helping people deal with boredom. Many prison governors didn't mind. It created relatively docile individuals. To a certain extent, an informal don't-ask-don't-tell policy was in place. Then the drug tests came in.
Cannabis is testable for 28 days or so after use. Heroin is testable for just two. So the first effect of the tests was to push prisoners towards a much harder drug. Then came Black Mamba and Spice, so-called synthetic cannabis products which were really nothing of the sort. They're just chemicals sprayed on to plants. It's incredibly dangerous and often prompts violent, unpredictable behaviour from prisoners, along with fits and often death. They seem to have been a factor in 19 prisoner deaths between 2012 and 2014.
Ministers are hoping a new range of drug tests will be able to include these types of drugs. Perhaps they will, but the usual cause-and-effect game of drug prohibition will surely follow: new drugs will come on the market designed to side-step the tests.
Efforts to keep these drugs out of prison should obviously continue, but the quickest, easiest way to stop prisoners taking them is to end mandatory drug testing, or at the very least to remove cannabis assessments from the test. The Daily Mail might not like it, but a prisoner on cannabis is a far less severe threat to himself, those around him, and the maintenance of order in a prison than someone on Spice or Black Mamba.
The second liberal solution is the old-faithful, the one which never goes away because no-one ever has the bravery to act upon it: reducing the size of the prison estate. The way to do this is to confess that prison does not work, and it works least of all for those serving short sentences. Three out of five short-term prisoners will reoffend within a year. There is almost no evidential case for custodial solutions to crime.
We have to get over this tabloid mentality about punishment. But Gove, who seems admiringly open-minded and brave in other regards, isn't prepared to move on this.
"I would not over-fixate on numbers," he told the committee. "There is a danger of being paralysed by the thought that we cannot make any change until we reduce the population. That is the argument put by some. I think that we should make changes within the current population."
It's a disingenuous argument. This isn't about taking people already in prison and reducing their sentences, it's about a more ambitious forward-looking programme which discourages custodial sentences for nonviolent offenders and trades instead in predominantly community sentences. It's about getting that inmate-to-officer ratio down so they can pay proper attention to the people in their care, have availability to take them to the prison library or gym, put them on educational and vocational courses, and encourage continued contact with family – all the things we know reduce reoffending.
But reducing sentences and discouraging custodial responses are again liberal solutions. The lock-em-up brigade in the tabloid press and on the Tory backbenches won't like it. So they are considered off-the-table. Instead, we'll stick to the policies which created this crisis in the first place.
Ian Dunt is the editor of Politics.co.uk
The opinions in politics.co.uk's Comment and Analysis section are those of the author and are no reflection of the views of the website or its owners