The coalition is to legislate to prevent the automatic early release of serious offenders.
Justice secretary Chris Grayling said in future those guilty of the worst offences would have to convince an independent parole board of good behaviour and rehabilitation to justify release before completing their custodial term.
He had used his speech to the Conservative party conference in Manchester earlier this week to point out the New Labour government "actually went so far as to enshrine in law the automatic release for all prisoners given fixed sentences".
Grayling told delegates: "As I've said before, I don't like the concept of automatic early release. I don't like the idea that from the off, a serious offender knows he's only going to serve half his sentence. Looking at that issue is my next task."
He has now release details of his plan to tackle the issue, following on from the coalition's move last year to made offenders serve two-thirds of their sentence before they become eligible for release.
"It's outrageous that offenders who commit some truly horrific crimes in this country are automatically released from prison halfway through their custodial sentence, regardless of their behaviour, attitude and engagement in their own rehabilitation," Grayling told the Telegraph newspaper.
"This government is on the side of people who play by the rules and want to get on. We need to teach criminals a lesson; you will be punished for your crime and you must earn your release, it is not an automatic right."
The justice secretary is proposing that those who have raped or sexually assaulted a child, terrorists and criminals guilty of offences like grievous bodily harm will lose the right to an early release.
Around 600 criminals behind bars are expected to be affected by the change.
Labour accused the government of distracting "rhetoric" drawing attention from its decision to end the practice of jailing offenders indefinitely, however.
Shadow justice secretary Sadiq Khan said there had been a "weakening of public protection against the most serious and violent offenders" with the demise of indeterminate imprisonment for public protection.
The measure had allowed judges to detain criminals until they were no longer deemed a threat, but the coalition announced it was scrapping them in 2011.
"Under Labour, judges and magistrates will set out in plain English a clear minimum and maximum time that will be served in prison, so victims of crime know how long sentences will be," Khan added.
Grayling's reforms, part of the 'rehabilitation revolution' initially championed by Ken Clarke, have seen a gradual tightening of criminal justice policy over the last year.
Earlier this week he used his speech in Manchester to announce simple cautions for serious criminal offences would be banned outright.
Grayling has already legislated to ensure those who receive two serious violent or sexual offences would receive an automatic life sentence.
A toughening of community sentences will become law by December.