Labour has failed to win back the formerly safe seat of Blaenau Gwent, as it again fell to independents.
Dai Davies, the agent of the constituency's former MP, Peter Law, whose death prompted the by-election, comfortably secured more votes than Labour's Owen Smith.
In a separate by-election in Bromley and Chislehurst Labour finished fourth, but the Conservatives' narrow margin of victory in this safe seat will concern David Cameron.
London Assembly member Bob Neill won the seat for the Conservatives, but with a much reduced majority of just 633 votes from the Liberal Democrats, compared to 13,342 in the 2005 general election.
Lib Dem leader Menzies Campbell described the poll as a "stupendous result" for the party.
In Blaenau Gwent, Labour also failed to win back its Welsh Assembly seat, which fell to Peter Law's widow Trish Law.
Until 2005, the Westminster seat was considered the safest in Wales. But Labour's decision to use all-women shortlists angered Mr Law and after running against them in the general election, he overturned a majority of 19,000 to win the seat.
The 58-year-old died in April after a year-long battle with a brain tumour prompting yesterday's by-election.
The victorious Mr Davies said: "The political revolution is starting in Blaenau Gwent tonight. Political parties take note. You take people for granted at your peril."
Labour party chairwoman Hazel Blairs remained upbeat that Labour had cut the independent majority from 9,000 to 2,500.
"Blaenau Gwent was pretty unique circumstances. There was almost a family feud down there because of what happened in the last set of elections. Over the last year people have come back to us in significant numbers," she told Today.
However, she described the Bromely by-election as "an absolute disaster" for David Cameron: "If we were seeing a resurgent Tory party he ought to have romped home there and he didn't."
Mr Cameron largely stayed away from the Bromley and Chislehurst by-election after local activists snubbed his 'A-list' of MPs and selected Bob Neill.
He secured 11,621 votes and will succeed Eric Forth who died in May after holding the seat since its creation in 1997.
Ben Abbots, who won 10,988 votes for the Liberal Democrats, said the closeness of the result showed his party could "make strong advances at the expense of both other parties".
Leader Menzies Campbell said: "This is a stupendous result for the Liberal Democrats in one of the safest Conservative seats in the country. It shows that there is no confidence in Cameron's Tories in the Conservative heartlands."
Tory party chairman Francis Maude admitted the result was a "wake-up call" for his party and it had "a long way to go".
"David [Cameron] has been rightly driving a process of change in the party and the simple truth, from this election result, is that we have to drive that change faster, wider and deeper," he told the same programme.
Blaenau Gwent by-election results
|Steffan Lewis||Plaid Cymru||1,755|
|Amy Kitcher||Lib Dem||1,477|
|Alan Hope||Official Monster Raving Loony Party||318|
Bromley and Chislehurst by-election result
|Ben Abbots||Lib Dem||10,988|
|Paul Winnett||National Front||476|
|Steve Uncles||English Democrats||212|
|Anney Belsey||Money Reform Party||33|