Duncan Smith tells Tory conference ‘quiet man is turning up the volume’