The new prime minister should only come from the Leave camp, former Conservative leader Iain Duncan Smith has said.
He told the BBC’s Andrew Marr show that it would be “very difficult” for a public who voted to leave the EU to have a leader who had opposed this.
Prime Minister David Cameron announced on Friday that he will step down by October.
He had urged the country to vote Remain, but was defeated by 52% to 48%.
In other developments, potential leadership contender Boris Johnson has been meeting with Conservative MPs Jake Berry and Ben Wallace at his home.
The BBC understands he also met with Justice Secretary and fellow Leave campaigner Michael Gove.
- Follow the latest developments on our live page
- Who might succeed David Cameron?
- What next for the Conservative Party?
- Brexit: What happens now?
“The government itself now had a view… which was to remain, and so now we need to change that position and actually deliver on this very clear mandate from the British people,” said Mr Duncan Smith, who campaigned for Leave.
He said he was “incredibly sad” Mr Cameron had chosen to go as he wanted him to “remain to help stabilise the situation and get us moving”.
“He’s done a pretty good job throughout all the way, in some tough circumstances, particularly during the coalition,” Mr Duncan Smith said.
“But I do think it would be very, very difficult for the public who have voted for leaving the European Union to find that they then had a prime minister who actually was opposed to leaving the European Union.
“So I think it is quite clear that – at least the leadership end of it – but I would like all the others, Remain and us, to come together.”
Mr Duncan Smith also ruled himself out of the future Conservative leadership contest.
He said the House of Commons had “an obligation to deliver on the British people’s verdict” and progress with leaving the EU.
“The Conservative party is in a majority in the House of Commons and my view about this is there are also a number of Labour MPs who are very clear they want this to be delivered on,” he said.
When questioned about the Leave campaign’s assertion that leaving the EU would free up £350m a week extra to spend on the NHS, Mr Duncan Smith said the NHS would receive “the lion’s share”.
“It is not a promise broken, I never said that during the course of the election,” he said.
“What I said was we would be able to spend the lion’s share of that money, now the government is now able to spend, so people can say that there is more money available now for the NHS, categorically more, which is what’s required and that’s the key point.”
Asked if Ukip leader Nigel Farage would be involved in the cross-party exit negotiations with Brussels, Mr Duncan Smith suggested the party’s sole MP Douglas Carswell would be consulted.
“There is a Ukip MP who is part of that process in a parliamentary sense, and I’m very happy to discuss… with Ukip about what their expectations are, but the government itself, right now, actually dictates how this will happen,” he said.
‘Control of borders’
And on immigration, he said the Conservative government had to honour its manifesto pledge to bring net migration – the difference between the number of people coming to the UK for at least a year and those leaving – down to below 100,000.
Mr Duncan Smith said: “I said consistently throughout that we need to get control of our borders and the only way to do that is to leave the European Union…
“Once we do that we are in a much stronger position to achieve that objective of bringing down migration to tens of thousands within this Parliament – I will stand by that.”