First Secretary of State Damian Green said one of the reasons for Brexit was the perception that too many migrants had been allowed to enter the country – and “we should respect that”.
The Tories pledged to reduce net migration to the tens of thousands in 2010, but the goal has never been met.
This week, Scottish Conservative leader Ruth Davidson rekindled the debate by opening questioning that target and calling for a “mature debate” on the issue.
Ms Davidson has also argued that overseas students should be removed from net migration figures, contradicting Theresa May on the issue.
Mr Green, speaking from Scotland, said a “sustainable level” of immigration would be in “the tens of thousands”.
“It is clear that one of the forces behind the Brexit vote was a feeling in some parts of the UK that immigration had been allowed to be too high for too long, and I think we should respect that,” he added.
“We’re not going to stop immigration overnight, nobody has ever suggested that, that’s never been UK Government policy and it won’t be UK Government policy,” Mr Green said during his visit to Edinburgh.
“Immigration policy is always a balancing act between forces pushing in different directions.”
In the wake of the Brexit vote, the Scottish Government has been pressing for power over immigration to be devolved to Holyrood.
But Mr Green dismissed the suggestion, saying: “Immigration has always been a reserved power and that is for obvious practical reasons, that nobody wants any kind of border control inside the UK.
“I can’t see that changing.”