UPDATED: 2.10PM The Government is toying with the idea of a land tax to bring down foreign ownership, but it could also apply to New Zealanders.

If brought in, the tax would be used to restrict overseas buyers of residential property, but the Prime Minister has warned it would also apply to New Zealanders who've been out of the country for more than three years.

New Zealand First leader Winston Peters said there are obvious problems with that idea.

"I'm a person in the military, I'm an ambassador or a high commissioner working offshore, or someone engaged in the service of my country - why on earth would you apply that tax to them?"

Mr Peters told Jack Tame it doesn't make sense to tax a New Zealander that's gone off-shore for a few years.

LISTEN ABOVE: Winston Peters talks to Jack Tame

"The problem is off-shore foreigners are buying into New Zealand land and houses, that's the problem. It's not New Zealanders buying land and going off-shore for ten years."

Mr Peters is accusing the John Key of trying to manipulate the public by hinting the move could also hit Kiwis. He said John Key has never wanted to restrict foreign ownership, and this is just another trick.

He said Mr Key is trying to give the impression of considering options, while putting people off the idea of controlling foreign ownership.

"He's trying to give it the colour of balance so that people might be dissuaded as to land tax or positive action against foreign buying be a good idea."

Meanwhile, Labour is accusing the Prime Minister of flip flopping, after signalling that he's considering the restrictions on foreign house buyers.

The party's housing spokesperson Phil Twyford said it's a sure sign that John Key has painted himself into a corner with his lack of action on the issue.

"Standing on the side of speculators and the wealthy, who are making a killing out of the housing crisis, and in the meantime he's been prepared to completely cut loose 'generation rent'."

Mr Twyford said the change of heart is odd, and makes him suspicious that the Chinese President has something to do with it.

"Because the Chinese President has expressed concern on a number of occasions that corrupt Chinese money is being funnelled into the New Zealand real estate market as a way of money laundering."

ACT leader David Seymour said a tax won't fix the shortage of houses.

"The fundamental problem with New Zealand's housing market is that we have not beat the 1974 record for home building in any year since then."

Mr Seymour said the Government campaigned on a promise of no new taxes, but if brought in, this would be the third this term.

"One new tax was a bit naughty, two new taxes was bad behaviour, but three new taxes is a total betrayal of people who voted for a centre-right Government."