Australian Immigration Minister Peter Dutton has addressed the stoush between himself and Justice Minister Andrew Little, saying he makes no apology for deporting criminals and will continue to do so.

He did not directly address an issue central to Little's criticism this week of Australia's deportation policy, the good character test under which people can be deported arbitrarily, which Little called the grounds "amorphous and tenuous".

Speaking at a press conference in Australia today, Dutton was asked about Little's comments.

"I'm no friend of criminals, let's be very clear about it ... and I'm going to continue to do it, and the law is very clear.

"If you're not an Australian citizen and you've committed an offence which puts you outside the conditions of your visa or puts you in breach of the Migration Act, you stand to have your visa cancelled. I don't make any apology for that," Dutton said.

Little, who appeared on the ABC's Foreign Correspondent programme this week along with Foreign Affairs and acting Prime Minister Winston Peters, was critical of Australia's deportation policies, saying there appeared to be a "venal, political strain" to them and "certainly not consistent with any humanitarian ideals that I thought both countries once shared".

Speaking to Australian radio yesterday, Dutton said Little might want to "reflect a little more" on the transtasman relationship.

"There's a lot that we do for New Zealand ... We're a big land mass between them and boats coming from Indonesia and Southeast Asia," Dutton said.

"New Zealand don't contribute really anything to the defence effort that we've got where we're trying to surveil boats that might be on their way to New Zealand.

"So I hope that Andrew Little reflects a little more on the relationship between Australia and New Zealand where we do a lot of the heavy lifting. We intercept boats which stop them from making their way to New Zealand.

"There's a lot to this relationship and I was really disappointed in Mr Little's comments during the week. I hope that he doesn't repeat them," Dutton said.

Little said yesterday he wasn't sure which comments Dutton was referring to.

"I think it's the general tone that I've, as [Foreign Minister] Winston Peters has on the issue of the 17-year-old, called out Australia on what looks like a practice that doesn't accord with basic human rights.

"My comment about venality was about the fact that the politics in Australia suggest some widespread attraction to treating people from other countries ... in this sort of this way.

"I think there are plenty of politicians in Australia who hide behind that political calculus and I think that political calculus is venal."

He said deportations on "so-called" good character grounds were concerning "because that is such as amorphous tenuous kind of provision on which to deport people, and an area where I think the two countries would benefit from at least trying to get a better understanding".

In his interview, Dutton also expressed frustration at not being able to deport New Zealanders with dual citizenship.

"In limited circumstances, I can strip citizenship from somebody who's been naturalised and the AAT [Administrative Appeals Tribunal] can overturn my decision in that regard. I think it's a stupid situation, I think it's a deficiency in the law. I've tried to get it through the Senate, Labor won't support the change," Dutton said.

The AAT last week heard an appeal the 17-year-old New Zealander who had been held in an adult immigration detention centre in Melbourne after his visa was revoked. He has been in Australia with his family for around seven years.

He was released this week but his freedom could be short-lived as Dutton can still overrule the AAT's decision. He is taking legal advice on his options.

The youth, who has a string of convictions, is back with his family in New South Wales in the meantime.

Peters also weighed in on the case recently, saying Australia was breaching the UN Convention on the Rights of the Child and telling it to honour its obligations.