Open warfare has broken out after National criticised Speaker Trevor Mallard's handling of Parliament and criticised him for telling the media a National MP had described PM Jacinda Ardern as a "stupid little girl" – a comment it says has never been proven.

National's shadow Leader of the House Gerry Brownlee has written to the Speaker, saying National's confidence in him was "significantly shaken".

Brownlee questioned whether the Speaker had told media a National MP had called Ardern a "stupid little girl" in Parliament earlier this month, a comment that did not appear to be heard by anybody else and could not be heard on Parliament TV.

Mallard has now confirmed to the Herald he did tell a journalist he had heard that comment. "I was asked by a journalist what the comment was. I answered."

On May 9 when the "stupid little girl" comment was said to be made in Parliament, Mallard had called for a National MP to apologise for a "very sexist remark" but nobody owned up and Mallard would not repeat the comment when Brownlee asked what it was.

Newshub reported on 15 May that a source had told it the remark was "stupid little girl" but it could not be heard on Parliament TV.

Brownlee said if Mallard had passed on such information it was unacceptable.

No National MPs had heard the comment in question and all denied saying it. "If the source is the Speaker that is unacceptable. The Speaker cannot go briefing the media against the Opposition."

Brownlee has asked for an explanation by 2pm Thursday and said National would decide what other steps it would take after considering the response.

"We are not rushing to any position like [a motion of no confidence.]

"All Speakers at times get Parliament a little bit aggravated. It's a very difficult job. But when a Speaker gets to a position where it's abundantly clear that perhaps their neutrality is not as strong as it should be then I think that's a difficult situation."

It came on the same day National's deputy leader Paula Bennett walked out of Parliament after the Speaker docked five questions from National because he took offence at a comment made by Brownlee while Bennett was asking questions of the PM.

Bennett later said she had done so out of frustration at Mallard's system of docking questions when he was displeased with an MP because it was arbitrary and unpredictable and made it hard for the Opposition to do its job.

In his letter, Brownlee said National was concerned about Mallard's running of Parliament.

"Your arbitrary taking of questions from the Opposition and the disruption to proceedings are seriously limiting our ability to do our job of holding the Government to account."

Brownlee told media that system effectively meant the Speaker was determining the level of accountability the Government faced. "That's not acceptable."

National are not the only ones who have been upset by the Speaker. Police Minister Stuart Nash objected when he was likened to a "naughty child" by Mallard, another episode Brownlee referred to in his letter. And NZ First MP Shane Jones objected when the Speaker docked three questions for laughing.

Read Gerry Brownlee's letter below