Trevor Mallard has until 2pm to respond to a letter from Shadow Leader of the House Gerry Brownlee.

National is upset at the way the Speaker has handled the fuss about whether someone in the debating chamber called the Prime Minister a stupid little girl.

Brownlee says he asked Mallard what remark he was concerned about, and he refused to say, but then told the media what it was.

"None of the recordings indicate that that was actually said, all of our guys have indicated that they did not say it and no one heard it."

He is accusing Mallard of pushing the sexist remark story and told Larry Williams he wants to know if Trevor Mallard has been briefing the media on the issue.

"Obviously in the process of him considering that he'd reflect on the wisdom of a speaker actually doing that. Beyond that I wouldn't predict anything that might happen."

Brownlee says the Speaker has to be very neutral and can't start briefing the media against the Opposition.

He also says Mallard has taken questions off the National Party which is unacceptable because he is determining the level of scrutiny the Government comes under.

LISTEN ABOVE AS GERRY BROWNLEE TALKS TO LARRY WILLIAMS