When it was his turn to comment, Peters said he was the victim of a conspiracy, and told Joyce, "too many of your people are talking to me" because they don't like the dirty politics.
When someone laughed in the audience, the Northland MP turned to them.
"You can laugh about privacy. But one day madam, they will come for you - and when they do, you won't be so happy. Privacy is a part of our society. We are a great democracy.
"They broke the law and they are not going to get away with it."
Peters said he had been targeted by National "because they think I am weak to deal with."
"They are doing that because they know when we want a policy change, we will get it."
All parties apart from the Greens attacked Labour's failure to say definitively it would or would not introduce a capital gains tax or how much its water tax would be.
Steven Joyce accused Labour's Grant Robertson of dealing in "weasel words".
Peters said people were entitled to know before the election what taxation rates were, not after it.
"What are your water charges going to be before the election, not after it, and there's a serious danger that a lot of people in rural and provincial New Zealand are going to be wiped out from some of the conversations I hear if it becomes a reality."
Peters has pointed his finger at the National Party as the possible leak to the media of the news he had to repay overpayments for his superannuation since 2010, saying it was an attempt to destroy NZ First.
That followed revelations that two ministers - Paula Bennett and Anne Tolley - were briefed on the overpayments by Government department heads under a "no surprises" policy. National leader Bill English said he did not believe the leak had come from National and was assured by Bennett and Tolley they had not passed on the information.
Ministerial Services are now looking into the handling of the information by the ministerial offices involved. Both the Ministry of Social Development and Inland Revenue are also now investigating to ensure the leak was not by a public servant.