It's not just backpackers, but international students who're potentially being exploited in return for free place to stay.
The Ministry of Business, Innovation and Employment is now investigating the legality of board-for-wages operations in New Zealand, despite their popularity overseas.
First Union's migrant workers advocate Denis Maga said there needs to be more people on the ground to investigate the accommodation and horticultural industry.
"And we do believe that the inspections or investigations being conducted are just the tip of the iceberg."
Mr Maga said backpackers often don't speak out, because they're only in the country for a short time and international students are too busy focusing on obtaining another job to get a visa.
He said they're currently investigating a case involving international students and backpackers on an orchard in the North Island, who're working for $10 an hour in return for accommodation.
"Some of them having been living in caravans where there's no heater or a proper room to accommodate them so it's not even a good human dwelling."
The Backpacker Youth Tourism Association (BYATA) is welcoming any attempt to crack down on accommodation providers exploiting young travellers.
BYATA's Kirsty Phillips said although this is a common practice they'd be expecting the businesses involved to be paying the appropriate taxes and those working were entitled to do so legally.
"And we'd obviously be very disappointed to hear of operators that were employing staff outside of the law. We can't condone that kind of behaviour."
Ms Phillips said the industry needs to be giving these backpackers the best experience they can.