By Hamish Cardwell of RNZ

A pair of little blue penguins that were removed to the harbour by police have apparently returned to their refuge under a Wellington sushi shop.

The penguins were first spotted at dawn outside the Wellington railway station and took shelter under the nearby shop, Sushi Bi.

Police said the "waddling vagrants were removed from their sushi stand refuge earlier today by Constable John Zhu" after being told they were there at about 6.30am.

The pair were temporarily detained by Wellington police before being released back into Wellington Harbour, they said.

However, the owners of the shop tonight told RNZ the birds had returned to the their refuge.

On Saturday evening a little blue penguin was found close by on Featherston Street, and was also returned to the sea.

The Department of Conservation said they were monitoring the site as this breed of penguin would likely try to return to the site again and again, despite being moved on.

DOC Wellington operations manager Jack Mace said the birds were quite common in Wellington harbour, but it was the first time he had heard of them making themselves at home at the station.

He said at this time of year the penguins were pairing up and looking for sites where they could lay eggs.

Mr Mace said people should keep their distance from the penguins, which can have a nasty bite.

Wini Morris works at the sushi shop where the penguins hid.

She said she heard them making a cooing, humming sound, and that the birds likely hid near the grills beneath the shop - where it was nice and warm.

"It's pretty insane the idea that some penguins are camping out under your shop.

"But it's adorable I think - they're probably terrified but it's adorable."

The little blue penguins conservation status is considered to be at risk to declining.

- RNZ