Fish & Game is accusing the dairy sector of using concerns over the chemical dicyandiamide, also known as DCD, to try to force the lowering of standards on nitrate pollution.
The substance is used to stop nitrogen seeping from fertiliser into waterways, but major suppliers have pulled it from sale after traces were found in milk.
Bryce Johnson of Fish & Game says some farmers are using the controversy to push for decreased pollution standards so they can continue to keep cattle stock levels high.
"The farmers that have been using nitrogen inhibitors are farming right at the limits of farming – that's why they use them," Mr Johnson told Firstline this morning.
"Because of that, they can have very high stocking rates and put on higher levels of fertiliser. The concern we have is that when that tool is taken away, they should therefore reduce their stocking rates and reduce their fertiliser application.
Mr Johnson says the Fonterra Shareholders' Council asked the Government and local councils to loosen restrictions on nitrogen runoff.
"We are saying hey, that's not on. If you don't have the tool available, you've got to back off in terms of the intensity of your farming and look after the environment – which after all, this industry has struggled to do for many years."
- VIDEO: Bryce Johnson talks to Rachel Smalley on Firstline
Mr Johnson says if farmers are forced to lower stock levels in order to keep under pollution limits, they shouldn't be compensated.
"[The farming industry" has been receiving a substantial environmental subsidy from the wider public for years now, so the fact they have now lost this particular tool, and it may be just momentarily, they should – at least in the interim – back off."
Rather than be compensated, he says farmers have a "duty" to "back off".
"The natural environment of New Zealand belongs to all of us. The water quality is important, and all New Zealanders want their rivers and streams to be safe for swimming and fishing, and gathering food. Same as the coast, which is where a lot of these waters end up, obviously.
"So I think there is a duty on this industry to back off."
Federated Farmers CEO Conor English said DCD contamination only had to be reported on a technicality, and that it has "no health harm effects unless you drink tonnes of the stuff".
Prime Minister John Key said yesterday a person would have to drink "a swimming pool full of milk" to experience any side effects of DCD.