How often is a whistle blower used as a derogatory term? Can one say "I don't want to blow the whistle, I am not this kind of a man."
This is an interesting question because I think it's going to hit a bias in ELL.SE user demographics.
The expression to blow the whistle means to expose corruption, that is, conspiracy or extortion by means of state power or state-backed corporate power.
So what you feel about whistleblowers has an awful lot to do with how you feel about state power.
ELL.SE users draw very heavily from the tech trades, and, at least in the US, that is a population that tends toward having negative feelings about state power, or who are at least very touchy about how state power is used. I count myself in this cohort. On top of that, the Snowden revelations have primed a lot of people already skeptical about abuses of power to feel very negatively indeed about it. Among this population whistle blowing is seen as something heroic.
It can be hard for those of us in this demographic to realize there is also a sizable demographic in the US which is very authoritarian in both personality and culture. They actually think authority and systems of authority are good things, and that people who challenge authority are immoral. For them whistleblower is a negative term and blowing the whistle means something very like betrayal. At best, if a whistleblower is vindicated subsequently, people with such sympathies to authority tend to see the whistleblower as, at best, a "necessary evil", and someone of low character who happened to be right -- like a criminal who turned on other criminals. These are the people saying, "Even if Snowden was right, he had no right to do what he did."
(This is sort of off topic, but if you're interested in this sort of thing in US culture, you may want to check on The Authoritarians by Altmeyer, "Red Family, Blue Family" by Muder, Red Families v. Blue Families by Cahn and Carbone, and the work of Jonathan Haidt on morality across cultures (somebody else's paraphrase here, his TED talk here)).
As to the valence of blow the whistle in any other English-speaking culture, I can't speak.