I haven’t gotten this in a work environment but I have had it in Mr R’s family. At get togethers, everyone is expected to kick in money to support relatives’ mission trips. I do not support their religion or trips in any way and I’ve always said no. Could I kick in $20 to keep the peace? Sure but then I’ll keep getting asked. So I say “no.”
As someone said upthread, “no” is a complete sentences. Adding details only gives them an in to try and guilt trip you into donating.