I obviously don't know all of the issues, but I don't think there is a reason that you would have to choose. I think that relationships are made better when each partner has their own interests. That way, you two can be more than just the sum of your parts. If you did everything the same, you wouldn't have anything to talk about or learn from each other!
I think PP had it right when she suggested a "date night." If you and your FH set aside time each week to spend only with each other, then he will see that this fundraiser is not going to prevent you from spending time with him.
I really don't think you should have to give something up that is so important to you just because it isn't important to him.