Will be fine!!! I don't use ginger in my pumpkin pies (gives me indegestion)....so I don't use it! By the time everyone gets to the pie, they'll be so full they won't notice...LOL
I also don't like nutmeg in my apple pies so i don't use it - people love it.