I would consider using any color and calling it the "neutral". It would simply be the unifying color in the quilt. Honestly, the traditional colors considered as neutrals aren't my favorites in the color world.
I've used yellow paint as a wall color and it worked as my neutral for the room decor.
Check the definition of neutral and apply the concept to your fabric colors. You can see how different ones can be justified. Others may disagree but it works for me. Go ahead and have fun with your quit design.