It's a gorgeous collection. Wow. I'd audition white, black and grey tone-on-tones and see which one speaks to you. The grey might be sophisticated and soft so as to showcase the fabrics, or it might dull the colors. Black can make colors pop, but sometimes it's too harsh. White is pure, but sometimes has the dulling effect as well. So that's why I hesitate to commit without seeing some of your auditions.
I agree to not use the solid black or white blocks. Your eye would stop on them and be distracted.