No, I would not switch thread colours with every fabric switch of the scrappy binding.
For a scrappy binding, I would likely choose a thread that coordinated with the backing I was stitching the binding too. Better still, would be if I could find a thread that coordinated with the backing as well as the assorted binding colours/fabrics.
Some hints that may help you minimize the stitches showing ...
Shorter stitches tend to be less visible.
A couple of years ago, I switched to using a ladder stitch when sewing on bindings.
Two reasons ... the bindings seemed to be stitched tighter to the quilt, and the stitches less visible.