You fell into the situation that I love to be in. I love panels, but not always crazy about the coordinating fabric. I think it makes it to much like everybody else's quilt, personal opinion I know. I like to be a little different in that way. I might get one or two key fabrics in the same line, that is if I want a boarder or backing, but that is about all. I will then search for the "right" fabric to go with it. Once I have what I think goes with the panel, I have fun creating the layout that I like. Have fun looking for fabric that will work with your panel and enjoy the process you have gotten into. It is a wonderful thing!
Can't wait to see what you come up with.