I have to really like the fabrics together before I start the quilt. It's a gut feeling... If you are uneasy about throwing your pieces all together, I'd trust that feeling. You won't like the finished quilt, either.
I would wait and start collecting more fabric... looking for things to go with what you have. Get a feel for what you want to do with each type. It's worth it to wait to collect enough of the right pieces to go together. That's the artistry of quilting.