Sometimes I am given fat quarters and whole cloth to work with, a wished pattern stated....sometimes a quilt pattern will be remembered, and a beautiful quilt will result.
I do both...start with fabric.........start with a pattern. I learned originally to start with a pattern first with a DVD. Some friends talked about colors, which I purchased. Then, I worked with fabric, and looked at patterns. I prefer to find the pattern first, the find the fabric. Both methods have produced great quilts. There is no "right" way.