OH YES!!! The first time I ever tried to do a Drunkards Path. After two blocks I gave it up and never looked back. Do I regret letting it get the best of me? Not in the least. There's plenty of other patterns out there to make.
However, if you really really want to make this quilt, set the fabrics and pattern aside. Every few months drag it out and try again. One of these days it may just go together like a dream.