I usually start with the pattern and then change it ---- I always do that --- it is a habit I developed when I used to crochet a lot and I just can't seem to help myself. LOL I think I have only followed one pattern all the way and it is a UFO right now. I've made up and improvised almost every project I have done.
You are not losing brain cells ---- you are developing artistic habits. Have fun with it!