Turn it inside out...the first fitting should be to pin the shoulders up, adjusting more near the neck. That should help alot.
I have a feeling that you didn't buy the pattern according to your measurments but by dress size, so....
you may need to adjust a dart, or make a dart to pull the fabric where it needs to be.
something just popped into my mind....you did use 5/8" seams didn't you? It is something that I forget when I go back and forth between quilting and sewing.