I think that might be the problem: those three nubs on the washer are what the screw stops against when you loosen the knob. If the washer's turned the wrong way, you won't be able to turn it far enough. You shouldn't have to remove the screw every time.