With the needle plate off, turn the hand wheel and watch that "finger".
What does it do? Does it spin around, or does it stay mostly in the same spot? What happens if you grab it gently with your fingers (basically, you act as a needleplate as far as the finger is concerned)
while trying to turn the handwheel? Does it try to rotate out of your fingers, or does it stay put in your fingers?
Are you having trouble taking the bobbin case out, or the gib screw and the bobbin case base out? (The photos on the link I gave you will show you what each of these parts are.) If you can't get the bobbin case out, that's a completely different issue.
BTW, be very careful with the gib screw. The second they're removed from a machine, they tend to grow wings and disappear.