If you put the needle in backwards it will do just what you described. Can you post a picture of the machine so we can help you orient the needle? Generally if the bobbin is in the front the needle will insert flat side to the back of the machine. If the bobbin is on the left the needle will install with the flat side to the right. If it is a Singer anything goes. Not really but that might be enough info for now.