I'm assuming something is pushing the movement of the needle or needle bar off it's position since it's not forming stitches. These buttonholers should not do that, but for some reson it seems like yours might do.
Check if the needle is inserted the correct way; further check for grime and dust behind the needle clamp where the shank of the needle goes. Both of these will give erratic stitch formation.
If you use the very soft light weight thread meant for overlocking it can make stitch formation act up. Test sewing with sew-all polyester or medium weight cotton is often the easiest.
I'm guessing it's something basic that needs to be checked, cleaned or threaded correctly. Make sure the bottonholer is oiled and moves freely.