Do you have the correct spring? The old tensioner with the simple adjustment nut takes
this type of spring. I couldn't fine any info, but I think the numbered dial version is different. Make sure it goes into place in the notches on the stud, it should hold it in place. Choose a notch that gives it the right kind of spring for the job.