Welcome from NJ.
What brand is your new machine? My Viking feet just kind of 'pop' off by gently tugging forward. My old White machine needed a bit more muscle when 'popping' the foot off while raising the presser foot lever. My vintage Singers have the thumb screw that Rodney mentions. Perhaps if we know which machine you have we can better help you.