When I used safety pins I used size 0. Higher numbers are thicker as well as longer. However, there is also great variability in pin quality depending on manufacturer. Some are very sharp and smooth; others are more blunt and/or not as smooth. I agree with throwing away the 3's.
if you use anything, make it Fray Block instead of Fray Check. It dries in soft rather than hard. But I would wait and see what happens after washing. You may not need it.