You would pin in 2 different situations, when piecing and when sandwiching. I think you are asking about pinning when piecing? Sally Collins, who wrote a couple books on precision machine piecing recommends parallel to the edge. I do both ways, depending on what I'm pinning. I usually pin seam intersections parallel to the edge, but if I'm pinning a long strip, like a border, I pin perpendicular, with the point towards the edge. I try to keep the point from extending past the edge.
In either case, I hear my mom's voice: "pin on the table!". In other words, if at all possible, align your pieces on your work surface, hold the pin as parallel to the fabric as possible, and slide it in. Don't pin in the air just holding it in your hands, stabbing the pin straight into the fabric, which can cause the fabric to move when you weave the pin thru. This is more of a garment construction technique. If you watch a garment construction video, you can see what I mean.
Also, you want to use the thinnest pins you can find.
Phew!