When it comes to tote bags I try and err in the direction of over-construction. I use felled seams, and if I don't make the handles go all the way under the bag, I extend them at least an inch into the bag and secure with boxed X stitching: Sew a square just smaller than the width of the handles, then sew an "X" across the box.
I made a bag for a friend, actually, it was just a fabric gift bag, embroidered with her initials to hold her real gift, made out of cotton. Since it was just the gift wrapping, I didn't make it too strong. A few months later, she brought it to me to fix because the side seam had come loose. I almost fainted when she told me she was carrying books AND her laptop in it!