I do sometimes...
Flannel, I *always* pre-wash.
If I'm using different types of fabric together, I usually pre-wash, but definitely wash the fabrics I know will shrink if I'm using fabric that generally doesn't, i.e cotton and fleece.
If it's likely to bleed, I wash.
If I'm feeling particularly lazy or it doesn't fit one of the above scenarios... I don't.