I used to sleep in a lightweight sleeping bag that I rolled and left at the head of the bed during the day. Maybe that's a better solution for you?
Now I just wear a lot of clothes to bed.
Not exactly sexy, but functional, and a lot less off-putting than my icy feet on his legs :)