When my oldest was beyond taking an afternoon nap, we still had 'quiet time' from 1-3 p.m. During this time she was expected to be in her room playing or reading quietly while her little sister napped and Mommy had some time to herself. Once both children were beyond napping, we kept quiet time, because it was good for them to have some down time during the day. it also taught them to entertain themselves and not need constant intervention.
My girls are 9 and 11 now, so official quiet time is long gone, but they've come to view their rooms as a place of solace and frequently head there themselves when they need to relax or get away during the day. I even notice them heading into the camper in the afternoons when we camp with other families, just to decompress a little bit.
I hope you'll give it a try. It has benefits for everyone involved.