I'm not the athletic type either, but when I started walking during my lunch and breaks each day at work, I noticed it coming off my stomach and hip areas. It took me about 4 months before I noticed, but it did happen...and I'm in my mid 40's, so my metabolism is not exactly awake anymore.
After I injured my leg, I wasn't able to do the daily walks and my size has increased back to where it was.
We use to walk 15min, 20min, and 15min a day, 5 days a week. It came out to about 1.5=2.0 miles per day in those 3 short bursts of time.