You may want to practice with someone to have some thoughts ready in your mind. I agree that No is a complete sentence. I wouldn't say I'm sorry, because that sounds like an apology.
"No, I can't do it anymore"
But you always have
I can't do it anymore
Why?
My schedule and finances don't permit.
But I can't afford anyone else
That is too bad and I can't do it anymore.
Or whatever words you think will roll off your tongue!