I sell on ebay. I am super fanatical about the way I package because the mail DOES toss things around.
If someone reported it to me, as you did, I would refund, no questions asked. That's the chance you take when you sell.
However, at most leave a neutral. Those red dots can really
be damaging, unless you really feel it warrants it and it doesn't sound like you do.