I am a rescuer/foster for felines (of over 14 years). What you did was right, period. His response was out of line and hateful.
His video should be plenty of proof as to the identity of the littering cat(s). Although it is likely the camera is a fake and only has a red light that works. Otherwise he must be made of money to afford such devices or a techy.
IF I were your neighbor (and this is truly hypothetical - as my pets are fully "contained" on my property) and I was worried about my neighbor trapping my cat, I would be eager to share pics of my cats with the trapping neighbor so they knew which ones were mine.
And as for you . . . keep (humanely) trapping the cats that trespass and if you want to be kind to your neighbor, take the trapped cat to him each time and say, "Found this guy in my trap while I was seeking out neighborhood strays." But no apology needed. After a few days of returning his cats, he might need a new strategy as his cats are obviously not reading the sign that clearly says they should stay out of the cat nappers yard (read sarcasm).
He (and his cat) are lucky I am not their neighbor. I would have taken the trapped cat to a feral TNR (Trap-Neuter-Release) program affiliated veterinarian for surgery and ear notching! Imagine his surprise when his cat returns home a day later and has it’s belly shaved and unique ear!