I have never had any of my cats attack me like that but I did have two in my lifetime that attacked others unprovoked. I used to have littermates that were indoor cats (fixed male and female). They were allowed supervised outside time and during one of those times the female just spontaniously attacked the male. I thought it was because of a scent thing and brought them both inside. Rubbed them each down with the same towel to comingle scents. The behavior persisted. After a few days of these unprovoked violent attacks I took her to the vet and they ended up putting her on a mild sedative for about a week (kitty valium??). That seemed to solve the issue.
Many years later I had a rescue cat. DH and I were rough-housing around and he started tickling me. I started laughing and screaming and the cat attacked him! Turned out it was, unusual for a cat, protective behavior. We were able to duplicate it anytime we wanted. Another time he attacked our 90lb dog when she was yelping cuz I accidently stepped on her. Talk about adding insult to injury. He was also fixed.
How old are your cats? I am wondering if neutering may help to curb the aggresiveness. If the behavior persists you may have to get him on a mild sedative just to break the cycle.