I need advice: Caught my boyfriend wearing my bra and panties!

And he was ummm... pleasuring himself!

This is just too weird! He's a really sweet guy, and I love him a lot, but I don't know if I can handle this.
I know this sort of thing doesn't necessarily mean he's gay, but the guy I fell in love with was A GUY! I'm not sure that I can respect anybody who gets his jollies from wearing my undies, and, frankly, I feel kind of violated. I had to throw out a really cute bra & panty set - I just couldn't see myself ever wearing them again.

Am I overreacting?

ED NOTE: See also "How do you feel about MANTIES?"