Is It Wrong To Bigoted Against Bigots?

Is it just as wrong to be bigoted against somebody because they hold oppressive beliefs as it is to be bigoted against someone for their race or gender? I thought of this during our discussion in class on Thursday. I personally don't think I would knowingly befriend someone who was violently misogynistic. Not only would I probably not have much in common with this person, but it would be dangerous for me to befriend this person. Similarly, I'm guessing most black people wouldn't want to be friends with a KKK member. Again, this is dangerous for them. Maybe we can be friends with people who hold prejudices against other groups (such as white nonracist people befriending KKK members, or men befriending misogynists), but I still don't like the idea of that. To me, being oppressive is not a flaw that I can look past. It's pretty much a dealbreaker and I don't think I'm a bigot because of that.