I know of one major team in my field. I tried to passively discuss it, was graduating and I was so naive. Not a good idea! At the time I had a few papers in review.

All my work got rejected (everything was being accepted before, and the rejected batch was a full level higher quality). So yes, it did set back my career for two and a half years if I have to approximate. I am probably never submitting to certain publications ever again to avoid having them as reviewers. I am gradually moving away from their field entirely. I was so short on publication then that I couldnt even move away to nearby fields, having accomplished vengeful rejections in my field. These days I wish I had done a second PhD in a different field, just to get away from them! I'd have had a much easier career path.

This is by no means an exaggeration, I am actually downplaying what happened.