I'm sure you had great fun on your trip, but for most people I believe that going with your workmates would make it much, much harder to truly disconnect.
Sure, maybe you aren't sitting around with laptops and whatnot, but I'm sure all sorts of work talk will pop up.
On the other hand, assuming this supplements rather than replaces a normal vacation, it could just be a fun and relaxed way of brainstorming. After all, at an interesting company, chances are work talk will pop up because people genuinely enjoy it. Just talking about work a bit does not ruin a vacation!
This reminds me of what one of my professors talked about recently: a skiing trip is great because you can enjoy yourself and talk about research on the ski lift. So taking your entire research group on a skiing trip would be a nice compromise between taking a break and having time to talk and think about what you're working on.
I actually think it sounds great. Partly because I love skiing and partly because I also love talking about CS.
Sure, maybe you aren't sitting around with laptops and whatnot, but I'm sure all sorts of work talk will pop up.