In many places, even on a warm summer day the pool may be uncomfortably cold. Lots of thermal inertia, and if it's shaded at all it may not get enough solar energy to warm up. Without a heater the swimming season can be very short indeed.

Also, it's pretty darn awesome to fire up the pool once a year in the winter, like for Christmas, and host a swimming-in-the-snow party. Wouldn't want to keep it heated, but for a couple days it's not too bad.