To be fair, the analog output of the RPi delivers a crappy low-pass filtered 11-bit PWM signal. Even the cheapest USB audio adapter will give you a better signal. (Personally, I have been quite happy with https://www.pollin.at/p/logilink-usb-2-0-5-1-audiocontroller...)
Yeah, you're not wrong. I more disappointment with the general trend rather than this specific instance; phones and tablets are where I miss it the most.
With Raspberry Pi's specifically, although I have used the headphones jack to connect speakers, I generally use either HDMI audio, or else I just don't have any audio connected.