It's not miniscule, and actually worse when packed in oil because the oil-soluble fatty acids diffuse into the oil.

But even when packed in water, or just canned+cooked in own juices on the higher end, a substantial portion of the fats and oils escape the flesh and float to the surface during cooking. Draining the can discards fatty acids, it can be a surprisingly substantial portion.