Yup! The details of Linux networking API are often surprising.
I think if it's possible to change the ping algo to reduce the potential for wall clock change affecting the measurement.
For example, instead of using SO_TIMESTAMP to measure wall clock diffed with the send time, maybe it's possible to grab the wall clock just before recv() and use wall clock to measure the event-loop time only (which is way shorter than the ping rtt).
If you want more of this stuff, follow my twitter: https://twitter.com/majek04/status/1687023191506530304
Lately it seems I'm spending way too much time on ping. This is from last month:
https://blog.cloudflare.com/the-day-my-ping-took-countermeas...