You can audit your Signal client app (which is open source) and see exactly what it sends, and how.
Now realistically, most people can do that, so they have to trust that somebody who can actually did it. Security experts have been and are looking into messengers like Signal: they have an interest to do so as security researchers, because that would look good on their CV.
Of course there is trust somewhere, you cannot do without trust. But Signal is amongst the best you can find.
I know that part, that's why I asked "how can one tell the app blob in their phone is the same as the one produced if you build the code?".
The problem being that apps as installed from the App Store come (to my knowledge, could be wrong) encrypted. So you can't just compare an unencrypted local build with the encrypted installed app (and you can't decrypt the app or encrypt your build the same way, because you don't have the key, e.g. Apple has it - and iirc, there's no access to it, because it's held in the secure enclave in iOS case).
Now realistically, most people can do that, so they have to trust that somebody who can actually did it. Security experts have been and are looking into messengers like Signal: they have an interest to do so as security researchers, because that would look good on their CV.
Of course there is trust somewhere, you cannot do without trust. But Signal is amongst the best you can find.