Your intuition is correct but, unfortunately, it’s not simply a convolution.
A great example of this is DiffuserCam [1], which uses a diffuser as a “lens”, combined with a carefully measured point spread function (PSF) of a single point for each axial plane to estimate the “true” PSF that actually varies throughout the entire volume. So although in this work they estimate the reconstruction by making a translational invariance assumption, this is done for practical reasons and quality is lost.
To fully recover a lossless image as you suggest, one would have to measure a PSF for each point in the volume.
A great example of this is DiffuserCam [1], which uses a diffuser as a “lens”, combined with a carefully measured point spread function (PSF) of a single point for each axial plane to estimate the “true” PSF that actually varies throughout the entire volume. So although in this work they estimate the reconstruction by making a translational invariance assumption, this is done for practical reasons and quality is lost.
To fully recover a lossless image as you suggest, one would have to measure a PSF for each point in the volume.
[1] https://waller-lab.github.io/DiffuserCam/