Running unikernels on SEL4 is a perfectly sane thing to do. SEL4 does not provide the network stack, or much application interface, so a unikernel is a great thing to put on top.
I really do feel like at the end of this container experiment we're going to reinvent microkernels. Possibly badly, but I'm hopeful that it won't work out that way.