Exactly, and I've kept an eye on Nature Materials and other journals that cover advances in semiconductor cooling but so far there hasn't been anything very exciting. I was hoping the ultra thin silicon guys would use diamond as a substrate for its thermal properties (sort of like silicon on sapphire but silicon on diamond instead) but apparently not a thing yet.
The brain also runs at a much lower frequency than a CPU. A neuron can only fire about 200 times per second, or 200Hz. This is important because power consumption is directly proportional to frequency squared.