"Two: History shows that for the general public, and even for scientists not in a key inner circle, and even for scientists in that key circle, it is very often the case that key technological developments still seem decades away, five years before they show up.
In 1901, two years before helping build the first heavier-than-air flyer, Wilbur Wright told his brother that powered flight was fifty years away.
In 1939, three years before he personally oversaw the first critical chain reaction in a pile of uranium bricks, Enrico Fermi voiced 90% confidence that it was impossible to use uranium to sustain a fission chain reaction. I believe Fermi also said a year after that, aka two years before the denouement, that if net power from fission was even possible (as he then granted some greater plausibility) then it would be fifty years off; but for this I neglected to keep the citation.
And of course if you're not the Wright Brothers or Enrico Fermi, you will be even more surprised. Most of the world learned that atomic weapons were now a thing when they woke up to the headlines about Hiroshima. There were esteemed intellectuals saying four years after the Wright Flyer that heavier-than-air flight was impossible, because knowledge propagated more slowly back then."
Maybe it's hundreds of years out, but maybe it's not. Since it's hard to know probably better to try and work on the safety aspects now.
Sure - this isn’t to imply that that can’t happen or even that that isn’t the more common case.
It’s only to say that just because something may seem far away it may not be - even if you’re the person that will invent it only two years later (and therefore are probably in the best position to know).
Given the high stakes of unsafe AGI and this uncertainty it’s probably worth some people working on goal alignment.
This is somewhat unrelated to the recent release though.
I'm not sure where I read a discussion about this, but I do think it was something they've talked about before.