In this article, Gwern proposes replacing AI Dungeon's free-text parser with a menu of CYOA options. I don't think it will work, at least not with the current generation GPT-3.
GPT-3 won't be able to generate a good CYOA until it can generate a good non-interactive novel. Today, GPT-3 can generate text, so you could try to use it to generate a whole novel. People have, but the novels it generates aren't worth reading.
"CYOAs" are interactive novels. I run a company, Choice of Games, that publishes hand-written interactive novels. Our novels use hidden state to provide an experience that's longer, richer, and deeper than traditional paper-based CYOA books. (The author of TFA links to one of our blog posts in the footer of this article. https://www.choiceofgames.com/2011/07/by-the-numbers-how-to-... )
Our approach is to pay professional authors to write these interactive novels. (And we have professional editors who read and edit their work.)
Could we use GPT-3 to avoid paying professional authors? No, of course not. Writing an interactive novel with interesting, dramatic choices is harder than writing a non-interactive novel, and GPT-3 can't even do that.
The feature that makes AI Dungeon interesting and unique is its ability to improvise in response to player's actions. Presenting a pre-computed menu of options is what hand-written interactive novels already do; to succeed, it would have to compete with hand-written interactive novels on quality.
In other words, when GPT-3 computes an entire novel, it's indistinguishable to the reader from a (bad) hand-written novel. If AI Dungeon were to auto-generate an interactive novel, particularly in the way Gwern describes here, with the choices already pre-computed and crowdsourced in advance, the result would be indistinguishable from a (bad) hand-written interactive novel.
I think it's possible that if some future generation of GPT could generate a good-enough novel, then it could also generate a good-enough interactive novel, but this thing has gotta learn how to crawl before it can learn how to walk.
I think what Gwern is proposing is perhaps better seen by you as an optimization for the current AID rather than a "true" CYOA novel implementation. Because most of what you said applies to AID as well. I toyed with it a bit, but even short snippets of text are often incoherent and I have to give it a lot of grace to operate. In particular, while I don't necessarily mind the way it'll just introduce a new character, I don't like the way they disappear equally quickly and without fanfare.
While AID's nominal attraction is the ability to react to anything, I think based on the evidence only a small percentage of the users end up using it that way. (It's possible they're all the long term users, in which case they are important, but I still think it's a small proportion of the users.) The vast majority of the users the vast majority of the time will be selecting from a small handful of options that would constitute the vast majority of responses.
To the extent you'd find the resulting CYOA rather unappealing, I'd say the current AID is pretty much unappealing in exactly the same way, for exactly the same reasons, and given that AID hasn't exactly taken the world by storm I imagine this is the majority view. (Though it may have the sort of inner core rabid fanbase that you can still build on as a business.)
> Today, GPT-3 can generate text, so you could try to use it to generate a whole novel. People have, but the novels it generates aren't worth reading.
Novels are definitely out of reach for these language models. GPT-3 currently has a limit of 2048 tokens. This is short-story territory but definitely not "book" length. I agree in this regard.
> GPT-3 won't be able to generate a good CYOA until it can generate a good non-interactive [story].
I disagree here, however. One of the best things about AI-generated text is that it can generate a multitude of options for story progression. Sure, not all progressions will be good, but the most important thing in CYOA is the _choice_. As long as there are some good options, users can choose which progressions they like best, even if the other options suck.
For example, with this toy example [1], not all the options for the progression of the story are great. But if you pick and choose which progressions you like best, you can arrive at a pretty good ending, such as [2]. -- Albeit this is entirely subjective to what each individual finds entertaining
GPT-3 won't be able to generate a good CYOA until it can generate a good non-interactive novel. Today, GPT-3 can generate text, so you could try to use it to generate a whole novel. People have, but the novels it generates aren't worth reading.
"CYOAs" are interactive novels. I run a company, Choice of Games, that publishes hand-written interactive novels. Our novels use hidden state to provide an experience that's longer, richer, and deeper than traditional paper-based CYOA books. (The author of TFA links to one of our blog posts in the footer of this article. https://www.choiceofgames.com/2011/07/by-the-numbers-how-to-... )
Our approach is to pay professional authors to write these interactive novels. (And we have professional editors who read and edit their work.)
Could we use GPT-3 to avoid paying professional authors? No, of course not. Writing an interactive novel with interesting, dramatic choices is harder than writing a non-interactive novel, and GPT-3 can't even do that.
The feature that makes AI Dungeon interesting and unique is its ability to improvise in response to player's actions. Presenting a pre-computed menu of options is what hand-written interactive novels already do; to succeed, it would have to compete with hand-written interactive novels on quality.
In other words, when GPT-3 computes an entire novel, it's indistinguishable to the reader from a (bad) hand-written novel. If AI Dungeon were to auto-generate an interactive novel, particularly in the way Gwern describes here, with the choices already pre-computed and crowdsourced in advance, the result would be indistinguishable from a (bad) hand-written interactive novel.
I think it's possible that if some future generation of GPT could generate a good-enough novel, then it could also generate a good-enough interactive novel, but this thing has gotta learn how to crawl before it can learn how to walk.