The 1st NovelAI Stream & Q&A Summary
First off, thank you all for making the first NovelAI Twitch stream a blast! We’ve had a fantastic time answering questions, chatting, and seeing your excitement for the future of NovelAI! Now it’s time to summarize everything we’ve covered during the stream!
Let’s reintroduce the three hosts of the stream: Behind the goose images, we can find Kurumuz, NovelAI’s project lead. TabloidA is the designer responsible for the beauty and accessibility of UI. Finally, we have Aini, the community manager.
4.28.2021 Team gets together
6.15.2021 10 AM PT, beta launched with Calliope 2.7B.
7.02.2021 Sigurd V3 for all subscription tiers
7.30.2021 Custom AI Module training for Opus tier
10.06.2021 Phrase Bias
11.10.2021 Sigurd V4 for Opus, Snek GPT-6b model for python code for Opus & Genji GPT-6b Japanese Storyteller model for all tiers
12.18.2021 Free Trial, UI Overhaul, Tutorial
02.06.2022 Euterpe V2 for all tiers (including free trial) + Euterpe Custom AI Module Training
Our immediate focus is to release a finetuned GPT-NeoX 20B AI model. We are tentatively estimating this to be done within the next seven days, given that everything works out according to plan.
NovelAI TTS: A neural network-based improved Text to Speech engine hosted by us.
Scripting: Enabling our users to program truly unique and custom NovelAI experiences.
Official NovelAI Explore Platform: A centralized official community-driven NovelAI content repository (featuring all NovelAI formats such as scenarios, presets, modules, scripting experiences, themes, etc.), alongside the ability to import external NovelAI content directly into the service. The internal NovelAI repository will not feature NSFW content. NovelAI aims to make it easy for other content creators to access their decentralized collected works without leaving the platform. This service will feature an API specification that users build into their platforms to access their externally hosted content through NAI.
We are happy to hear your suggestions and feedback as we build out this service.
A brand new, improved type of Text Adventure AI & UI: Increased interactive capabilities of understanding and functionalities.
20B Neo-X AI Model when?
Most likely within the next week.
How does 20B compare to the other AI Models?
There isn’t as much of a significantly felt jump in performance as we experienced between 6B and 13B. But we hope to push its capabilities with a good finetune.
What is the context size of 20B at this time?
Right now, it runs at 1600 tokens, but we are aiming for the usual 2048 token context.
Is 20B slow compared to 13b?
Yes, generation time is almost two times slower than 13b.
Is 20b going to release with Custom AI Module support?
We hope to push for 20b to release with custom AI module training but can’t make any promises.
Will the 20B model illustration be male?
Yes, the next model is a male, keeping up the alternating genders of the NovelAI AI Model family.
Is NovelAI doing good financially?
We usually don’t want to focus on financial questions, but the service is doing good.
Is NovelAI Multiplayer planned?
Multiplayer has been on our internal wishlist since the beginning, but we do not have an ETA at this time. Our multiplayer vision would be more along the lines of collaborative writing — imagine google documents.
Why did other platforms release 20B before you, although you had early access?
Our finetune dataset is very big, and we wanted to finish training in order to deliver the best possible user experience.
How will scripting compare to the scripting in AIDungeon? What code languages will scripting use?
We hope to greatly expand upon scripting and its capabilities. We also aim to support Javascript and WebAssembly as our supported scripting languages.
Will there be easy picture insertion for Custom AI Modules & what happened to NovelAI’s image generation prototypes that floated around
We prototyped pixel generation back in December, but it ultimately fell to the wayside as other projects took priority. We do have planned uses for implementation in the future, such as Custom AI Modules and possibly in the upcoming NovelAI Explore platform.
Will images be pre-generated?
No, images will be uniquely generated based on your user input.
Do you have samples for AI-Generated images?
You can see some of our AI-generated images over on our #bob-ross discord channel, but the Discord bot isn’t running the pixel version we prototyped back in December- please keep in mind these were prototypes and we are not sure how the final image implementation into NovelAI will look!
Prompt: mermaid princess. Your hair is a dreamlike shade of pink, flowing endlessly below your waist. Your ears are slightly pointed, as are your canines. A magnificent pink mermaid tail replaces your legs.
Prompt: Exploding kittens
Prompt: Yellow Water Lake
Will there be shelves within shelves (nested shelves)?
Yes, nested shelves have already been designed visually, but the functionality has not yet been implemented.
What is GooseAI, and will it take the focus off of NovelAI?
GooseAI allows NovelAI to team up with CoreWeave to apply our capabilities and technical knowledge in a B2B space. We hope to help anyone previously hindered by OpenAI’s cost and TOS to build AI-based products. The cost structure for OpenAI charges for input tokens which can be an issue for interactive products such as chatbots or storytelling applications that reply on sending a large number of tokens all the time. We want GooseAI to make available for these types of applications therefore GooseAI only charges for the tokens that are generated. It does not take attention from NovelAI but helps us expand Anlatan.
Is your new TTS based on 15.ai?
No, our new TTS service is hosted internally.
Are you planning to add Pro Writer 2 to the Default Presets?
Pro Writer’s addition is planned. We’ve already got permission from Basileus to do so.
What’s your quality control process for preventing bugs like TFS and top-k in the future?
We try to do our best to catch bugs by writing test cases for our node generations but sometimes problems with nodes happen, which are a little trickier to catch. We do hope to improve upon this in the future.
Does it cost more compute to generate with 20b in comparison to 13b?Yes, it is nearly double the cost due to rotary embeddings and size differences.
Are we planning to surpass advanced models such as AIDungeon’s legendary Dragon?
We are confident that we’ve surpassed Dragon’s performance in some aspects already.
Does the team play NovelAI outside of testing?
Yes, but most of the focus relies on playing for quality testing purposes.
Will you support rephrasing (type a sentence and let the AI reformulate it in a specific style)?
It is not on the roadmap at this time, but rephrasing is definitely on our wishlist.
Will you add a chatbot UI or stop sequence functionality that restricts/stops AI outputs?
We are not planning to implement a chatbot interface. However, we do hope to implement a stop sequence functionality that forces the AI to stop generation when a specific stop sequence string or token is used.
Will Knowledge Graphs ever happen?
No, Knowledge Graphs will not happen. We came to the conclusion that Knowledge Graphs are not an ideal approach for our purposes, but we’ve looked into alternative methods to accomplish the same goals, which we hope to bring to you soon™
Do you have plans for automated AI summary functions?
We are exploring alternative ways to do this, but only if we manage to polish it and make it well.
What’s your favorite AI Module?
Kurumuz: Mercantile Wolfgirl Romance — Aini: I prefer to train Custom Modules on my own writings to pull from my own worldbuilding and atmospheric vectors. It does feature all my phone playing typos though, so I’m not sharing that!
Will there be improvements to Custom AI Module training speed?
Yes, possible if Kurumuz finds the time to do so.
Will you add any other language AI Models?
We actually tuned an English model into Japanese for research which is our Genji model. so there is a chance we might do it for other languages in the future.
Are there plans to ways to improve AI memory?
We’ve created some new experimental tech for our upcoming Text Adventure, which we hope to implement across all AI models as well.
Will there be more NovelAI Twitch Streams?
Yes, but we haven’t decided on a schedule yet! We want to host regular fun streams, so If you have any content you’d like to see, things we should cover, or general suggestions regarding stream days and time, then please let us know!
Is NovelAI interested in partnerships with Indie Game Developers?
Definitely! Hit us up at contact@novelai.net or contact Kurumuz directly at eren@novelai.net
Are there plans for custom model hosting on GooseAI?
Please direct any GooseAI related questions to sales@goose.ai
Are NovelAI gift subscriptions still planned?
We have to continue looking into this internally and see if we can offer it.
Are there plans for tutorials on how to instruct the AI? As in what sentence structure and punctuation to use in Memory, Author’s Note, and Lorebook in order to get the best results / Will you host an official knowledge base on the NoveAI website?
Yes, we hope to implement better in-depth tutorials and possibly an internal NovelAI knowledge base, somewhat like the existing NovelAI Community UKB but more general in tone and accessibility.
Is it possible to do inline generation that considers the entire context after the inline position?
Maybe, but we would have to look into it further.
Will the existing inline generation come to mobile?
We will see if we can offer the functionality on mobile!
How were the default scenario authors chosen, and can one become a featured author at this time?
We picked high-renown members of the AI Dungeon Explore community who already knew to fully utilize the features of AI writing, such as Author’s Notes, Memory, and Lorebooks. We haven’t looked into expanding on default scenarios or authors at this time due to the constant updates that require the reworking of the scenarios and settings. It becomes quite challenging to coordinate regular updates across a large number of scenarios in many different perspectives with their respective authors. We hope to remedy this problem with our own official Explore / NovelAI content repository. We try our best to include high popularity content with unique features such as the generator scenarios written by Pause and lion. If there are scenarios you think we should feature, please reach out to us.
How have you been viewing the popularity increase of NovelAI?
We can barely come to terms with how much we’ve grown. Seeing some of our favorite streamers play NovelAI has been wild.
How much has the platform grown?
We might be able to release general statistics in a graphic for our anniversary so maybe tune back into our stream for our anniversary (if we host one, haha)!
Are there plans to add anything like the World System in AIDungeon?
We feel our Lorebook and Modules are pretty much sufficient but hope to rework and expand upon the Lorebook UI at some point.
Do you have plans for AI-generated music/ambiance?
We’ve recently discussed it and maybe creating something self-composing, or along those lines in the future.
How do AI Modules work?
Instead of inserting regular text tokens, you insert a learned dataset (the AI Module) that will affect the probabilities of your AI outputs all across your story.
We hope to create a graphic for it since the impromptu stream sketch TabloidA made didn’t really illustrate/explain it that well…
Will you add clickable Token Probabilities like in GooseAI?
We will probably implement this in NovelAI as well.
Will the smaller AI Models (6b Sigurd and 2.7b Calliope) continue to be updated?
Yes.
Will we ever have the option to see more than the top 10 tokens in the token probabilities viewer?
The way we push the probabilities to the frontend is not very efficient at this time, so maybe if we do a rework of our backend, we could possibly implement that.
Have you ever experimented with implementing NovelAI into video games?
Sadly it’s out of our expertise, and we don’t really have the time for it.
Could training steps be donated to other people?
Maybe, but no promises on that end.
Is NovelAI currently recruiting?
Shoot an email at contact@novelai.net
Are there plans for inserting token embeddings instead of text tokens?
Maybe but this would require further research.
Any thoughts on a pay-as-you-go option for NovelAI?
We don’t think that it would be a sustainable system for NovelAI.
Are there any plans for a $5 NovelAI subscription tier?
We would need to consider this in-depth.
Would it be possible to have multiple TTS voices alternating for the dialogue?
This might be doable with scripting in the future.
We started out the actual NovelAI playing session by teasing a few of the new TTS voices with some random generations alongside the occasional pogchamp prompt.
You can see everyone’s reactions to the first TTS voice we revealed in this Twitch Clip from the stream: https://clips.twitch.tv/SnappyBigOwlOptimizePrime-hds8nc6wRyj_Tnbo
Watch the chat reactions!
For the first story, we used the latest iteration of our Special Text Adventure AI Module (featuring an increased context size, which is already patched into NovelAI now!) fresh off the press alongside a new Text Adventure-focused Ouroboros preset. TabloidA made it a point to eat any available currency in the wall-less establishment or a rat bartender and a nearby bank until it ended in fiery destruction.
For the second story, we pulled in a very early work in progress version of the planned future reworked Text Adventure AI that keeps track of a few more things in the background. It was definitely an early showcase, but it was neat to see the AI’s thoughts when you turn cats with guns into catgirls and teach them how to become pleasing maids before attempting to kill them for no good reason.
For the final story, we switched to our latest work in progress, our 20b Neo-X AI Model, to see how it would handle a longer context prompt provided in chat by OrionON1K. The story setup focused on a merchant with a general store, beginning with customers entering the store. We had to wrangle it a little, but the customers turned out to be thieves pressuring us into a partnership. Later that day, our merchant was followed by a shady figure and approached with a mysterious book. It was fantastic to see the AI connect the dots when we were followed by the aforementioned shady person which led to a nervous confrontation — considering that 20b only has a context of around 1600 tokens, it did do quite nicely in remembering that our character was a shopkeeping merchant.
After this, we reached the three-hour mark, and the three of our hosts were pretty much spent mentally, but we answered a handful more questions before calling it a night:
When will NovelAI come out of Beta?
We’re feeling pretty close, but there are still some refinements to be made. Hopefully, near the time that we finish Scripting & the Explore plans.
Is there a way to get the AI to understand things like no and not / negations?
Try to hammer it in without negating it, instead of describing a character as “not tall,” find different descriptors such as “short,” “small height,” or maybe even “tiny built” — hammer in those descriptors by using some synonyms a few times in your context (whether it be through story text or maybe your Lorebooks) and the AI should be able to pick up on that without too many issues. This is because the AI pulls from the words that you use to describe things. If you simply mention the negating descriptor “not tall,” the AI will see the word (tokenized version) “tall” and merely raise the probabilities of words that are commonly found near it in the training data.
Will the AI context be expanded beyond 2048?
We might be able to implement a higher context for our smaller AI models if there is a demand to do so.
Will other developers be on future streams?
We’ll see if some of the other developers are willing to come on and get over their shyness! Possibly in a VRChat kind of setting.
Does NovelAI have an official YouTube channel?
Not yet, but hopefully soon!
These are our major plans for 2022!
So thanks again for tuning in and we hope our roadmap will put some of your concerns about our immediate focus list at ease!
Let us know what type of streams you would like to see next so we can prepare some more fun live interactions for you! Follow us on Twitch for future stream notifications: https://www.twitch.tv/novelai/
As always, please share your thoughts, feedback, and suggestions with us on any of our official channels!
