Show HN: Graph-based AI for longform writing

#1

Post by Broge » Wed, Feb 22, 2023, 7:41 PM UTC

Hey everyone, I wanted to share a new tool we've created called Jotte (https://jotte.ai) which we believe can be a game-changer for AI-generated longform writing like novels and research papers.

As you may know, current AI like ChatGPT and GPT-3 have a token limit of around 4000 tokens or 3000 words, which limits their effectiveness for longer writing tasks. With Jotte, we've developed a graph-based approach to summarize information and effectively give AI "unlimited" memory.

Jotte remembers recent details like the meal a character ate a page ago, while avoiding getting bogged down by irrelevant details like the blue curtains mentioned 5 chapters ago. We've created a proof of concept and would love to hear your thoughts on it.

Do you think this approach could lead to better longform writing by AI? Let us know in the comments!

Show HN: Graph-based AI for longform writing
jotte.ai

Re: Show HN: Graph-based AI for longform writing

#2

Post by Mizza » Wed, Feb 22, 2023, 8:48 PM UTC

I am glad to see more stuff with graph based AI here.

I have a running bet with a friend about whether future is going to be OBM (One Big Model) or LoLM (Lots of Little Models). I'm strongly in the LoLM/graph camp and have been working in that direction as well: https://github.com/Miserlou/Helix

Re: Show HN: Graph-based AI for longform writing

#3

Post by danielbln » Wed, Feb 22, 2023, 9:09 PM UTC

Mizza wrote: post #2

I am glad to see more stuff with graph based AI here. I have a running bet with a friend about whether future is going to be OBM (One Big Model) or LoLM (Lots of Little Models). I'm strongly in the LoLM/graph camp and have been working in that direction as well: https://github.com/Miserlou/Helix

Seeing how langchain is gaining popularity and development rapidly, I would agree. Chaining lots of specific models and tools seems to be the way forward.

Re: Show HN: Graph-based AI for longform writing

#4

Post by Broge » Wed, Feb 22, 2023, 9:56 PM UTC

Mizza wrote: post #2

I am glad to see more stuff with graph based AI here. I have a running bet with a friend about whether future is going to be OBM (One Big Model) or LoLM (Lots of Little Models). I'm strongly in the LoLM/graph camp and have been working in that direction as well: https://github.com/Miserlou/Helix

Yeah I feel like for development, OBM is great and super flexible.

But when you actually want to deploy, a lot of tiny, more efficient models would probably be the best bet.

I read somewhere that the a company ended up fine-tuning FLAN-T5 instead of going GPT-3, which I can imagine saved them lots of $$.

Re: Show HN: Graph-based AI for longform writing

#5

Post by howon92 » Wed, Feb 22, 2023, 10:04 PM UTC

Congrats on the launch! I'm not your target market but am curious to learn how this gives AI "unlimited" memory. Whenever I try to use GPT-3 API, I'm blocked by the token limit for most practical applications. My two cents for the product itself is it seems more like a tool for developers than novel writers. Have you done any beta testing with your target users?

Re: Show HN: Graph-based AI for longform writing

#6

Post by richdougherty » Wed, Feb 22, 2023, 10:42 PM UTC

I would love a tool like this to combine requirements, comments and fragments of code into a larger program.

Re: Show HN: Graph-based AI for longform writing

#7

Post by LesZedCB » Wed, Feb 22, 2023, 10:56 PM UTC

Mizza wrote: post #2

I am glad to see more stuff with graph based AI here. I have a running bet with a friend about whether future is going to be OBM (One Big Model) or LoLM (Lots of Little Models). I'm strongly in the LoLM/graph camp and have been working in that direction as well: https://github.com/Miserlou/Helix

helix looks amazing! that's exactly the kind of thing i'm looking to burn through openai credits with.

Re: Show HN: Graph-based AI for longform writing

#8

Post by nicolas_17 » Wed, Feb 22, 2023, 10:56 PM UTC

What exactly is the use case for "longform writing by AI"? Profiting from mass-produced zero-effort novels?

Re: Show HN: Graph-based AI for longform writing

#9

Post by Broge » Wed, Feb 22, 2023, 10:57 PM UTC

howon92 wrote: post #5

Congrats on the launch! I'm not your target market but am curious to learn how this gives AI "unlimited" memory. Whenever I try to use GPT-3 API, I'm blocked by the token limit for most practical applications. My two cents for the product itself is it seems more like a tool for developers than novel writers. Have you done any beta testing with your target users?

Thanks for the kind words!

The unlimited part comes from the AI knowing just enough context to stay coherent in any situation. Current long-form text techniques usually just summarize the past n tokens, and maybe the previous summary as well. The problem with this is that it quickly loses specifics of anything that happened just outside the window.

What Jotte's graph-based approach does is have weighted summaries, allowing the important information to stay in there much longer.

Definitely agree that the interface is still pretty rough, but we wanted to just get public reception on this sort of thing. We've done some testing on hyrid dev/writers, but it needs a more rigid structure before we even try to test this on pure writers.

Re: Show HN: Graph-based AI for longform writing

#10

Post by gjstein » Wed, Feb 22, 2023, 11:12 PM UTC

nicolas_17 wrote: post #8

What exactly is the use case for "longform writing by AI"? Profiting from mass-produced zero-effort novels?

Just as ChatGPT seems pretty capable at summarizing text, an AI with "unlimited memory" could potentially answer analytical questions about larger datasets and non-linear data (in the sense that prose is read from start-to-finish).

The OP is most excited about this ability to remember to create more structured longford outputs with internal consistency (e.g., asking questions about a fantasy universe that respects the characters that exist elsewhere in the story or universe).