Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Building an email-to-calendar LLM

ngacho.com

Post reply on HN
Print view View original

21–30 of 46 posts

Re: Building an email-to-calendar LLM

#21

Post by Mathnerd314 »

robertclaus wrote: post #13

"Setting up LLMs to output structured data is incredibly hard." resonated strongly with my experience working in similar one-off projects. I've almost always implemented some level of fuzzy-matching to validate and convert the LLM output back into my expected structured format. I've also noticed that the LLMs are much better at writing code than structured JSON (no real surprise given the popularity of code assistant…

I think it's the training data, there is not a lot of JSON. It's much easier to get it to generate list-style data, like "foo:\n* prop1 - val1\n* prop2 - val2", or similar formats, as the models seem to have seen a lot of that sort of data.

Re: Building an email-to-calendar LLM

#23

Post by bboygravity »

robertclaus wrote: post #13

"Setting up LLMs to output structured data is incredibly hard." resonated strongly with my experience working in similar one-off projects. I've almost always implemented some level of fuzzy-matching to validate and convert the LLM output back into my expected structured format. I've also noticed that the LLMs are much better at writing code than structured JSON (no real surprise given the popularity of code assistant…

You're aware ChatGPT4 has a json only mode?

Re: Building an email-to-calendar LLM

#24

Post by throwaway11460 »

darby_eight wrote: post #20
refulgentis wrote: post #15

Earlier quoted context omitted.

A 3B model runs on Android phones from 2 years ago at 6 tkns/s.

I'm not sure what you're comparing this to or how you're making this comparison—can you enlighten us? (Somehow I doubt whatever caldav software the above poster references takes more than a second to process multiple emails.)

I don't think it's a comparison - they're just saying that it's fast enough even on old mobile hardware, so it can be used on new hardware even faster.

I don't have a problem with a background task taking a minute or something...

Re: Building an email-to-calendar LLM

#25

Post by refulgentis »

darby_eight wrote: post #20
refulgentis wrote: post #15

Earlier quoted context omitted.

A 3B model runs on Android phones from 2 years ago at 6 tkns/s.

I'm not sure what you're comparing this to or how you're making this comparison—can you enlighten us? (Somehow I doubt whatever caldav software the above poster references takes more than a second to process multiple emails.)

Cost isnt a relevant factor for this

Re: Building an email-to-calendar LLM

#26

Post by [deleted] »

IanCal wrote: post #12
denton-scratch wrote: post #5

Just use CALDAV; it's designed for making calendar entries automatically via email. I'm not hip with the fashion for putting an LLM into everything. I think it's lazy.

That's a great solution to a different problem. Unless caldav has a process for extracting dates and actions from unstructured emails? But that doesn't seem related to caldav.

[deleted]

Re: Building an email-to-calendar LLM

#27

Post by mofosyne »

jakecodes wrote: post #17

Hey! It's awesome to read other people's solutions to this. I've been working on solving this for the past 2 years or so and I went through much of the same struggles in the beginning until we came up with a solution which is fairly complex, to get LLM's to output data in a way we can use. The big problem is that 95% accuracy is not good enough for calendars. People lose confidence after 1 failed attempt. Trying to g…

Due to the vagueness of human language, could we also output a degree of confidence in the translation

Re: Building an email-to-calendar LLM

#28

Post by sanity »

htrp wrote: post #11
simonw wrote: post #9

Earlier quoted context omitted.

Apps Script is positively ancient at this point, first released in 2009: https://en.wikipedia.org/wiki/Google_Apps_Script It's one of the Google products I worry least about, mainly because there are 15+ years of existing Google Sheets documents that people have built using it at this point. I don't think even Google would lightly break THAT many of their existing (often paid) users.

> I don't think even Google would lightly break THAT many of their existing (often paid) users. Says everyone using GCP services that get deprecated.

So true, today I was trying to decide whether to use Google's Vertex vector search in a project - until I remembered Google's track record with pulling the rug on services. Made it an easy "no".

Re: Building an email-to-calendar LLM

#29

Post by nosefurhairdo »

robertclaus wrote: post #13

"Setting up LLMs to output structured data is incredibly hard." resonated strongly with my experience working in similar one-off projects. I've almost always implemented some level of fuzzy-matching to validate and convert the LLM output back into my expected structured format. I've also noticed that the LLMs are much better at writing code than structured JSON (no real surprise given the popularity of code assistant…

Some ideas in this forum thread on the same topic: https://genai.stackexchange.com/questions/202/how-to-generat...

Re: Building an email-to-calendar LLM

#30

Post by iot_devs »

I am working on GabrielAI, which is a tool to filter and autodraft reply for Gmail and outlook and of course it uses LLM under the hood.

https://getgabrielai.com

In cases like this the author could just set up a filter like:

"If the email contains a task for me" (or some variation)

Then add a Gmail label to it.

In this way the author will immediately find all the actionable emails for him in a specific folder, much faster to skim and to keep track of all of them.

Another option would it be to have GabrielAI generate a Draft like "reply acknowledging the task and put a to-do date in the email in 1 week"

This would allow Google to track the email and the deadline.

Post reply on HN