Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Launch Lamini: The LLM Engine for Rapidly Customizing Models as Good as ChatGPT

lamini.ai

Post reply on HN
Print view View original

11–20 of 75 posts

Re: Launch Lamini: The LLM Engine for Rapidly Customizing Models as Good as ChatGPT

#11

Post by mckirk »

brookst wrote: post #10
primordialsoup wrote: post #8

Congrats! I went to your demo and asked for words that end in agi. This is what I got: -- agi, agi, agi, agi, agi, agi, agi These are some of the words that end in agi. You can also use the word agi in a sentence. For example, "I am going to the grocery store to get some agi." These are some of words that end in agi. These are some words that end in agi. maximize, maximize, maximize, maximize, maximize, maximize, max…

It's a fair criticism, and ChatGPT does better, but this isn't a great test of model quality. All LLMS that rely on tokenization struggle with being introspective on language. Try asking chatGPT to count how many e's are in a sentence, or to list all words that start with "to" and end wide "de". I haven't heard anyone describe the phenomenon clearly, but I expect it is a challenge with reasoning over both intent of t…

The problem is that these models do not have any working memory they could use to carry out such tasks, which are on a meta-level when seen from a language perspective. They can only go with their 'gut instinct' for selecting the next word, they can't 'consider and ponder the problem internally' first.

Re: Launch Lamini: The LLM Engine for Rapidly Customizing Models as Good as ChatGPT

#12

Post by TwoFactor »

primordialsoup wrote: post #8

Congrats! I went to your demo and asked for words that end in agi. This is what I got: -- agi, agi, agi, agi, agi, agi, agi These are some of the words that end in agi. You can also use the word agi in a sentence. For example, "I am going to the grocery store to get some agi." These are some of words that end in agi. These are some words that end in agi. maximize, maximize, maximize, maximize, maximize, maximize, max…

Thats an interesting test. Here's what I got from ChatGPT:

---GPT-3.5---

Here are some words that end in "agi":

Strategy

Swarajya

Arthroplasty

Sialagogue

Podagric

Gynecology

Physiognomy

Ophthalmology

Esophagitis

Otalgia

--- GPT-4 ---

Here are some words that end in "agi":

Swaggy

Raggi

Magi

Gagi

Stagi

Please note that some of these words may not be commonly used or may be specific to certain dialects or regions.

Re: Launch Lamini: The LLM Engine for Rapidly Customizing Models as Good as ChatGPT

#13

Post by jameslevy »

mckirk wrote: post #11
brookst wrote: post #10

Earlier quoted context omitted.

It's a fair criticism, and ChatGPT does better, but this isn't a great test of model quality. All LLMS that rely on tokenization struggle with being introspective on language. Try asking chatGPT to count how many e's are in a sentence, or to list all words that start with "to" and end wide "de". I haven't heard anyone describe the phenomenon clearly, but I expect it is a challenge with reasoning over both intent of t…

The problem is that these models do not have any working memory they could use to carry out such tasks, which are on a meta-level when seen from a language perspective. They can only go with their 'gut instinct' for selecting the next word, they can't 'consider and ponder the problem internally' first.

Perhaps prompting the model to first describe its approach to answering the question. This type of chain-of-thought technique can yield better results.

Re: Launch Lamini: The LLM Engine for Rapidly Customizing Models as Good as ChatGPT

#14

Post by jasonjmcghee »

sharonzhou wrote: post #2

Hi HN! I’m super excited to announce Lamini, the LLM engine that gives every developer the superpowers that took the world from GPT-3 to ChatGPT! I’ve seen a lot of developers get stuck after prompt-tuning for a couple days or after fine-tuning an LLM and it just gets worse—there’s no good way to debug it. I have a PhD in AI from Stanford, and don’t think anyone should need one to build an LLM as good as ChatGPT. A w…

You’re building some seriously exciting stuff! Looking forward to diving in.

Re: Launch Lamini: The LLM Engine for Rapidly Customizing Models as Good as ChatGPT

#15

Post by armchairhacker »

TwoFactor wrote: post #12
primordialsoup wrote: post #8

Congrats! I went to your demo and asked for words that end in agi. This is what I got: -- agi, agi, agi, agi, agi, agi, agi These are some of the words that end in agi. You can also use the word agi in a sentence. For example, "I am going to the grocery store to get some agi." These are some of words that end in agi. These are some words that end in agi. maximize, maximize, maximize, maximize, maximize, maximize, max…

Thats an interesting test. Here's what I got from ChatGPT: ---GPT-3.5--- Here are some words that end in "agi": Strategy Swarajya Arthroplasty Sialagogue Podagric Gynecology Physiognomy Ophthalmology Esophagitis Otalgia --- GPT-4 --- Here are some words that end in "agi": Swaggy Raggi Magi Gagi Stagi Please note that some of these words may not be commonly used or may be specific to certain dialects or regions.

Stagi isn't a word (unless you count Lojban). Gagi isn't a word unless you could Filipino slang.

Re: Launch Lamini: The LLM Engine for Rapidly Customizing Models as Good as ChatGPT

#16

Post by sp332 »

mckirk wrote: post #11
brookst wrote: post #10

Earlier quoted context omitted.

It's a fair criticism, and ChatGPT does better, but this isn't a great test of model quality. All LLMS that rely on tokenization struggle with being introspective on language. Try asking chatGPT to count how many e's are in a sentence, or to list all words that start with "to" and end wide "de". I haven't heard anyone describe the phenomenon clearly, but I expect it is a challenge with reasoning over both intent of t…

The problem is that these models do not have any working memory they could use to carry out such tasks, which are on a meta-level when seen from a language perspective. They can only go with their 'gut instinct' for selecting the next word, they can't 'consider and ponder the problem internally' first.

The problem is that the input is tokenized before the model gets it as input. It does not see the individual letters "t" + "o". It gets one single token, #1462. The word "toe" is another single token, #44579. Maybe over time it could learn from context that inputs that start with #44579 also satisfy the constraint of starting with #1462, but that's a lot of work and it's not going to happen for all combinations of letters.

Re: Launch Lamini: The LLM Engine for Rapidly Customizing Models as Good as ChatGPT

#17

Post by mejutoco »

armchairhacker wrote: post #15
TwoFactor wrote: post #12

Earlier quoted context omitted.

Thats an interesting test. Here's what I got from ChatGPT: ---GPT-3.5--- Here are some words that end in "agi": Strategy Swarajya Arthroplasty Sialagogue Podagric Gynecology Physiognomy Ophthalmology Esophagitis Otalgia --- GPT-4 --- Here are some words that end in "agi": Swaggy Raggi Magi Gagi Stagi Please note that some of these words may not be commonly used or may be specific to certain dialects or regions.

Stagi isn't a word (unless you count Lojban). Gagi isn't a word unless you could Filipino slang.

To be fair the question did not specify the language and included a disclaimer about it.

Re: Launch Lamini: The LLM Engine for Rapidly Customizing Models as Good as ChatGPT

#19

Post by jeffybefffy519 »

sharonzhou wrote: post #2

Hi HN! I’m super excited to announce Lamini, the LLM engine that gives every developer the superpowers that took the world from GPT-3 to ChatGPT! I’ve seen a lot of developers get stuck after prompt-tuning for a couple days or after fine-tuning an LLM and it just gets worse—there’s no good way to debug it. I have a PhD in AI from Stanford, and don’t think anyone should need one to build an LLM as good as ChatGPT. A w…

Im confused, what are you actually offering? Does my fine tuning data get shared with your platform’? Does the model get fine tuned on your end or my own system? Do you host the model?

Re: Launch Lamini: The LLM Engine for Rapidly Customizing Models as Good as ChatGPT

#20

Post by chaxor »

brookst wrote: post #10
primordialsoup wrote: post #8

Congrats! I went to your demo and asked for words that end in agi. This is what I got: -- agi, agi, agi, agi, agi, agi, agi These are some of the words that end in agi. You can also use the word agi in a sentence. For example, "I am going to the grocery store to get some agi." These are some of words that end in agi. These are some words that end in agi. maximize, maximize, maximize, maximize, maximize, maximize, max…

It's a fair criticism, and ChatGPT does better, but this isn't a great test of model quality. All LLMS that rely on tokenization struggle with being introspective on language. Try asking chatGPT to count how many e's are in a sentence, or to list all words that start with "to" and end wide "de". I haven't heard anyone describe the phenomenon clearly, but I expect it is a challenge with reasoning over both intent of t…

It's not a character based model (likely - although it's closed source so anything is technically possible behind the scenes) so this makes some sense. The system can infer some relationships, which may be why 'agy' is conflated with 'agi' interestingly, but the tokenization process yields sequences of 'symbols' or indexes that are decided to English - so the system has a more difficult task when asked about 'e's (probably something like token 4893) and has to determine which tokens (e.g. [358,284840, 58292, 4830104, 57282, 4829193, 58282, 384, 24945] contain 'e's or token 4893). None of them do directly it seems - but 58292 may be 'ee' - so you would get this wrong as well.
Post reply on HN