Live data from Hacker News

ChatGPT vs. open source on harder tasks

github.com

11–20 of 70 posts

Re: ChatGPT vs. open source on harder tasks

#11

I'm employed by Microsoft, but not in any relation to the authors of this tool or this post. First time I saw github.com/mirosoft/guidance was on the HN front page, actually :) I've been waiting for a tool exactly like guidance, something that lets me compose prompts the way I think about prompts. Langchain was too much of a headache for me. Playing with guidance, it feels like this is how programmatically interactin…

langchain caused more problems than it fixed. Really ugly software.

Re: ChatGPT vs. open source on harder tasks

#12

I'm employed by Microsoft, but not in any relation to the authors of this tool or this post. First time I saw github.com/mirosoft/guidance was on the HN front page, actually :) I've been waiting for a tool exactly like guidance, something that lets me compose prompts the way I think about prompts. Langchain was too much of a headache for me. Playing with guidance, it feels like this is how programmatically interactin…

langchain caused more problems than it fixed. Really ugly software.

Please elaborate. So far it seems promising to me, but I started using it just recently.

Re: ChatGPT vs. open source on harder tasks

#13

I'm employed by Microsoft, but not in any relation to the authors of this tool or this post. First time I saw github.com/mirosoft/guidance was on the HN front page, actually :) I've been waiting for a tool exactly like guidance, something that lets me compose prompts the way I think about prompts. Langchain was too much of a headache for me. Playing with guidance, it feels like this is how programmatically interactin…

are we going to now invent a programming language that communicates with LLMs? I find that funny

Re: ChatGPT vs. open source on harder tasks

#14
post #5

Open source will be behind always since it is harder to organize a large group of people and once it publishes it will be copied by big players, however if an organization like the original OpenAI gets formed that is truly non-for-profit and support the community, I believe it can beat big players. I think only Meta is the company that can support such an organization at the moment.

To counteract your point, OpenAI was a "truly non-for-profit"

Re: ChatGPT vs. open source on harder tasks

#15

Earlier quoted context omitted.

langchain caused more problems than it fixed. Really ugly software.

Please elaborate. So far it seems promising to me, but I started using it just recently.

for example see this:

https://itsbehnam.com/DetachGPT-Switching-to-Open-Source-Unc...

Re: ChatGPT vs. open source on harder tasks

#16

I'm employed by Microsoft, but not in any relation to the authors of this tool or this post. First time I saw github.com/mirosoft/guidance was on the HN front page, actually :) I've been waiting for a tool exactly like guidance, something that lets me compose prompts the way I think about prompts. Langchain was too much of a headache for me. Playing with guidance, it feels like this is how programmatically interactin…

are we going to now invent a programming language that communicates with LLMs? I find that funny

I'm not an LLM, but I am a native English speaker and English is pretty awful for conveying information succinctly. The lengths you must go with prompts to ensure you're getting the right response is extremely similar to the lengths you go to when delegating something to someone or describing requirements to a project. I wonder if whatever language we end up using to "converse" with LLMs would have value within an organization/email/Jira.

Re: ChatGPT vs. open source on harder tasks

#17
post #2

I wish the comparison was with GPT-4 instead of GPT-3.5. When comparing model intelligence, I wish that only the most capable models were compared.

Me too. GPT-4 is much much better at following the given instructions to the letter. If you specify a format you want to receive your response in it will almost always follow that format, while 3.5 will very often fail to do so.

Edit: out of curiosity I posted the meeting summarization example to GPT-4 and indeed it produced both the relevant snippets and the correct answer to the stated question.

Re: ChatGPT vs. open source on harder tasks

#18

Earlier quoted context omitted.

are we going to now invent a programming language that communicates with LLMs? I find that funny

I'm not an LLM, but I am a native English speaker and English is pretty awful for conveying information succinctly. The lengths you must go with prompts to ensure you're getting the right response is extremely similar to the lengths you go to when delegating something to someone or describing requirements to a project. I wonder if whatever language we end up using to "converse" with LLMs would have value within an or…

> I’m not an LLM, but English is already pretty awful for conveying information succinctly. The lengths you must go with prompts to ensure you’re getting the right response is extremely similar to the lengths you go to when delegating something to someone or describing requirements to a project. I wonder if whatever language we end up using to “converse” with LLMs would have value within an organization/email/Jira.

People already adopt context-specific specialized languages in contexts like that for clarity, what will probably happen is that LLMs will get better at adapting or being adapted to specific contexts.

Re: ChatGPT vs. open source on harder tasks

#19

I'm employed by Microsoft, but not in any relation to the authors of this tool or this post. First time I saw github.com/mirosoft/guidance was on the HN front page, actually :) I've been waiting for a tool exactly like guidance, something that lets me compose prompts the way I think about prompts. Langchain was too much of a headache for me. Playing with guidance, it feels like this is how programmatically interactin…

are we going to now invent a programming language that communicates with LLMs? I find that funny

Why do you find that funny?

Re: ChatGPT vs. open source on harder tasks

#20
post #2

I wish the comparison was with GPT-4 instead of GPT-3.5. When comparing model intelligence, I wish that only the most capable models were compared.

The authors share your wish:

> While Vicuna is somewhat comparable to ChatGPT (3.5), we believe GPT-4 is a _much_ stronger model, and are excited to see if open source models can approach _that_.

Post reply on HN