Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

ChatGPT vs. open source on harder tasks

github.com

Post reply on HN
Print view View original

1–10 of 70 posts

Re: ChatGPT vs. open source on harder tasks

#3

Post by furyofantares »

tikkun wrote: post #2

I wish the comparison was with GPT-4 instead of GPT-3.5. When comparing model intelligence, I wish that only the most capable models were compared.

I agree, I only use 4 because by far the thing that matters most for my usage is the quality of the output at this stage in the game.

But 3.5 is generally available, fast, and costs a WHOLE LOT less for API users; those are real capabilities that matter though by "capable" you only meant the quality of the output. It's worth comparing to 3.5 also.

Re: ChatGPT vs. open source on harder tasks

#5

Post by chrgy »

Open source will be behind always since it is harder to organize a large group of people and once it publishes it will be copied by big players, however if an organization like the original OpenAI gets formed that is truly non-for-profit and support the community, I believe it can beat big players. I think only Meta is the company that can support such an organization at the moment.

Re: ChatGPT vs. open source on harder tasks

#6

Post by andy_xor_andrew »

I'm employed by Microsoft, but not in any relation to the authors of this tool or this post. First time I saw github.com/mirosoft/guidance was on the HN front page, actually :)

I've been waiting for a tool exactly like guidance, something that lets me compose prompts the way I think about prompts. Langchain was too much of a headache for me. Playing with guidance, it feels like this is how programmatically interacting with a llm should be. Also, really happy to see the MIT license.

I'd love to see ports to other languages.

Re: ChatGPT vs. open source on harder tasks

#8

Post by yokoprime »

tikkun wrote: post #2

I wish the comparison was with GPT-4 instead of GPT-3.5. When comparing model intelligence, I wish that only the most capable models were compared.

GPT-3.5 is good enough for many cases. I know Microsoft for their Bing chat implementation is using a pre-prosessor to select the most appropriate model to use.

Re: ChatGPT vs. open source on harder tasks

#9

Post by doctoboggan »

furyofantares wrote: post #3
tikkun wrote: post #2

I wish the comparison was with GPT-4 instead of GPT-3.5. When comparing model intelligence, I wish that only the most capable models were compared.

I agree, I only use 4 because by far the thing that matters most for my usage is the quality of the output at this stage in the game. But 3.5 is generally available, fast, and costs a WHOLE LOT less for API users; those are real capabilities that matter though by "capable" you only meant the quality of the output. It's worth comparing to 3.5 also.

At first I was skeptical of the difference between 3.5 and 4 but there really is a large gap in the models understanding of complex issues between them. I make a habit of always selecting gpt4 but forgot to yesterday. Within 3 responses I could immediately tell I was interacting with a less capable model.
Post reply on HN