Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Phind Model beats GPT-4 at coding, with GPT-3.5 speed and 16k context

phind.com

Post reply on HN
Print view View original

291–300 of 358 posts

Re: Phind Model beats GPT-4 at coding, with GPT-3.5 speed and 16k context

#291

Post by webappguy »

rushingcreek wrote: post #241
CSMastermind wrote: post #238

I was just testing to see the comparison and ran into a message saying I was out of GPT-4 queries, despite having deliberately selected Phind as my model. Now I'm confused if the results I was seeing really were from a different model than GPT-4 or not.

Ah you were likely using the Pair Programmer. The Phind Model is not yet supported in the Pair Programmer, only the default search mode. Please try again using that.

If this is the case the ux is off, as pair programmer is on and the model clearly says Phind. I'd recommend making the model more clear before and after search, and instead of adding a bubble tag to a search box in the prompt response list, change the background color of search query boxes

Re: Phind Model beats GPT-4 at coding, with GPT-3.5 speed and 16k context

#292

Post by [deleted] »

slowhadoken wrote: post #256
make3 wrote: post #39

Earlier quoted context omitted.

What they're citing isn't what the LLM "scraped", it's what the retrieval model fed to the LLM. You're not guaranteed that it's what it actually used to give you the output, and it's also definitely not all the text that it used to get appropriate knowledge to generate the answer, as this is split over whatever millions of examples for the language and for human language in a non human-understandable way

I've heard this coldtake before but OpenAI's source code isn't open to academic scrutiny. So I don't understand why some people are so confident about how it works. It's certainly not magic and Phind seems to be capable of it citation.

[deleted]

Re: Phind Model beats GPT-4 at coding, with GPT-3.5 speed and 16k context

#293

Post by halfjoking »

rushingcreek wrote: post #23
tinco wrote: post #20

Will you be offering the model as an API service? The product my team is working on would benefit from a significantly faster and possibly better performing model than GPT-4. If you're planning on keeping pace with competitive models we'd love to integrate the use of your model into our service.

If we get enough demand that's definitely something we'll consider. We're still a small team, however, and we do everything in our power to not get distracted from our main mission.

If you offer an API you don't have to maintain a Visual Studio plugin. Trying to compete with tools like Cursor would be the real distraction.

And Cursor is just the start - there will be innovative workflows built on top of APIs you can't predict. You're missing out not having developers build an ecosystem for you.

Re: Phind Model beats GPT-4 at coding, with GPT-3.5 speed and 16k context

#294

Post by webappguy »

rushingcreek wrote: post #15
seidleroni wrote: post #12

The results I get are so-so. The rubric I use to evaluate coding LLM's is to ask it to create a Python script that determines if the contents of a given directory have been changed since the last time the script was run. This should be done recursively and handle files being added, removed, or modified and be based off the contents of the files and not the timestamps. When I ask it as one statement it performed ok, b…

Thanks for the feedback. We're working on improving consistency and precise instruction following in followups.

If you can make this best in class for code outside just human eval, wow, that's the differentiator. Add cursor, replit and vscode support after. But best in class for code, it would be my daily driver

Re: Phind Model beats GPT-4 at coding, with GPT-3.5 speed and 16k context

#295

Post by jstummbillig »

alex-moon wrote: post #289

I tried my standard "trick" question I use for LLMs: "Give me five papers with code demonstrating the state of the art of machine learning which uses geospatial data (e.g. GeoJSON) as both input and output." There is no such state of the art. My hand-wavey understanding is that GIS data is non-continuous, which makes it useless for transformers, and also contextual, which makes it useless for anything else. Will defe…

ChatGPT 4 with web browsing: https://chat.openai.com/share/19a425b5-ed37-469e-860d-65ee70...

ChatGPT 4 without web browsing: https://chat.openai.com/share/7e11b4a6-52f2-441a-8614-7266c3...

Re: Phind Model beats GPT-4 at coding, with GPT-3.5 speed and 16k context

#296

Post by alex-moon »

jstummbillig wrote: post #295
alex-moon wrote: post #289

I tried my standard "trick" question I use for LLMs: "Give me five papers with code demonstrating the state of the art of machine learning which uses geospatial data (e.g. GeoJSON) as both input and output." There is no such state of the art. My hand-wavey understanding is that GIS data is non-continuous, which makes it useless for transformers, and also contextual, which makes it useless for anything else. Will defe…

ChatGPT 4 with web browsing: https://chat.openai.com/share/19a425b5-ed37-469e-860d-65ee70... ChatGPT 4 without web browsing: https://chat.openai.com/share/7e11b4a6-52f2-441a-8614-7266c3...

ChatGPT 4 seems to be better than it was when I was using it (mere months ago)!

Re: Phind Model beats GPT-4 at coding, with GPT-3.5 speed and 16k context

#297

Post by hasoleju »

I don't use LLMs in my workflow frequently. When I do I have a hard time making sense out of the very specific and long answers to my questions. Especially if I don't know the answer it is hard to figure out if the long answer of the model indicates the right direction or misses my point completely.

Maybe I'm not knowledgeable enough. But asking questions I already know the answer of has no real life use case other than testing the model. Which of course might be a valid use case for some.

Having a way to let the user specify his own level of knowledge might help to receive answers that are better tailored to the user asking the question.

Re: Phind Model beats GPT-4 at coding, with GPT-3.5 speed and 16k context

#299

Post by lysecret »

hasoleju wrote: post #297

I don't use LLMs in my workflow frequently. When I do I have a hard time making sense out of the very specific and long answers to my questions. Especially if I don't know the answer it is hard to figure out if the long answer of the model indicates the right direction or misses my point completely. Maybe I'm not knowledgeable enough. But asking questions I already know the answer of has no real life use case other t…

Have you tried custom instructions ? I use this:

My dad always used to say: Everything that can be said, can be said simply. I prefer top-down structured, short and thoughtful responses.

Re: Phind Model beats GPT-4 at coding, with GPT-3.5 speed and 16k context

#300

Post by raducu »

rushingcreek wrote: post #143
ta8645 wrote: post #141

Earlier quoted context omitted.

How do you enable "Ignore Web Context"? I don't see that option anywhere on the page you linked, am I just being blind?

It's in the model dropdown under the search bar.

You mean "Ignore Search Results" ?
Post reply on HN