Is there a noscript/basic (x)html prompt somewhere?
Phind Model beats GPT-4 at coding, with GPT-3.5 speed and 16k context
151–160 of 358 posts
Re: Phind Model beats GPT-4 at coding, with GPT-3.5 speed and 16k context
#152Earlier quoted context omitted.
We get 100 tokens a second with batch size 1. Those 2000+ figures are for large batches.
Ah, that's fair, and faster than any of the LMDeploy stats for batch size 1; nice work! Using an H100 for inference, especially without batching, sounds awfully expensive. Is cost much of a concern for you right now?
I don't think they're saying they're doing batch size of 1, just giving performance expectations of user facing performance
Re: Phind Model beats GPT-4 at coding, with GPT-3.5 speed and 16k context
#153Phind has been pretty nice to use for some rubber ducking with C#. Only disadvantage is using Discord to wall communication.
Re: Phind Model beats GPT-4 at coding, with GPT-3.5 speed and 16k context
#154I just spent a few minutes doing a comparison between Phind and GPT-4 for a very high-level question on a distributed job queue. I gave them both the same fairly vague sketch of a kind of system I would like to build. Here are my impressions: In the positives of Phind: * Phind was able, even eager, to recommend specific libraries relevant to the implementation. The recommendations matched my own research. GPT-4 takes…
Do you have custom instructions? Everyone needs to mention and post prompts else entirely antidotal
Re: Phind Model beats GPT-4 at coding, with GPT-3.5 speed and 16k context
#155Earlier quoted context omitted.
I've been thinking along the same lines. The token window IMO should be a conceptual inverted pyramid, where there most recent tokens are retained verbatim but previous iterations are compressed/pooled more and more as the context grows. I'm sure there's some effort/research in this direction. It seems pretty obvious.
But some of the earlier tokens are also the most important ones, right? Like the instructions and rules you want it to follow.
They are. Moreover, the idea that AI companies are missing and/or not implementing this “obvious” tactic is hilarious. Folks, these approaches have profound consequences for training and inference performance. Y’all aren’t pointing out some low hanging fruit here, lol
Re: Phind Model beats GPT-4 at coding, with GPT-3.5 speed and 16k context
#156I just spent a few minutes doing a comparison between Phind and GPT-4 for a very high-level question on a distributed job queue. I gave them both the same fairly vague sketch of a kind of system I would like to build. Here are my impressions: In the positives of Phind: * Phind was able, even eager, to recommend specific libraries relevant to the implementation. The recommendations matched my own research. GPT-4 takes…
Do you have custom instructions? Everyone needs to mention and post prompts else entirely antidotal
We support custom instructions at https://phind.com/profile.
Re: Phind Model beats GPT-4 at coding, with GPT-3.5 speed and 16k context
#157Earlier quoted context omitted.
Ah, that's fair, and faster than any of the LMDeploy stats for batch size 1; nice work! Using an H100 for inference, especially without batching, sounds awfully expensive. Is cost much of a concern for you right now?
I don't think they're saying they're doing batch size of 1, just giving performance expectations of user facing performance
Yeah, and this is basically what I was asking.
100 tokens/s on the user's end, on a host that is batching requests, is very impressive.
Re: Phind Model beats GPT-4 at coding, with GPT-3.5 speed and 16k context
#158I know it isn't popular, but I wish there was a way to use this inside Emacs. Or, vim. I just don't want to use VS Code anymore.
In Vim, I tried to assign a shortcut to send the selected text to Phind (or any other LLM) and came up with this: :' y|call system('firefox ?q='.shellescape(@*).' &') The only problem left is that the text is not urlencoded. There probably is some elegant way to urlencode it. But I did not come up with one yet.
https://stackoverflow.com/a/76488059 claims to have one, though it's not explained.
Re: Phind Model beats GPT-4 at coding, with GPT-3.5 speed and 16k context
#159To the folks in this thread comparing the model with GPT-4, are you comparing it with GPT-4 in ChatGPT, or with GPT-4 on Phind? Because it should be the latter for a fair comparison. The Phind response seems to be heavily based on the top search results, which may affect the quality of the response.
(An even more interesting question would be to compare ChatGPT GPT-4 with Phind GPT-4, i.e. GPT-4 with relevant web results in context.)
Re: Phind Model beats GPT-4 at coding, with GPT-3.5 speed and 16k context
#160First off, congrats on building such a cool product. I love that I can just "jump into it" which is great.
Note that I'm not really a power user of these GPT style tools- here are my questions:
Is it possible to get right to the code without the ELI5 and general information?
Do you guys offer an API? I was browsing on my small iphone so maybe I missed this info.
Could you give an overview for someone like me how something like phind works technically? You mentioned those H100s, but at a very high level without revealing any "secret sauce" how does this GPT work from my input to getting a response?
Good luck!