Phind Model beats GPT-4 at coding, with GPT-3.5 speed and 16k context

I just spent a few minutes doing a comparison between Phind and GPT-4 for a very high-level question on a distributed job queue. I gave them both the same fairly vague sketch of a kind of system I would like to build. Here are my impressions:

In the positives of Phind:

* Phind was able, even eager, to recommend specific libraries relevant to the implementation. The recommendations matched my own research. GPT-4 takes some coaxing to get it to recommend libraries. Phind also provided sample code using the libraries it recommended.

* Phind provides copious relevant sources including github, stackoverflow and others. This is a major advantage, especially if you use these AI assistants as a jumping off ground for further research.

* Phind provides recommendations for follow on questions that were very good. One suggestion to the Phind team: don't remove the alternate follow on questions once I select one. A couple of times it recommended a few really good follow up questions but as soon as I selected one the others disappear.

In the positives of GPT-4:

* GPT-4 gave better answers. This is my subjective opinion (obviously) but if I was interviewing two candidates for a job position and using my question as the basis for a systems-design interview then GPT-4 was just overall better. In many cases it added context beyond my question, recommending things like logging and metrics for example. It seemed to intuit the "question behind the question" in a much better way than the literal interpretation of Phind. This is probably highly case-dependent, sometimes I just want an answer to my explicit question. But GPT-4 seemed to understand the broader context of the question and replied with that in mind leading to an overall more relevant response.

* GPT-4 handled follow-up questions better. This is similar to the previous point - but GPT-4 gave me the impression of narrowing down the scope of the discussion based on the context of my follow-up question. It seemed to "understand" the direction of the conversation in a way that felt like it was following context.

NOTE: this was not a test on coding capability (e.g. implementing algorithms) but on using these AI coding assistants as sounding boards for high-level design and architecture decisions.

We get 100 tokens a second with batch size 1. Those 2000+ figures are for large batches.

I gave it two tries, GPT-4 was much better in both cases. Tried with two Leetcode questions. It came back with an empty response for one, and provided a worse code (O(n2) solutions when it can be done with linear time) for the other one.

GPT-4 on the other hand provided a good answer for both questions. Also I guess the UI is buggy w.r.t code formatting, it things the following line is a code and switches to a code block.

``` You are given an array prices where prices[i] is the price of a given stock on the ith day. ```

The only downside for GPT-4 for me right now, is its slowness.

I suggest you try enabling "Ignore search results" from the model dropdown for these types of questions. The web results can be distracting for the model for Leetcode-type questions.

How have you liked using TensorRT-LLM? Did you come from faster-transformers, vLLM, LMDeploy, TGI, something else?

We started migrating to it the day it came out, very glad to have it, but lots of little annoyances along the way. Biggest one has been loading our model repository; having to hardcode the location of the engine file means we can't use the built-in ways Triton has for downloading from GCS!