Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Phind-405B and faster, high quality AI answers for everyone

phind.com

Post reply on HN
Print view View original

111–120 of 163 posts

Re: Phind-405B and faster, high quality AI answers for everyone

#112

Post by okasaki »

mdp2021 wrote: post #105
Retr0id wrote: post #89

Earlier quoted context omitted.

> As an AI assistant, I should be more careful I hate this kind of thing so much.

Absolutely. Behaviour that in normal life in clean societies would be "eliciting violence": automated hypocritical lying, apologizing in form and not in substance, making statements based on fictional value instead of truthfulness...

What?

Re: Phind-405B and faster, high quality AI answers for everyone

#113

Post by shultays »

Recently I asked an AI following question

  const MyClass& getMyClass(){....}
  auto obj = getMyClass();

  this makes a copy right?
And it was very confident about it not making a copy. It thinks auto will deduce the type as a const ref and not make a copy. Which is wrong, you need auto& or const auto& for that. I asked it if it is sure and it was even more confident.

Here is the godbolt output https://godbolt.org/z/Mz8x74vxe . You can see the "copy" being printed. And you can also see you can call non-const methods on copied object, which implies it is a non-const type

I asked the very same question to phind and it gave the same answer https://www.phind.com/search?cache=k3l4g010kuichh9rp4dl9ikb

How come two different AIs, one was supposed to be specialized on coding, fails in such a confident way?

Re: Phind-405B and faster, high quality AI answers for everyone

#114

Post by prosunpraiser »

shultays wrote: post #113

Recently I asked an AI following question const MyClass& getMyClass(){....} auto obj = getMyClass(); this makes a copy right? And it was very confident about it not making a copy. It thinks auto will deduce the type as a const ref and not make a copy. Which is wrong, you need auto& or const auto& for that. I asked it if it is sure and it was even more confident. Here is the godbolt output https://godbolt.org/z/Mz8x74…

You prove the point that these are just token generation machines whose output is psuedo-intelligent. It’s probably not there yet to be blindly trusted.

Re: Phind-405B and faster, high quality AI answers for everyone

#116

Post by kreyenborgi »

druskacik wrote: post #102
CSMastermind wrote: post #99

Earlier quoted context omitted.

These are the types of questions I want to ask it: > What degrees are held by each of the current Fortune 100 CEOs? > What job did each of the current NFL GMs hold before their current position? > Which genre would each of the current Billboard Hot 100 songs be considered part of? > How many recipients of the Presidential Medal of Freedom were born outside of the US? > Which US car company has the most models in thei…

There is a method that could help immensely when answering questions like these. E.g. some of these question may be answered quite quickly using WikiData [0] (answer to question about the recipients of Medal of Freedom, query written with the help of Claude), instead of just scraping and compiling information from potentially hundreds of websites. I believe this idea is quite under-explored compared to just blindly p…

Yeah, I've used gpt to create wikidata queries for me, it worked great :-)

Re: Phind-405B and faster, high quality AI answers for everyone

#117

Post by magicalhippo »

ithkuil wrote: post #109
magicalhippo wrote: post #104

Earlier quoted context omitted.

I've been playing with Gemma locally, and I've had some success by telling it to answer "I don't know" if it doesn't know the answer, or similar escape hatches. Feels like they were trained with a gun to their heads. If I don't tell it it doesn't have to answer it'll generate nonsense in a confident voice.

The models weights are tuned towards the direction that would cause the model to best fit the training set. It turns out that this process makes it useful at producing mostly sensible predictions (generate output) for text that is not present in the training set (generalization). The reason that works is because there are a lot of patterns and redundancy in the stuff that we feed to the models and the stuff that we a…

In the Physics of Language Models talk[1], he argues that the model knows it has made a mistake, sometimes even before it has made it. Though apparently training is crucial to make the model be able to use this constructively.

That being said, I'm aware that the model doesn't reason in the classical sense. Yet, as I mentioned, it does give me less confabulation when I tell it it's ok not to answer.

I will note that when I've tried the same kind of prompts with Phi 3 instruct, it's way worse than Gemma. Though I'm not sure if that's just because of a weak instruction tuning or the underlying training as well, as it frequently ignores parts of my instructions.

[1]: https://www.youtube.com/watch?v=yBL7J0kgldU

Re: Phind-405B and faster, high quality AI answers for everyone

#120

Post by renewiltord »

shultays wrote: post #113

Recently I asked an AI following question const MyClass& getMyClass(){....} auto obj = getMyClass(); this makes a copy right? And it was very confident about it not making a copy. It thinks auto will deduce the type as a const ref and not make a copy. Which is wrong, you need auto& or const auto& for that. I asked it if it is sure and it was even more confident. Here is the godbolt output https://godbolt.org/z/Mz8x74…

One of the oldest tricks to make LLMs perform better is to ask them to "think step by step". I asked your question to Claude with that one

    ```
    const MyClass& getMyClass(){....}
      auto obj = getMyClass();
    ```

    Does this make a copy. Let's think step by step.
This might help you if you're trying to get these to help you more often.
Post reply on HN