Viewing profile — rnosov

The report cites both GPT-3.5 and GPT-4 scores on page 7 [1]. I've checked the numbers and they compare FreeWilly2 to GPT-3.5. For example, HellaSwag score of 85.5% corresponds to …

> we are training a full suite of models, with the first becoming available in the coming weeks. Sounds like they already have the compute and began training.

For example, you can look up semantically relevant ( to a user query ) paragraphs from some internal document. Then, include them in the LLM context so it would know how to answer …

row 7 is the name of the dataset, you might need to load it yourself

I'm going through the dataset with your datasette tool and it looks like it might be a good idea to clean things up a bit. There are many duplicates[1], creepypastas[2] and other s…

They are leveraging Apple’s Metal Performance Shaders[1] not the neural engine. From the chart, it looks like you might get ~20x max boost on inference over plain CPU. Obviously, i…

It looks like you're on python 3.11 which has some issues with Pytorch. Downgrade to python 3.10 and try running it again.

A question to the author. Can you perform an ablation study with respect to the chunks? In other words, if you put in the context irrelevant/random chunks from the document would t…

1) I've looked at both codebases and this one is definitely a derivative of the nanoGPT. You can compare all three implementations yourself as they are actually surprisingly compac…

I might be missing something but it looks to me that actually running this "open" model requires special hardware only accessible with a cloud subscription with 60 000 USD / week m…

The FTC link has an example of the only newspaper in town refusing to deal with customers who are also running ads on a radio station. Do you think if the newspaper dressed such re…

That would be anticompetitive practice that is actually against the law in many countries[1]. In the unlikely event of OpenAI ever engaging in such things they will be sued into ob…

ToS are not the law. It would be similar to your power company claiming copyright over the code written using "their" electricity. Not going to happen. I wouldn't be too concerned.…

They are instruction tuning it using the dataset released by stanford-alpaca team. The dataset itself is synthetic (created using GPT-3) and somewhat noisy and in my view can be ea…

This has nothing to do with facebook. The foundational model here is GPT-J which is opensource and safe to use. Sadly, it is inferior to state-of-the-art models such as LLaMA.

This is a really troubling development. Anyway, kudos to you for standing up to the man.

There are benchmarks in the original LLaMA paper[1]. Specifically, on page 4 LLaMA 13B seems to beat GPT-3 in BoolQ, HellaSwag, WinoGrande, ARC-e and ARC-c benchmarks (not by much …

You describe supervisor approach as: > One common suggestion is to have another LLM look at the input intently with the instruction to determine whether it is malicious. Preflight …

Quite an interesting article. The Vice example is hilarious. But for all doom and gloom you haven't addressed the most obvious mitigation - Preflight Prompt Check [1]. It would be …

There is a big AI division of Google located in the UK (DeepMind). Could be related to that.

> https://arxiv.org/abs/2210.17323 I've read the paper and to be honest I'm not sure what to make of it. Their headline benchmark is perplexity on WikiText2 which would not be part…

I don't think that the model they are using is private. You can download it[1] and run yourself. Looks like it is GPT-NeoX which was fine tuned using the open source LAION OIG data…

You might want to tune the sampler. For example, set it to a lower temperature. Also, the 4bit RTN quantisation seems to be messing up the model. Perhaps, the GPTQ quantisation wil…

The post has a workaround for the PyTorch issue with Python 3.11. If you follow the repo instructions it will give you some rather strange looking errors.

Seconded. Do write it up. I see vast.ai listing interruptible instance with a single A100 80GB at $1/hour which is pretty reasonable. ChatGPT plus is $20/month which would be rough…