41–50 of 186 posts
text-davinci-001 and text-davinci-002 were trained through FeedMe and SFT, while text-davinci-003 was RLHF; the models themselves have more variance at high temperature.
Off topic > 3 months later, reading a paper while on board a boring flight home, I have my answer. I noticed people from hacker news routinely read scientific papers. This is a habit I envy but don't share. Any tips or sites for someone interested in picking up more science papers to read.
Highly doubt that. It’s very hard to actually read scientific papers when you are not actively doing research.
You can’t just read a research paper in isolation. It’s next to useless. You need to understand its context, where it stands with regard to its sources and what it brings which is actually new and valuable. It’s nearly impossible to do properly if you are not fully immersed in a research subject.
I don’t even know how you would scheme introduction and sources to filter articles which are immediately obviously useless without being immersed in a field.
I guess you can obviously go though lists of papers which have be deemed worthwhile by someone else or got prices. That solves the filtering issue but then nearly every time you will be better served reading a text book presenting the ideas in said papers.
I fully expect the HN readership to contain a significant amount of students and actual researchers which explain why you encounter people reading papers but these people aside I would be surprised if the habit is common.
This is _excellent_ work, I've been adamantly against MoE for a set of reasons, this is the first compelling evidence I've seen that hasn't been on Substack or a bare repeating of rumor. I had absolutely no idea GPT4 was nondeterministic and I use it about 2 hours a day. I can see why a cursory looking wasn't cutting it, they "feel" the same in your memory, a lot of similar vocab usage, but are formatted entirely dif…
GPT4 web chat for two hours a day? I buy that. Using the API repeatedly for the same inputs, eg developing a program, and the non-determinism is hard to miss.
Not sure I understand the excerpt from the referenced paper. Is it saying that part of its more-efficient inferencing relies on mixing tokens from completely-separate inputs – eg, from other users? And then, depending on what other inputs chance into the same grouping, the relative assignment-to-'experts' varies, and thus the eventual completions? If so, I'd see that as not just introducing non-determinism, but also…
Off topic > 3 months later, reading a paper while on board a boring flight home, I have my answer. I noticed people from hacker news routinely read scientific papers. This is a habit I envy but don't share. Any tips or sites for someone interested in picking up more science papers to read.
You learn pretty quickly that if you want answers, it's better to just go straight to the source, rather than have it filtered through someone else, where the message can (and often does) get twisted.
What are you interested in reading about? Maybe some people can recommend you some papers to start with.
Earlier quoted context omitted.
Build the habit. When google doesn't return a good result to a specific question, switch to scholar.google.com and start reading abstracts. Everything may seem like an opaque maze at first, but just keep reading and patterns start emerging quickly and become useful.
I don't mind reading research papers, but they're really annoying to read on a phone screen. I remember a few years ago, an HN comment shared a link to some tool that could convert a PDF to single column text and make it more readable on a phone screen, but I can't find it. Anyone remember this or have the link?
+1. I've already read probably 100 research papers this year in search of solutions to some technical problems, mostly while lying on bed with a tablet. I won't read as much without it.
Off topic > 3 months later, reading a paper while on board a boring flight home, I have my answer. I noticed people from hacker news routinely read scientific papers. This is a habit I envy but don't share. Any tips or sites for someone interested in picking up more science papers to read.
Off topic > 3 months later, reading a paper while on board a boring flight home, I have my answer. I noticed people from hacker news routinely read scientific papers. This is a habit I envy but don't share. Any tips or sites for someone interested in picking up more science papers to read.
Step 1. Find papers you're interested in Step 2. Open them Step 3. read them