Ask HN: Is anyone else bearish on OpenAI?

Anything beyond one off asks is pretty hit or miss at least for me on if what ChatGPT is telling me is correct or not. Write me a complex SQL query that does this, write a python script that will do that, show me the regex that will find these patterns in a string, all of those work really nicely and do save time.

When anything gets more complex than that, I feel like the main value it provides is to see what direction it was trying to approach the problem from, seeing if that makes sense to you, and then asking it more about why it decided to do something.

This is definitely useful, but only if you know enough to keep it in check while you work on something, or worse if you think you know something more than you actually do, you can tell ChatGPT it's wrong and it will happily agree with you (even though it was correct in that case). I've tested both cases: correcting it when it was really wrong, and correcting it confidently when it was actually right. Both times it agreed that it was wrong and regenerated the answer it gave me.

> I've tested both cases: correcting it when it was really wrong, and correcting it confidently when it was actually right. Both times it agreed that it was wrong and regenerated the answer it gave me.

This is the peril of using what really is fundamentally an autocomplete engine, albeit an extremely powerful one, as a knowledge engine. In fact, RLHF favors this outcome strongly; if the human says "this is right", the human doing the rating is very unlikely to uprate responses where the neural net insists they're still wrong. The network weights are absolutely going to get pushed in the direction of responses that agree with the human.

I'm not bearish on OpenAI or AGI in general but I'm extremely meh about it. I'm not chomping at the bit to use it like so many are, and I constantly feel like a huge luddite or something for not being super excited about it.

The value and time it saves makes sense for folks who struggle with a search engine (many) or doing tasks that are typically considered menial, like writing emails or coding boilerplate.

However, if you can grok Google and don't mind doing tasks like that (I personally don't mind coding boilerplate stuff, especially since I can learn how the framework works that way), ChatGPT's value is limited (at least in my experience).

Example: I was struggling with a Terraform issue the other day. I used ChatGPT 4 to help me work out the problem, but the answer it gave was really generic, like mashing a few of the top answers on SO together. It also didn't answer what I needed help with. I knew enough about how Terraform worked to Google for the answer, which I eventually did a few minutes later. I could have kept crystalling my question for ChatGPT until I got what I wanted, but Google was easier.

I'm also not a huge fan of us just being okay with trusting a single and extremely corporate entity (OpenAI) being the de facto arbiter of the truth. At least Google shows you other options when you search for stuff.

All SRE type work. Where it specifically shines is whenever you need to dig deep into linux kernel specifics, networking specifics, code in different languages to handle issues (Rust (which I don't know), Go, Python, Bash, etc...), All sorts of error messages it knows so often what precisely needs to be done to resolve, using and configuring graphing, database, TSDB type queries, formulating a clever tcpdump query for a particular scenario, etc...

Google Used to be able to handle a lot of the non-procedural questions - but somewhere in circa 2022/2023 something started happening to it's results and I started getting back mostly SEO churn, to the point I was going back to using manuals and having to dig in and learn the fundamentals on a lot of things - which is unsustainable if you are touching 30-50 different technology stacks.

Chat changed all that - I can now go 3-5 levels deep in some stack, ask some incredibly nuanced question, get a reasonable answer that points me in the right direction, close off the issue and then move onto the next one.

Random example from today - we have a new product being deployed from a prominent vendor, and they started running into a ton of "out of socket" errors. This product was important to one of our telemetry organizations Q4 OKRs, and they weren't getting traction with the vendor, so they started reaching out internally. I got tagged as someone with a background in networking (though this was clearly not a networking issue) - and started poking around. The first thing that comes to mind of course, is file descriptions, but ulimit -n returned a massive number and "ss -a" showed there were a tiny number of sockets opened (like, 1300 - we have lots of nodes with 500K+ sockets opened - so this really didn't feel like a node or docker container issue).

So - I just reached out to Chat, and we started going back and forth, starting with "In linux, how many open sockets can there be at a time?". What's nice, about that wide open question, is you don't get a single answer, instead you get a briefing on Linux Sockets. File Descriptors, sure, but also memory, port range, TCP/IP stack limits, etc... It starts to lay out a roadmap towards solving the issue, answering the question you were interested in, rather than the one you asked.

I do a bit of back and forth on some scenarios, asking about /proc, and ss, etc.. seeing if I can track anything else out. And then, after spending about 5 minutes, and building context, I ask it "Is there anything else that can cause an error regarding too many sockets with low socket use" - at which point it lays out a number of scenarios, one of which is FD_SETSIZE.

So - we dig into FD_SETSIZE, and immediately that looks interesting - it's the limit that you can use with a select() call, and, even better, I get recommendations to use POLL or EPOLL (which anyone who has ever straced has seen a ton of).

I ask it how to determine FD_SETSIZE, discover it's 1024 on the client, which matches our low socket count, confirm that we should never increase the size of FD_SETSIZE, check the vendor code, see they've got it hard configured to talk with select() instead of poll - we recommend they give us a new build with poll() defined - and voila - problem goes away.

On to the next issue.

Where Chat excels is not in solving or completing anything, or, in fact, even being correct 100% of the time (it clearly isn't) - but it's an endlessly enthusiastic savant intern - frequently wrong in hilarious ways, but always willing to dig in and pull on technical threads and come up with things that I can try with a bunch of rapid iterations, and close off an issue. It's willingness to write code that is 90% correct just reduces the time, and cognitive load of constantly having to do it all yourself.