Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

What it feels like to work in AI right now

robotic.substack.com

Post reply on HN
Print view View original

681–690 of 768 posts

Re: What it feels like to work in AI right now

#681

Post by int_19h »

ThalesX wrote: post #518
jiggawatts wrote: post #395

Earlier quoted context omitted.

You can program GPT in English. Let me repeat that: You can program GPT in English . ENGLISH! You're complaining about the first nuclear test bomb being impractical and uninteresting. How will this change the world? That huge monstrosity had to be affixed to the top of a test gantry and took years of effort by a veritable army of the best and brightest to make! No way it could change war, or geopolitics, or anything.…

> You can program GPT in English. > Let me repeat that: You can program GPT in English. ENGLISH! How? Let me repeat that: How? I had a little script that from time to time parses a list of jobs from a specific board, extracts some categories, inserts them into an SQLite and have a frontend that displays them to me in a way I want. The board has since changed some things which would mean maybe 2 hours of commitment fr…

Let me give you a simple example. I had to deal with a desynced subtitle file recently. I described the exact nature of the desync (in terms like "at point X1 the offset is Y1, and at X2 it is Y2") to GPT-4 and asked it to write me a Python script to fix this. It did require a couple tweaks to run, but when it did, it "just worked".

Re: What it feels like to work in AI right now

#682

Post by int_19h »

JohnFen wrote: post #652
pixl97 wrote: post #648

Earlier quoted context omitted.

For one, GPT-4 requires far less prompt engineering and generally interprets intent better. The advantage of using English (natural language that is), the humans around you tend to speak it. I don't naturally speak powershell. Instead I want a script that searches for particular filenames, under a particular size, between a particular date in a directory path I specify. I told GPT I wanted that and in a few seconds i…

> The advantage of using English (natural language that is), the humans around you tend to speak it. This is true, but of limited utility. English is so bad at this sort of thing that even native-speaking humans are constantly misunderstanding each other. Especially when it comes to describing things and giving instructions. That's why we have more formal languages (even ignoring programming languages) for when we ne…

That's the other nice thing about ChatGPT - if you say it something and it misunderstands, you can correct it by saying, "no, actually, what I meant is ...". Which, again, is how people generally do that kind of thing outside of programming. The advantage is that you're still remaining on a much higher level of abstraction.

As far as formal languages... GPT doesn't know Lojban well, presumably because of its very small presence in the training data (and dearth of material in general). But it would be interesting to see how training on that specifically would turn out.

Re: What it feels like to work in AI right now

#683

Post by int_19h »

cudgy wrote: post #678
jazzyjackson wrote: post #662

Earlier quoted context omitted.

If anyone wants to see the output of GPT4 when asked to define the tables and some sample queries for a hackernews clone in sqlite: https://poe.com/lookaroundyou/1512927999932134

This is a very simple case that doesn’t reflect the complexity of a real project. Like so many attempts before to produce code, using a little effort, it degrades when the complexity level increases even slightly. Once there are more tables, ones that have names which cannot be easily translated from English, it breaks down quickly. These types of tools work ok for brand new projects, but work on existing projects wi…

Here's a more extreme example, using SQL as an API to give model access to game world state to reason about it.

https://gist.github.com/int19h/4f5b98bcb9fab124d308efc19e530...

Note that in this case it isn't even asked to write specific queries for specific tasks - it's just given one high-level task and the schema to work with.

You're right, though, that the effectiveness of this approach depends very much on schema design and things like descriptive table/column names etc (and even then sometimes you have to make it more explicit than a human would need). You really need to design the schema around the capabilities of the model for best results, which makes it that much harder to integrate with legacy stuff. Similarly, not all representations of data work equivalently well - originally, I gave the model direct access to the typed object graph, and it handles that much worse than SQL. So if your legacy software has a data model that is not easy to map to relational, too bad.

On the other hand, GPT-4 is already vastly better at this kind of task than GPT-3.5, so I think we can't assume that this will remain a limitation with larger models.

Re: What it feels like to work in AI right now

#684

Post by Analog24 »

zone411 wrote: post #677
Analog24 wrote: post #657

Earlier quoted context omitted.

Do you have any publications to back up your claims about your work? They seem more than a bit grandiose. If you're ideas are as novel and useful as you say then you should publish them. And I'm sorry, but you're completely wrong about companies recognizing commercial potential. I worked on Alexa for five years, it is a far harder problem than you think. It is nowhere near as simple as "we just weren't looking at the…

No, I have no plans to openly publish any of it. Some of my researcher employees have published their own stuff. I've previously written about how it was a huge commercial mistake for Google and others to openly publish their research, and they should stop. Indeed, now OpenAI has not published a meaningful GPT-4 paper, and DeepMind has also become more cautious. This mistake has cost them billions, and for what? Recr…

The fact that you think creating a writing assistant plugged into Word is equivalent to building a general purpose, always-on voice assistant tells me all I need to know.

Re: What it feels like to work in AI right now

#685

Post by int_19h »

SanderNL wrote: post #605
loandbehold wrote: post #358

Earlier quoted context omitted.

How is "LLM swarm computation" different that single bigger LLM?

The same reason why you don't let Mr Musk do all the work. He can't. One LLM is limited, one obvious limitation is its context window. Using a swarm of LLMs that each do a little task can alleviate that. We do it too and it's called delegation. Edit: BTW, "swarm" is meaningless with LLMs. It can be the same instance, but prompted differently each time.

Context window is a limitation, but have we actually hit the ceiling wrt scaling that? For GPT, you need O(N^2) VRAM to handle larger context sizes, but that is a "I need more hardware" problem ultimately; as I understand, the reason why they don't go higher is because of economic viability of it, not because it couldn't be done in principle. And there are many interesting hardware developments in the pipeline now that the engineers know exactly what kind of compute they can narrowly optimize for.

So, perhaps, there aren't swarms yet just because there are easier ways to scale for now?

Re: What it feels like to work in AI right now

#686

Post by int_19h »

rvz wrote: post #466
fomine3 wrote: post #264

Earlier quoted context omitted.

English speakers tend to miss a huge use case: Translation.

A 'use-case' done worse with LLMs especially with reliability. Translation is already done without a hallucinating LLM and can be done offline. Summarization of existing text is the *only* safe and serious use-case for LLMs.

GPT-4 translates much better than anything else out there, esp. when it comes to idioms and manner of speech.

Re: What it feels like to work in AI right now

#687

Post by ThalesX »

bavila wrote: post #671
ThalesX wrote: post #598

Earlier quoted context omitted.

How is this possible? What do you guys do at work? I haven't had success with neither GPT-4 (did you build your own API calling tool for it? Do you just paste it in their Playground?) nor with GitHub Copilot in actually delivering anywhere close to 90% of the time. It usually misses a whole lot of context. It feels like it would work for perfectly encapsulated small single purpose functions, which of course sounds gr…

I'm doing frontend work (React/TypeScript), so that probably helps since I am working with relatively small components.

I've used it to try and generate some rather small components in React / TypeScript myself, and what it did to arrays of refs with hook calls inside useState hooks initialization function, and the fact that I couldn't get it to fix its issues by doing what people suggest ("just copy paste the error"), or by trying to reason with it, made me not trust it so much. The output code is also pretty low quality in my experience and opinion.

Re: What it feels like to work in AI right now

#688

Post by int_19h »

onion2k wrote: post #324
brookst wrote: post #180

Earlier quoted context omitted.

You’re asking “what’s so big about GUIs? Literally nobody has asked to move a pointer around a screen”. It’s the use cases these thing enable that are important. Today, I wrote a draft product announcement. Only after I was done did I realize I had written it in a really impersonal third person (“users will be able to”). No big deal, but maybe 10-20 minutes of work to make it energetic and second person (“now you can…

That's very cool, and right now it's a good idea, but I strongly suspect GPT only looks clever and does a good job in isolation . If everyone starts using it product announcements will start looking very similar, and they'll lose a lot of their impact. This is definitely the case with cover letters for jib applications. The ones written by GPT appear to be pretty obvious - my guesses could be wrong, but after seeing…

That distinct ChatGPT style is mostly the product of their RLHF, so it's what you get by default if you don't ask for something more specific. But it's fairly easy to tweak the prompt to make it use whatever style you want, including more terse, less apologetic etc. Don't forget that "write about X in the style of Y" was one of the first things that GPT models could reliably do, long before chat.

Re: What it feels like to work in AI right now

#689

Post by int_19h »

glandium wrote: post #443
literalAardvark wrote: post #432

Earlier quoted context omitted.

GPT-4 is dramatically better with everything except speed. The minutia most people in this thread are complaining about is almost completely solved by GPT-4. For example I used it over like 30 minutes to conceptualize, solve, and write some code that draws a graphic for a simple physics problem (0-shot) that I could intuitively understand but had no (physics and math knowledge) tools to calculate properly and it was…

Specific examples don't generalize though. My experience with ChatGPT (GPT-3) is that it is vastly better at dealing with English than Japanese. That is still probably true with GPT-4, but that doesn't mean there hasn't been progress in GPT-4 with Japanese, which a sibling comment says is the case. But it's not because it does better at $task that you can extrapolate to a different one. People have reported GPT-3.5 b…

I cannot speak for Japanese, but I'd say that GPT-4 is better at Russian.

That the model is better in English is no surprise given that most of its training corpus is in English. In fact, based on the sentence structure of the output when it speaks in Russian, it's clear that what's happening there is some kind of real-time translation from English.

That aside, I have yet to see any task on which GPT-4 wasn't at least as good as, or better than, GPT-3.5. I'd love to experiment with that. Do you recall any specific examples?

Re: What it feels like to work in AI right now

#690

Post by int_19h »

burlesona wrote: post #222
cgearhart wrote: post #152

From my perspective it’s just _confusing_ to work in AI right now. We have some massive models that are doing some really neat stuff, and apparently hundreds of millions of people are using them—but I keep wondering: to do _what_, exactly? I’m not asking what the models can do, I’m asking what people want the models to do every day, all the time. I’ve been shown some neat pictures people made that they thought were c…

I don’t think you’re missing anything. I like the “calculator for words” analogy that was posted on HN a few days ago. It doesn’t seem like a revolutionary product, but it does seem like a fundamental innovation which will then unlock many more complex things in subtle ways. Calculators were arguably the predecessor to computers, which are kind of a big deal :)

It's a reckon-mill.

https://gist.github.com/int19h/6fa34a86923cd681396393b21b9ab...

Post reply on HN