Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The Rise of the AI Engineer

latent.space

Post reply on HN
Print view View original

131–140 of 159 posts

Re: The Rise of the AI Engineer

#131

Post by mercurialsolo »

With the rise of LLM's there is definitely a new skillset needed in the bag of engineers - one who know how to align/instruct LLM's towards an objective. This new role is essentially a mix of standard programming logic + ability to instruct LLM's using language (natural language programming). That's been the primary difference I have seen in this emergent field of designing agents.

Most agents need some form of formal instructions tied with designing prompts - which are a mix between formal programming and natural language. This needs a slightly different skill set from the traditional backend engineer or services engineer. This is not a ML or data role either.

Rather than just a prompt engineer, the case for which could be made like a specialist designer - this role maybe more akin to a full-stack developer role who knows how to align LLM's or Diffusion models towards a multi-step goal.

Re: The Rise of the AI Engineer

#133

Post by [deleted] »

mercurialsolo wrote: post #131

With the rise of LLM's there is definitely a new skillset needed in the bag of engineers - one who know how to align/instruct LLM's towards an objective. This new role is essentially a mix of standard programming logic + ability to instruct LLM's using language (natural language programming). That's been the primary difference I have seen in this emergent field of designing agents. Most agents need some form of forma…

[deleted]

Re: The Rise of the AI Engineer

#134

Post by __loam »

swyx wrote: post #57
cj wrote: post #52

Earlier quoted context omitted.

> what else do you need to see to believe? (geniune question) It's not that I don't believe in LLMs. It's that the level of media attention AI is getting isn't backed by the same level of real world use cases, yet. (All of the use cases you listed are awesome, no denying that, but I don't think those alone justify the amount of hype in mainstream media)

i mean, theres nothing i can do about mainstream media hype, but i guess my main point is this is a growing field with real money and utility behind it, and so will professionalize. if I am correct on that then AI Engineer will be a thing (because it is Least Bad title for the thing)

You are actively contributing to the hype cycle lol.

Re: The Rise of the AI Engineer

#135

Post by __loam »

lispisok wrote: post #129
cbm-vic-20 wrote: post #7

I'm a natural skeptic, and I believe we're still on the rising edge of the "AI" hype cycle. Five years ago, it was "blockchain", and everyone was trying to ram blockchain into everything, attracting lots of VC and media attention, etc. It seems that blockchain is beyond the honeymoon phase: I haven't seen an NFT or even a Bitcoin headline in HN for a while. So I'm trying to wrap my head around what an "AI Engineer" i…

A lot of people trying to refute you claiming the latest AI hype has actual value. The new models people are raving about do have value way above what blockchain had but I'd argue the AI hype is also way higher. Overall I'd say the gap between AI hype and AI value is bigger than blockchain despite blockchain value basically being zero if you dont count transactions for illicit goods.

ChatGPT was quite literally optimized to be convincing to humans thanks to HFRL. I think it's going to be a huge negative externality thanks to the amount of ai noise it's going to be pumping onto the internet for the next couple of years. It's going to get harder and harder to connect with real humans and get information that we know is true rather than just proliferated by an automated system that knows nothing about quality. That negative impact might be even worse than blockchain if we see fewer and fewer people participating in creative skills, or even being able to make a living thanks to the lies the AI industry has sold to MBAs.

Re: The Rise of the AI Engineer

#136

Post by __loam »

visarga wrote: post #103
cbm-vic-20 wrote: post #7

I'm a natural skeptic, and I believe we're still on the rising edge of the "AI" hype cycle. Five years ago, it was "blockchain", and everyone was trying to ram blockchain into everything, attracting lots of VC and media attention, etc. It seems that blockchain is beyond the honeymoon phase: I haven't seen an NFT or even a Bitcoin headline in HN for a while. So I'm trying to wrap my head around what an "AI Engineer" i…

When your task is too nuanced to be described in a prompt and a few demonstrations you need to use one of the fine-tuning scripts to bake into the model much more supervised data. That's still doable for AI Engineers, prepare data, call LoRA fine tuning script, deploy model. The problem is having a good supervised training set. But recently it can be generated with LLMs + plugins and language chains, basically amplif…

I've been encountering machine learning on and off for the last 8 years or so. The idea that people are generating data with LLMs to then turn around and train LLMs without a hint of irony is hilarious and astonishing to me.

Re: The Rise of the AI Engineer

#137

Post by __loam »

buffalobuffalo wrote: post #62
cbm-vic-20 wrote: post #7

I'm a natural skeptic, and I believe we're still on the rising edge of the "AI" hype cycle. Five years ago, it was "blockchain", and everyone was trying to ram blockchain into everything, attracting lots of VC and media attention, etc. It seems that blockchain is beyond the honeymoon phase: I haven't seen an NFT or even a Bitcoin headline in HN for a while. So I'm trying to wrap my head around what an "AI Engineer" i…

I'm also a skeptic, but for a slightly different reason. There are currently two types of business use cases that seem to be the focal point of this generation of AI. 1) Tooling. This one I think will probably bear fruit. It will likely result in huge productivity gains (I mean, it already has for me). But i don't know if it will result in a paradigm shift. 2) Agents. This is where most of the hype is focused. The id…

I disagree completely with your last point. Flooding the internet with automated bullshit and people locking down their data because they're worried they'll get scraped is going to be a hugely negative consequence, to say nothing of what this will do to people like artists and writers because of the perception of this technology that the people paying them are going to have.

Re: The Rise of the AI Engineer

#138

Post by __loam »

sanderjd wrote: post #34
forrestbrazeal wrote: post #3

Hey Shawn! Always enjoy your writing. I think you've done well laying out what a lot of people want "AI Engineer" to mean at this moment in time. My concern (coming fresh out of the absolute semantic nightmare that was the "serverless" community) is that the term AI is so hopelessly overloaded, and has been such a moving target over the years, that it's unlikely that a plurality of people will ever share your mental…

Hmm, I think "the professional practice of building stuff with AI as your pair programmer" is just "software engineering". That is, it's just one more tool to use to do the existing work. We never had "Search Engineering" or "StackOverflow Engineering" to describe the practice of building stuff using web search and stack overflow as tools...

Imagine calling yourself a compiler engineer because you use a compiler.

Re: The Rise of the AI Engineer

#139

Post by __loam »

foobiekr wrote: post #120
z3c0 wrote: post #114

Earlier quoted context omitted.

Heavily disagree, and this just seems like posturing from the other side of things. I understand AI rather heavily, and have been a part of the scene far before the explosion of even the first GPT model. Frankly, I've been quite irked by all the people who claim themselves "ML engineer" when most of what they do is glue huggingface models together. They're doing none of the work required to make models, which is quit…

Strong agreement. What we have right now is a lot of people doing light glue code and pretending it is engineering.

They're building slapdash, code by night bullshit apps that barely fucking work and calling it engineering. When I see those "AI landscape" images with logos from 200 different companies, I know we're in a bubble. A lot of these guys are going to take VC money then crash and burn when someone else builds a marginally better version of their GPT api wrapper. The real winners will be companies like Nvidia cashing in on this hype cycle.
Post reply on HN