Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The Rise of the AI Engineer

latent.space

Post reply on HN
Print view View original

151–159 of 159 posts

Re: The Rise of the AI Engineer

#151

Post by mschuster91 »

oblio wrote: post #150
mschuster91 wrote: post #149

Earlier quoted context omitted.

The thing is it can be used to get rid of the tedious and error-prone busywork. Instead of having highly paid accountants manually type monetary amounts into SAP, the accountants can now just look over and check if the AI was accurate in transcription.

Why AI, though? Why not OCR and regular software? What does AI bring to this table? Except for VC money.

OCR can't grasp context, it needs incoming bills to be formatted in exactly the same way or at least in somewhat consistent area patterns.

AI doesn't have that problem, so you can use it to detect areas of interest and then follow up on that with classic OCR.

Re: The Rise of the AI Engineer

#152

Post by buffalobuffalo »

__loam wrote: post #137
buffalobuffalo wrote: post #62

Earlier quoted context omitted.

I'm also a skeptic, but for a slightly different reason. There are currently two types of business use cases that seem to be the focal point of this generation of AI. 1) Tooling. This one I think will probably bear fruit. It will likely result in huge productivity gains (I mean, it already has for me). But i don't know if it will result in a paradigm shift. 2) Agents. This is where most of the hype is focused. The id…

I disagree completely with your last point. Flooding the internet with automated bullshit and people locking down their data because they're worried they'll get scraped is going to be a hugely negative consequence, to say nothing of what this will do to people like artists and writers because of the perception of this technology that the people paying them are going to have.

I actually agree with you. I think the societal impacts of this are going to be huge. I was speaking more to the monetization potential of AI. And I guess I should have specified that this refers legal monetization, since I'm guessing that for a time, illegal bot operators will do quite well.

In terms of societal impact, I suspect that this will ultimately result in the death of all open platforms on the internet and a withdrawal from online spaces in general.

Re: The Rise of the AI Engineer

#153

Post by swyx »

__rito__ wrote: post #60

> "the fundamental gatekeeping that still persists in the market" This person just called me a gatekeeper when just our definitions of what an AI Engineer is different... all while discussing the ambiguities of the term "AI Engineer". Well, that's the internet for you! My intentions were never of gatekeeping, our definitions of the term are wildly different. I laid the beginning part of a roadmap for a person who wan…

apologies for that - it came from a friend who was reviewing that HN thread and used that word - but it was my choice to repeat that word in my writeup. I didnt take your feelings into account when using what I knew to be a loaded word, and in retrospect it did nothing to further the strength of my argument at all.

I'm sorry. I've removed it.

Re: The Rise of the AI Engineer

#154

Post by z3c0 »

visarga wrote: post #144
z3c0 wrote: post #111

Earlier quoted context omitted.

No. No they are not. Do not keep proliferating this idea, as there are real consequences. Your input only informs the likelihood of the output. There is no model actually extrapolating rules from the information, so the premise of LLMs being extractive is 100% verifiably false. They summarize well due to being handed a roughly correct arrangement of tokens to mimic the order of. This should not be confused with extra…

You are talking from a top-down point of view. Of course they can make an error any time, after all they stochastically sample the output. All approaches to information extraction are upper bounded at 90-95% accuracy, I have extensive experience with this task and run many evaluations on invoices, receipts, forms and other doctypes. Human in the loop is still required. But in practice you can rely on good copying and…

I'm talking about an "at scale" point of view. A 90-95% success rate would be at worst 9:1 odds and at best 19:1. That means your best case scenario would be a likely failure on every 20th inference. This may be passable on an adhoc individual basis, but why do this when a deterministic solution can achieve beyond 99%, with proper error handling? Data normalization tools are rigid as a feature - LLMs are not, even at a temp of 0

Re: The Rise of the AI Engineer

#155

Post by wpietri »

janalsncm wrote: post #56
wpietri wrote: post #43

Earlier quoted context omitted.

For me that's absolutely the strongest ground for comparison. In recent decades, we've had two big waves of tech advance: the Web and mobile. A lot of people have lived through them both, giving them an expectation that another such wave should be along soon. You could see that in the decade of blockchain/ICO/DAO/NFT/web3 hype, where people, many with shaky credentials, touted the transformation soon to come, taking…

Fraudsters will do what fraudsters do. If it wasn’t crypto it would be scamming retirees or hoarding PS5s. They’re a largely insignificant part of the economy. At a macro scale private capital has a much larger impact. The issue is there’s a lot of dumb VC money floating around looking for a quick billion instead of investing long term fundamental research (boring!) that may produce results later on. It’s a fundament…

I don't think there's as clear a line here as you think. It's not like crypto was just some small-time grifters who would otherwise be running crooked pop-the-balloon games at carnivals. Notable portions of the "real" economy got in on the game. And it's hardly just crypto where people have been hoovering up credulous money claiming they had the next Google all ready to go.

Re: The Rise of the AI Engineer

#156

Post by majesticglue »

spmurrayzzz wrote: post #24
cbm-vic-20 wrote: post #7

I'm a natural skeptic, and I believe we're still on the rising edge of the "AI" hype cycle. Five years ago, it was "blockchain", and everyone was trying to ram blockchain into everything, attracting lots of VC and media attention, etc. It seems that blockchain is beyond the honeymoon phase: I haven't seen an NFT or even a Bitcoin headline in HN for a while. So I'm trying to wrap my head around what an "AI Engineer" i…

> So is the "engineering" part of this finessing the input and massaging the output? I don't know if I'll ever use the phrase "AI Engineer" myself, but there's plenty of meaningful engineering work in that space that strays pretty far from just calling some provider's APIs. A few that come to mind just for LLMs: - Custom fine-tuning of foundational models both in the classic sense and with more modern strategies like…

of all the things mentioned in this whole thread, this resonates the most. from my take, everyone here mentions AI engineering as someone just plugging in LLM (ie OpenAI) into every software application. However I believe there is a nice burgeoning of the "AI Engineer" where it involves more of the data processing and specialized AI fine-tuning that engineers have a role in, but of course it requires more specialization than just learning LangChain and calling it a day.

Re: The Rise of the AI Engineer

#157

Post by pvman »

z3c0 wrote: post #154
visarga wrote: post #144

Earlier quoted context omitted.

You are talking from a top-down point of view. Of course they can make an error any time, after all they stochastically sample the output. All approaches to information extraction are upper bounded at 90-95% accuracy, I have extensive experience with this task and run many evaluations on invoices, receipts, forms and other doctypes. Human in the loop is still required. But in practice you can rely on good copying and…

I'm talking about an "at scale" point of view. A 90-95% success rate would be at worst 9:1 odds and at best 19:1. That means your best case scenario would be a likely failure on every 20th inference. This may be passable on an adhoc individual basis, but why do this when a deterministic solution can achieve beyond 99%, with proper error handling? Data normalization tools are rigid as a feature - LLMs are not, even at…

Thanks for the info z3c0. Have you implemented open source solutions? Do you mind sharing some deterministic solutions to detect entities without huge dictionaries (NER)? Or maybe extractive QA if you used that instead?

Re: The Rise of the AI Engineer

#158

Post by z3c0 »

pvman wrote: post #157
z3c0 wrote: post #154

Earlier quoted context omitted.

I'm talking about an "at scale" point of view. A 90-95% success rate would be at worst 9:1 odds and at best 19:1. That means your best case scenario would be a likely failure on every 20th inference. This may be passable on an adhoc individual basis, but why do this when a deterministic solution can achieve beyond 99%, with proper error handling? Data normalization tools are rigid as a feature - LLMs are not, even at…

Thanks for the info z3c0. Have you implemented open source solutions? Do you mind sharing some deterministic solutions to detect entities without huge dictionaries (NER)? Or maybe extractive QA if you used that instead?

Deterministic? Not anything that is "one size fits all". If your documents are of an unpredictable shape, then ML is your best bet. For both NER and QA, BERT models are very capable.

To your first question, most of my ML work is proprietary, unfortunately. I am hoping to change that in the near future.

Edit: spaCy is a great library for NER, if you're hoping for an open-source solution that can have you hitting the ground quickly. SQuAD for QA.

Re: The Rise of the AI Engineer

#159

Post by __rito__ »

swyx wrote: post #153
__rito__ wrote: post #60

> "the fundamental gatekeeping that still persists in the market" This person just called me a gatekeeper when just our definitions of what an AI Engineer is different... all while discussing the ambiguities of the term "AI Engineer". Well, that's the internet for you! My intentions were never of gatekeeping, our definitions of the term are wildly different. I laid the beginning part of a roadmap for a person who wan…

apologies for that - it came from a friend who was reviewing that HN thread and used that word - but it was my choice to repeat that word in my writeup. I didnt take your feelings into account when using what I knew to be a loaded word, and in retrospect it did nothing to further the strength of my argument at all. I'm sorry. I've removed it.

Thank you so much.

As I mentioned, my definition was just different. And I didn't even mean to gatekeep. My spirit was: "if I can do it, many more people definitely can, too". So I went on and shared an outline of my journey. That's all.

No hard feelings.

Post reply on HN