Live data from Hacker News

Agentic AI systems violate the implicit assumptions of database design

arpitbhayani.me

21–30 of 115 posts

Re: Agentic AI systems violate the implicit assumptions of database design

#21

I totally agree on investing in a sane data model upfront. So many production systems have schemas that only made sense to the engineer that created them. I would be delighted if I can read a schema and understand what a column means without having to dig through a bunch of migration PRs. I recently encountered `is_as BOOL` in an important table. After way too much invested time we found out it meant "is active servi…

> I recently encountered `is_as BOOL` in an important table.

Sounds like a table designed by Forrest Gump.

Re: Agentic AI systems violate the implicit assumptions of database design

#23
post #12
post #2

Giving LLM agents direct, autonomous access to a real production databases with write access seems insane to me. NO ONE, agent or human, should have direct write access to production databases outside of emergency break glass scenarios. This is why we have stored routines and API layers to pre-define what writes are allowed. The facts that agents CAN autonomously write to a database does not imply that they should. F…

How does that even work in compliance-relevant scenarios where the audit trail shows some LLM messed with the data? Who, if anyone, is on the hook?

My guess is that if the database is subject to auditing then LLM access (obviously writes in particular, but even reads come with exfiltration risks) will be a hard "no" and instant red flag. When it's a person, there is a sense of accountability and opportunity for remediation.

I suppose that LLMs will be treated as a code artifact and liability will shift upstream towards who deployed/approved the access in the first place. Even though code is essentially deterministic, making that association fairly simple, it's going to boil down to this same paradigm.

Perhaps governance rules will evolve to even explicitly forbid it, but my gut feeling is that for what the future determines to be "practical" reasons (right or wrong) LLMs will warrant an entirely new set of rules to allow them to be in the chain at all.

+ EDIT: both my wife and I have experience in this area and the current answer is companies like KPMG don't have an answer yet. Existing rules do help (e.g. there better be good documented reasons why it was used and that access was appropriately scoped, etc), but there is enough ambiguity around these tools so they say "stay tuned, and take caution".

Re: Agentic AI systems violate the implicit assumptions of database design

#24
> The API failed silently because the database connection pool was exhausted downstream.

I work with a team that does stuff like this, returning a 200 and a body containing "error: I didn't do what you said because _insert error here_"

The problem is that you returned OK instead of ERROR when things were not OK and there was an ERROR.

Its a design that smells of teams trying to hit some kind of internal metrics by slightly deceptive means.

Re: Agentic AI systems violate the implicit assumptions of database design

#25

I totally agree on investing in a sane data model upfront. So many production systems have schemas that only made sense to the engineer that created them. I would be delighted if I can read a schema and understand what a column means without having to dig through a bunch of migration PRs. I recently encountered `is_as BOOL` in an important table. After way too much invested time we found out it meant "is active servi…

Postgres has COMMENT ON to help with this but descriptive names are helpful.

Re: Agentic AI systems violate the implicit assumptions of database design

#26

I totally agree on investing in a sane data model upfront. So many production systems have schemas that only made sense to the engineer that created them. I would be delighted if I can read a schema and understand what a column means without having to dig through a bunch of migration PRs. I recently encountered `is_as BOOL` in an important table. After way too much invested time we found out it meant "is active servi…

I think the best db schema I had the displeasure of working with was one where it was a requirement that every table and column name NOT have vowels, except for the few that could, and "the few that could" were governed entirely by a spreadsheet owned by the DB admin.

And so you got tables like LANDMRK and columns like RCR_RCRDR.

Re: Agentic AI systems violate the implicit assumptions of database design

#27
post #10

Giving an LLM write access is insane but I gave LLM’s read-only access to our database and it’s been a huge productivity win. Executives who wouldn’t take the time to build a report are happy to ask an AI agent to do so.

How do you validate that the reports are correct? What if an executive makes a wrong business decision because the LLM wrote a wrong SQL query?

[deleted]

Re: Agentic AI systems violate the implicit assumptions of database design

#28
post #4
post #2

Giving LLM agents direct, autonomous access to a real production databases with write access seems insane to me. NO ONE, agent or human, should have direct write access to production databases outside of emergency break glass scenarios. This is why we have stored routines and API layers to pre-define what writes are allowed. The facts that agents CAN autonomously write to a database does not imply that they should. F…

LLM agents are unlocking demand and supply for applications that wouldn't have been possible before due to time constraints though. There's a growing demand for single user or smaller scoped apps where giving LLM agents direct access means velocity. The failure/rollback model is much easier with these as long as we have good backup hygiene.

> There's a growing demand for single user or smaller scoped apps where giving LLM agents direct access means velocity. The failure/rollback model is much easier with these as long as we have good backup hygiene.

This makes no sense to me. For anything that has sensitive payment or personally identifieable data, direct access to DB is potentially illegal.

> The failure/rollback model is much easier with these as long as we have good backup hygiene.

Have you actually operated systems like this in production? Even reverting to a DB state that is only seconds old can still lose hundreds or thousands of transactions. Which means loads of unhappy customers. More realistically, recovery points are often minutes or hours behind once you factor in detection, validation and operational overhead.

DB revert is for exceptional disaster recovery scenarios, not something you want in normal day-to-day operations. If you are saying that you want to give LLM full access to prod DB and then revert every time it makes a mistake, you aren't running a serious business.

Re: Agentic AI systems violate the implicit assumptions of database design

#29

This article has all the correct conclusions and solutions based on one assumption that doesn’t have any hold in reality - that someone would be insane enough to allow direct DB access to an AI agent.

someone is potentially insane enough it would seem: https://news.ycombinator.com/item?id=47911579

Re: Agentic AI systems violate the implicit assumptions of database design

#30
post #4
post #2

Giving LLM agents direct, autonomous access to a real production databases with write access seems insane to me. NO ONE, agent or human, should have direct write access to production databases outside of emergency break glass scenarios. This is why we have stored routines and API layers to pre-define what writes are allowed. The facts that agents CAN autonomously write to a database does not imply that they should. F…

LLM agents are unlocking demand and supply for applications that wouldn't have been possible before due to time constraints though. There's a growing demand for single user or smaller scoped apps where giving LLM agents direct access means velocity. The failure/rollback model is much easier with these as long as we have good backup hygiene.

> single user

If you're just vibe coding a tool for yourself, you don't have 'production database' at all even if you use database technology for storage. Just like many Android apps use local sqlite DBs but they're not production databases.

Of course in this case no traditional wisdom about production databases matters to you. In other words, it's off-topic.

Post reply on HN