Common data model mistakes made by startups • Hacker News Board

Re: Common data model mistakes made by startups

#71

Post by Ericson2314 » Sun, May 23, 2021, 5:37 AM UTC

laichzeit0 wrote: post #68

Tabular-Iceberg wrote: post #61

Earlier quoted context omitted.

This is the rationalisation I get every time when I tell companies that their data model is a mess. Never mind that neither I nor the parent said anything about doing it up front. Of course they have to iterate, the problem is that there is no deliberate effort anywhere, it’s just piling more crap on top of old crap and deluding themselves that they are some kind of lean, agile visionaries because of it.

I think this is largely a consequence of microservices. What is the "data model" here? You're thinking database, to a microservice that's a repository implementation detail.

I hope this is sarcasm poking fun at why microservices are terrible and small companies should stop LARPing FANG and avoid them?

Re: Common data model mistakes made by startups

#72

Post by Ericson2314 » Sun, May 23, 2021, 5:38 AM UTC

SkyPuncher wrote: post #62

tomnipotent wrote: post #54

Earlier quoted context omitted.

I'm not aware of a single project, ever, that has gotten their data model right up front and not had to iterate on it countless times as it grew/evolved. Except maybe NASA. Even the best early data models fail after years of updates and evolution.

My rule is "make it easy for us to fix our mistakes". Even when we've spent a bunch of time planning out data, but we still got a lot of things wrong in hindsight. The reality is we didn't know enough about our product direction to make any truly informed decisions. In general, poor decisions seem to stem from working in ambiguity about product, rather than poor technical decisions.

Yes, do migrations well to make it easy to fix mistakes. Don't make vague stupid data models to try to avoid making mistakes by being noncommittal.

Re: Common data model mistakes made by startups

#73

Post by cerved » Sun, May 23, 2021, 6:11 AM UTC

irrational wrote: post #59

andrewprock wrote: post #43

Earlier quoted context omitted.

The chance that you don't have constraints set up correctly is indistinguishable from 100%.

I disagree. Any fairly competent DBA will know how to setup the constraints correctly. It's not rocket science. If you can think logically enough to program, you can think logically enough to set up constraints correctly.

Not everybody has a DBA :(

Re: Common data model mistakes made by startups

#74

Post by atatatat » Sun, May 23, 2021, 6:12 AM UTC

snowwrestler wrote: post #67

jhgb wrote: post #60

Earlier quoted context omitted.

> It is a rookie blunder to link them relationally to master data for products and PII &c. Is it always? If that data is immutable, for example?

How are you going to satisfy data compliance, which may require the deletion of PII upon request or expiration, if your PII data is immutable?

Was just considering this when I came across your comment.

I'm hoping someone here can suggest a one-way audit-log audit-trail sort of solution, because I need this for the medical industry.

Re: Common data model mistakes made by startups

#75

Post by brylie » Sun, May 23, 2021, 6:15 AM UTC

maneesh wrote: post #17

brylie wrote: post #10

If your company has a subscription business model, keep a history of user's subscriptions. They change over time and it is likely you will need to measure popularity and profitability of product offerings over time. Please don't force your analytics team to rely on event logs to reconstruct a subscription history.

Stripe manages this extremely well

That's a good point. Most subscription service providers, like Stripe, Chargrbee, Braintree, etc, use a fairly conventional one-to-many data architecture for Customers and Subscriptions.

Just take care to use the subscription service provider data model how it is intended. It is possible to design your integration in a way that goes against the grain and end up with gaps in your data. For example, by re-using a single subscription instance per customer and changing it's properties when the customer down/upgrades rather than creating a new Subscription instance.

Re: Common data model mistakes made by startups

#76

Post by cerved » Sun, May 23, 2021, 6:17 AM UTC

jiggawatts wrote: post #65

beachy wrote: post #31

Earlier quoted context omitted.

At the other end of the scale is a data model designed for extreme extensibility. If you ever hear anyone bragging that their data model is entirely metadata driven, and can be used to model anything - without changing the database - that's a huge red flag, as is looking in and seeing tables called "element", "business object" and the like. Unfortunately, for most serious Enterprise systems, a degree of flexibility i…

One underlying reason for this is that DBMS systems have an unnecessary source of complexity: They have a separate Data Manipulation Language (DML) and a Data Description Language (DDL). They really ought to be unified, but few (any?) mainstream SQL databases are homoiconic in this way. E.g.: It should be possible to take a query definition, request its columns ("schema only" execution), and then insert or merge the…

it sounds like you're making an interesting point but I'm afraid I don't follow, could you elaborate?

Re: Common data model mistakes made by startups

#77

Post by cerved » Sun, May 23, 2021, 6:31 AM UTC

jerrysievert wrote: post #55

the one that is missing for me, that is my personal pet peeve: an index for every column in the database. then wondering why inserts are slow. seriously?

why would anyone ever want to do that?

Re: Common data model mistakes made by startups

#78

Post by cjfd » Sun, May 23, 2021, 6:32 AM UTC

It sounds like quite a few of the problems that are mentioned here can be ameliorated using views.

Re: Common data model mistakes made by startups

#79

Post by [deleted] » Sun, May 23, 2021, 6:51 AM UTC

atatatat wrote: post #74

snowwrestler wrote: post #67

Earlier quoted context omitted.

How are you going to satisfy data compliance, which may require the deletion of PII upon request or expiration, if your PII data is immutable?

Was just considering this when I came across your comment. I'm hoping someone here can suggest a one-way audit-log audit-trail sort of solution, because I need this for the medical industry.

[deleted]

Re: Common data model mistakes made by startups

#80

Post by ako » Sun, May 23, 2021, 7:09 AM UTC

irrational wrote: post #59

andrewprock wrote: post #43

Earlier quoted context omitted.

The chance that you don't have constraints set up correctly is indistinguishable from 100%.

I disagree. Any fairly competent DBA will know how to setup the constraints correctly. It's not rocket science. If you can think logically enough to program, you can think logically enough to set up constraints correctly.

This should be the responsibility of the application developer creating the database schema and queries. A constraint is part of your application logic, not of the administration of the database.