Backend-as-a-service for web apps? • Hacker News Board

Re: Backend-as-a-service for web apps?

#61

Post by [deleted] » Fri, Dec 30, 2011, 6:13 AM UTC

captain-asshat wrote: post #59

alt_ wrote: post #44

Earlier quoted context omitted.

> It's an incredibly bad idea to allow SQL directly, for obvious reasons (would require executing user provided SQL among other things.) Could you elaborate on this? I've been thinking about providing SQL access to a data-heavy service, but I keep hearing that you never should. (Almost) all servers have granular access-control, views can further provide limited views of the data, SQL itself is mostly declarative, whi…

Well, the first problem you'll run into is, which SQL implementation do you use? I think pretty much all discussion of the topic stops here since there are so many differences between the actual SQL standard and what the varios modern RDBMS' actually use. The only real way past this hurdle is to create an intermediate SQL parser that uses your own interpretation of the standard, and at this point you may as well just…

[deleted]

Re: Backend-as-a-service for web apps?

#62

Post by 6ren » Fri, Dec 30, 2011, 6:13 AM UTC

captain-asshat wrote: post #59

alt_ wrote: post #44

Earlier quoted context omitted.

> It's an incredibly bad idea to allow SQL directly, for obvious reasons (would require executing user provided SQL among other things.) Could you elaborate on this? I've been thinking about providing SQL access to a data-heavy service, but I keep hearing that you never should. (Almost) all servers have granular access-control, views can further provide limited views of the data, SQL itself is mostly declarative, whi…

Well, the first problem you'll run into is, which SQL implementation do you use? I think pretty much all discussion of the topic stops here since there are so many differences between the actual SQL standard and what the varios modern RDBMS' actually use. The only real way past this hurdle is to create an intermediate SQL parser that uses your own interpretation of the standard, and at this point you may as well just…

That's a practical, real problem, yes. But something with the equivalent power of a relational algebra (e.g. SQL) seems needed, if web APIs are to be used by many different clients, and be back-compatible.

It's true that some aspects are different from those problems in an enterprise that SQL were designed to address (e.g. the web crosses companies; massive numbers of users; rarely mission-critical), so perhaps SQL itself won't be the ideal solution Even if it was, the usual approach of our industrial is to invent new standards whenever possible (new firms may do this partly to lock out old ones).

But it seems that the solution must be conceptually identical: an sufficiently expressive data model (e.g. relations), and a way to map between different representations with that data model (e.g. relational algebra).

Re: Backend-as-a-service for web apps?

#63

Post by 6ren » Fri, Dec 30, 2011, 7:32 AM UTC

matc wrote: post #52

nl wrote: post #49

Earlier quoted context omitted.

Okay..... I'm not sure what your point is. As you say, this is hard, and no one else has a solution either. But the nature of CouchDB can help here - views can be used to create backward compatible versions of new data, and reading a JSON document with extra fields won't break existing code. It's true it's not magic, though, if that is what you are after. Edit: I see now you are working on http://chronicdb.com/ which…

Views are the reflection of the light at the end of the tunnel, but solve less than half the problem to date. Views work on SELECT, but not on UPDATE/INSERT/DELETE. Also, what makes you say that database as an integration layer should always be avoided?

Views work on UPDATE/INSERT/DELETE with some restrictions, in MySQL 5.0 anyway: http://dev.mysql.com/doc/refman/5.0/en/view-updatability.htm... More expensive RDB's might too.

(I should have included this link in my other reply on problems with updating aggregated queries).

Re: Backend-as-a-service for web apps?

#64

Post by 6ren » Fri, Dec 30, 2011, 7:47 AM UTC

nl wrote: post #58

matc wrote: post #52

Earlier quoted context omitted.

Views are the reflection of the light at the end of the tunnel, but solve less than half the problem to date. Views work on SELECT, but not on UPDATE/INSERT/DELETE. Also, what makes you say that database as an integration layer should always be avoided?

Also, what makes you say that database as an integration layer should always be avoided? 15 years of writing software. Every time I've had to deal with a shared database it has caused problems, and every time I've built a system avoiding that antipattern it has worked much better. But it's not just me, here's some references: Enterprise Integration Anti-Patterns # - The Shared Database : http://ianfnelson.com/archive…

Database-as-IPC is about the transport aspect, not the logical integration (which is the problem of back-compatibility).

Ian F Nelson's blog post is about a DB that is tightly-coupled to the app: “my” application is using NHibernate as an ORM layer, so until this invasion we have been able to perform database schema refactorings with relative impunity.

Conventionally, RDB schemas are designed in terms of the data itself, not a particular app. In the Enterprise (where RDB are most used), data typically outlives different applications for the same task over decades, often written in different languages. Typically, the data needs also to be accessed by several different applications.

But here I'm talking about logical back-compatibility (surviving version changes) - the blog makes good points about "caching and application-level security". Where those vary with the application, it makes sense to separate them from the DB. But like Database-as-IPC, they are closer to the transport layer than to logical structure.

Re: Backend-as-a-service for web apps?

#65

Post by devs1010 » Sat, Dec 31, 2011, 7:59 AM UTC

mgkimsal wrote: post #46

devs1010 wrote: post #22

Isn't this what frameworks like rails, grails and spring roo are for? They make creating a basic CRUD app trivial. If you just need a trivial app then use one of these frameworks, anything more is obviously going to require customization on the backend code. I suppose there could be a niche for managed RAD (rapid application development) packages, or something like that, where they help people build an app using rail…

I've thought for a while about a service that would allow you to define a set of basic models/domains, including basic validation rules and relationships on them, then offer to generate the CRUD code, but in a variety of frameworks - grails, zendframework, rails, asp.net, django, etc. Beyond being a 'neat idea' though, I couldn't find any real use case for it.

Yeah... the people who can make use of this sort of thing already would probably be able to do the steps of generating the code in the RAD framework so I just don't think there's much room to actually make money on something like this but maybe if integrated with something like Heroku it could have potential.

Re: Backend-as-a-service for web apps?

#66

Post by devs1010 » Sat, Dec 31, 2011, 8:01 AM UTC

nl wrote: post #49

matc wrote: post #40

Earlier quoted context omitted.

This is what you are missing: > but that is always true You may have an old form you can't change. For example: - 10 apps connected to the same database. Changing the model breaks 10 apps. And you may not have access to change the code to any of those apps. - In healthcare, when either the database or app go down people literally die. This isn't as easy as it gets: no database vendor has a solution for this to date.…

Okay..... I'm not sure what your point is. As you say, this is hard, and no one else has a solution either. But the nature of CouchDB can help here - views can be used to create backward compatible versions of new data, and reading a JSON document with extra fields won't break existing code. It's true it's not magic, though, if that is what you are after. Edit: I see now you are working on http://chronicdb.com/ which…

Adding extra JSON fields can break code if you have matching entity types, say in Java, for example, and its trying to "cast" those JSON objects to that data type I am fairly certain it will throw an exception if the JSON object has a field thats not in the Java type

Re: Backend-as-a-service for web apps?

#67

Post by matc » Sat, Dec 31, 2011, 2:05 PM UTC

6ren wrote: post #60

matc wrote: post #54

Earlier quoted context omitted.

You are right, this aspect is not new. More like half-baked. I'm not sure what you mean by bidirectional. If this suggests having a separate copy for the old data and the new data then, in ChronicDB at least, no they are not bidirectional. Which is what one wants ideally: data consistency. If you mean that updates on the old version are immediately reflected in the new version, then yes. Basically, there's a single v…

Yes, the second one. So that from the old app's point of view, nothing has changed: it can create, read, update, delete as before. A single version of the truth etc. SQL's "CREATE VIEW" does this already (note: not all views can support CRUD, e.g. if you changed an aggregate column, like COUNT, there is no clear meaning for the affect on the underlying tables. If you increased count, what rows should it add? If you d…

1. Thank you for the link on views being updateable with some (natural) restrictions (http://news.ycombinator.com/item?id=3406952). Given this feature, SQL offers the primitives needed. Even if this feature is missing from various databases at the moment, it should be available eventually.

But updateable views are still not what you want! See below.

2. What ChronicDB adds in terms of backward compatibility is, effectively, automatic creation and management of these views for every schema change.

Rather than manually write compatibility views and rewrite/review an application to ensure it always uses these views, programmers instead access plain tables, not views. In fact ChronicDB creates no views at all!

You want to issue SQL queries that don't break. You don't want to manage views to do so. It's a subtle, yet major point. No database to date offers such automated versioning.

ChronicDB of course adds other features we wanted to have from our database (reversibility, data versioning, zero-downtime relocation) again in the spirit of saving the programmer/DBA from doing anything manually.

Re: Backend-as-a-service for web apps?

#68

Post by 6ren » Sat, Dec 31, 2011, 2:52 PM UTC

matc wrote: post #67

6ren wrote: post #60

Earlier quoted context omitted.

Yes, the second one. So that from the old app's point of view, nothing has changed: it can create, read, update, delete as before. A single version of the truth etc. SQL's "CREATE VIEW" does this already (note: not all views can support CRUD, e.g. if you changed an aggregate column, like COUNT, there is no clear meaning for the affect on the underlying tables. If you increased count, what rows should it add? If you d…

1. Thank you for the link on views being updateable with some (natural) restrictions ( http://news.ycombinator.com/item?id=3406952 ). Given this feature, SQL offers the primitives needed. Even if this feature is missing from various databases at the moment, it should be available eventually. But updateable views are still not what you want! See below. 2. What ChronicDB adds in terms of backward compatibility is, effe…

2. Yes, I agree, nice to have the "view" created automatically when you change schema.

Codd's idea was also that applications needn't be aware of the change (a view appears as a plain table)... but (in contrast to your approach) the views were managed by the DBA who was making the schema changes. So I think this feature was present from the very beginning, being the initial motivation for relational databases. [I looked up MySQL because that's the DB I have access to, but it's a general feature: http://en.wikipedia.org/wiki/View_(database)#Read-only_vs._u... ]

I would expect automatic versioning to be available too, perhaps only in the older and more expensive RDBs. I've come across quite a few research papers on schema evolution in relational databases - some from big RDB vendors. I don't recall any on automated versioning, but I wasn't looking for that. So this is just my guess.

However, if you're not seeing databases with automated versioning, then (even if some highend RDBs do have it) there will be customers who also aren't seeing it (or it's not applicable e.g. it's too expensive). A market for you. And if you can make it truly painless (or even just less painful), that's really something.

I don't see any problem with your other features. The one feature of mapping for back-compatibility is of interest to me, which lead me to study Codd's original paper in great detail over a few weeks. That's why I've been hassling you on this issue.