Ent: An Entity Framework for Go • Hacker News Board

Re: Ent: An Entity Framework for Go

#61

Post by bokwoon » Mon, Oct 31, 2022, 4:32 PM UTC

Fire-Dragon-DoL wrote: post #29

bokwoon wrote: post #24

Earlier quoted context omitted.

> at the same time, they provide little (or nothing) utilities to map the data coming from a complex query, back to objects Example of a query builder with built-in mapping capabilities (it’s mine): https://github.com/bokwoon95/sq I feel some exasperation when I see query builders that throw a query string back to the user and ask them to map the results themselves. That’s easily the most tedious and mistake-prone pa…

I skim-read it but couldn't find an example of what I think as challenging: a join of 3 tables (well, even two). When you join 3 tables (assuming has many and "has many through" relationships), what you get back is enormous rows and multiple rows, all of this in tabular form, but in the software is usually represented as a graph. I'd love a library that helps building back these massive rows into relationships. Pleas…

> I didn't see anything to help me build back graphs

Hmm you've certainly given me something to think about. Thanks.

BTW joins are not challenging, but you made me realize I didn't show any joins in my basic examples. Here is an UPDATE with JOIN in the meantime: https://bokwoon.neocities.org/sq.html#postgres-update-with-j....

Re: Ent: An Entity Framework for Go

#62

Post by valenterry » Mon, Oct 31, 2022, 5:51 PM UTC

codeflo wrote: post #8

valenterry wrote: post #5

> 100% statically typed and explicit API using code generation. Whenever I have to do code generation in my language of choice, I feel that the language should be able to do that without having to generate code. It's annoying to inspect and maintain and usually has quirks that I have to work around. How do Go developers feel about that?

ORMs have the problem that the shape of the interface they provide mostly depends on configuration or data (the SQL schema). In static languages at least, this means that code has to be generated at some point. Tooling and convenience wise, there are differences between the approaches, but conceptually, does it really make a huge difference whether this step happens in a separate tool before compilation (seen here),…

> ORMs have the problem that the shape of the interface they provide mostly depends on configuration or data (the SQL schema). In static languages at least, this means that code has to be generated at some point.

Maybe for most languages that's true, but in general I don't think it is.

> Tooling and convenience wise, there are differences between the approaches, but conceptually, does it really make a huge difference whether this step happens in a separate tool before compilation (seen here), or inside the compiler (in a macro system like Rust’s), or at runtime (as ORMs tend to do in JITted languages like Java and C#)?

Between runtime and compiletime or code generation there is certainly a big difference. Between compile-time (or macro time) and code generation I think too. The reason is that, for instance, if it happens at compile-time then you have types that you can work with. You can use those types and reshape them to create even more types. E.g. you could take the database schema types and generate graphql types. That means, the mapping from the database to the types of the programming language has to be done only once and then other libraries can build upon it without containing this part anymore.

But if you use code generation, essentially every library has to know how to get the types from the database, since it is very hard to take the generated code and then transform it into types or code for e.g. graphql. Unless you parse the generated code and then transform it. Not only is there now a problem of the order of code generation but I believe it is also inherently harder to parse arbitrary code compared to parsing arbitrary types, simply due to the fact that types are much more restricted in what they can express.

Re: Ent: An Entity Framework for Go

#63

Post by zeroc8 » Tue, Nov 01, 2022, 8:22 AM UTC

ThinkBeat wrote: post #53

I hear Entity framework and code as schema I get nervous The .Net entity framework has caused a lot of problems for clients in the past. I am not as familiar with the new one in .net core which is now just .net again I think. A huge mess managing deltas for the schema. Often used by people who do not know SQL nor relational databases that leads to enormously slow and resource intensive queries, (that can be done real…

We ran into huge problems using Hibernate. Hibernate is such a complex beast and and includes so many traps that I think it's just not worth it, at least not when you don't have an absolute Hibernate expert on board. In our case, people don't care, as long as it "works" - which of course causes a lot of performance problems down the road.

I'm not opposed to ORMs in general, but there has to be a middle ground.

Re: Ent: An Entity Framework for Go

#64

Post by jitl » Tue, Nov 01, 2022, 1:51 PM UTC

rowanG077 wrote: post #52

jitl wrote: post #13

Earlier quoted context omitted.

Why is that? I’ve done a few toy attempts to build out ORM-like model classes in Typescript that wrap an input schema type using typelevel features, and my profiles of the Typescript compiler show a lot of slowdowns coming from combinatoric explosions in the typechecker. In this language specifically, the type system has quite advanced expressiveness but using those features in practice runs into limitations of the c…

I said almost never not never, 64 variants is quite esoteric I'd say. The reason why I dislike codegen so much is that you fix one problem but introduce multiple new problems. How do you handle the generated code? Do you check it into source control? If so, how do you ensure people do not touch it. Manually modified code generated code is one of the worst places to be in for maintenance. You mentioned you wanted lang…

My general tactic for codegen hygiene like this is to check in the generated files, and on every CI re-run the code generator. After the generator runs, CI asserts there are no changes in Git. If there are any changes, the job fails.

This ensures that generated code is consistent with its inputs (including the generator logic) for every merge to main. It prevents people from editing the generated code, since their edits will cause a diff in CI and fail the build, but no one ever actually loses their work since the codegen does not run automatically/continuously during local development.

Checking in the files also makes it very easy to review changes to the generator since you can always tell how the output is changing.

The total burden for the above system is:

- a 30 line command called `Notion assert-clean`

- A 30 line CircleCI job that loops over the list of code generator commands, calls the command, then calls assert-clean.

We’ve used that tactic for years for simpler stuff like “make sure the SQL dump is consistent with the SQL migrations” and “put all the file names in this directory tree into a typescript file so we can tab complete them”.

The only necessary bit is that your code generator shouldn’t be spitting out enormous mountains of unreadable code.

Re: Ent: An Entity Framework for Go

#65

Post by Fire-Dragon-DoL » Wed, Nov 02, 2022, 2:34 AM UTC

bokwoon wrote: post #61

Fire-Dragon-DoL wrote: post #29

Earlier quoted context omitted.

I skim-read it but couldn't find an example of what I think as challenging: a join of 3 tables (well, even two). When you join 3 tables (assuming has many and "has many through" relationships), what you get back is enormous rows and multiple rows, all of this in tabular form, but in the software is usually represented as a graph. I'd love a library that helps building back these massive rows into relationships. Pleas…

> I didn't see anything to help me build back graphs Hmm you've certainly given me something to think about. Thanks. BTW joins are not challenging, but you made me realize I didn't show any joins in my basic examples. Here is an UPDATE with JOIN in the meantime: https://bokwoon.neocities.org/sq.html#postgres-update-with-j... .

Thank you for the examples. I see the joins example, but they seem to be about creating queries, not mapping data.

    sql = "select blog.name, post.content, author.display_name from blog join post on blog.id = post.blog_id join author on post.author_id = author.id"

Assuming the relationship: many blogs have many posts and posts have one author, I'd expect something along the lines of (sudo code):

    schemaOnTheFly = Blogs{}.HasMany(Posts{}.HasOne(Author{}) // Sorry the syntax for this doesn't really exist
    blogs := query.Exec(sql, params, schemaOnTheFly)

    fmt.Printf("%+v\n", blogs[0].Posts[0].Author)

That's what I'd expect. Do notice that the schema is per-query, I'll let the developer handle the sharing portion of the schema (might be shared by a few queries)

Re: Ent: An Entity Framework for Go

#66

Post by jitl » Wed, Nov 02, 2022, 8:42 AM UTC

nika1975 wrote: post #57

jitl wrote: post #13

Earlier quoted context omitted.

Why is that? I’ve done a few toy attempts to build out ORM-like model classes in Typescript that wrap an input schema type using typelevel features, and my profiles of the Typescript compiler show a lot of slowdowns coming from combinatoric explosions in the typechecker. In this language specifically, the type system has quite advanced expressiveness but using those features in practice runs into limitations of the c…

I guess you are aware of Prisma. What is your opinion of it in this context?

Prisma is a nice DB client library but does't do any of the things I'm interested in - reactive queries on the client, recursive graph traversal, raising level of abstraction for the organization. Schematizing the DB & a type-safe DB client is nice, but I'm more interested in stuff one or two levels of abstraction up. Like, after the data comes into memory Prisma is done and out of the picture. But getting the data into memory is the easy part IMO. Traversing it, adding permissions and business logic, managing & composing mutations, dealing with caching and reactivity.. that's the good stuff, and I'm not sure what if any Prisma offers there.

Re: Ent: An Entity Framework for Go

#67

Post by mbrodersen » Wed, Nov 02, 2022, 10:50 PM UTC

rowanG077 wrote: post #9

valenterry wrote: post #7

Earlier quoted context omitted.

> 1. Codegen is powerful, and often easier to understand than a mountain of typelevel magic. I disagree here. And I also don't like your framing. Yes, type-level logic is harder to learn than understanding code you already read everyday. It's also easier to count and calculate with your fingers and use concrete examples rather than learning abstract math. But we still do it, because in the end you learn it once and h…

Totally agree with you. Especially about point 1. Codegen is one of the bluntest tools there is and should be almost always avoided.

Codegen is a super power. It has consistently made me 5 to 10 times more productive compared with experienced developers I have worked with.

I have more than once implemented production code where 80% of the code is generated. With only the biz logic hand coded. Including a complex 50+ screen Typescript application developed from scratch in just 3 months.

However, as with any other tool, you have to use Codegen intelligently to get the benefits. I have more than once seen Codegen being used really badly. Making the people using it think that Codegen is always a bad idea. Codegen is a complex tool and there are many ways to use it incorrectly.

Re: Ent: An Entity Framework for Go

#68

Post by mbrodersen » Wed, Nov 02, 2022, 10:55 PM UTC

valenterry wrote: post #5

> 100% statically typed and explicit API using code generation. Whenever I have to do code generation in my language of choice, I feel that the language should be able to do that without having to generate code. It's annoying to inspect and maintain and usually has quirks that I have to work around. How do Go developers feel about that?

The advantage of Codegen is that it works for any language. Whether the language has macros or whatever. Another advantage is that you can generate code for multiple languages (say Typescript and C#) from the same spec, making sure the client/server code stays 100% in sync.

Codegen is my favourite super productivity tool. I routinely write production code where only the core biz logic is hand coded. The rest is fully automated. Most Excellent!