Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ideas to improve the user and developer experiences of databases

dnlhg.com

Post reply on HN
Print view View original

51–56 of 56 posts

Re: Ideas to improve the user and developer experiences of databases

#51

Post by theptip »

sa46 wrote: post #46
theptip wrote: post #35

Earlier quoted context omitted.

You don’t need to go this path; modern(-ish) APM systems like Datadog will let you trace SQL commands in the context of the request being made. You can group by endpoint, or group by sql (merging IDs that differ), or chop and filter the data however you need. The DB is not in a position to give you all the stats you need; instead the DB should support e.g. exporting plan info to further enrich the whole-request metri…

I've used Datadog but if you need per query tracing from the database logs, I think the only way is to embed a trace ID in a SQL comment (or in an no-op column) like the grandparent showed. Datadog is tracing the request from the application akin to something like openTrace("Query Name", queryStr) db.Query(queryStr) closeTrace() The gap with that style of application-level tracing is that the database logs give no in…

I guess what I'm getting at is questioning why you need to look at the database logs (beyond what your APM system can pull from the DB's metrics).

What are you getting from the DB logs that you can't get elsewhere?

Re: Ideas to improve the user and developer experiences of databases

#52

Post by RussianCow »

syastrov wrote: post #6

I’d like a tool that automatically suggests which indexes to add/remove, including use of different types of indexes and partial indexes, based on the queries being performed. Even better if it can automatically test the results of that on a production workload. Or suggest schema changes that would improve normalization or performance.

If you're using Postgres, check out dexter[0].

[0]: https://github.com/ankane/dexter

Re: Ideas to improve the user and developer experiences of databases

#53

Post by eloff »

mamcx wrote: post #50
eloff wrote: post #48

Earlier quoted context omitted.

String parsing is not an appreciable part of the query execution cost. I'm not sure what the goal here would be otherwise?

SQL, alike JS, is not a good target for the myriad of languages(and orms) than target it, and it have intrinsic limitations that are historical artifacts. Is not the cost of string concatenation, is to provide the benefits of a good byte code that could allow, for example, type check input, schemas and other stuff on client side before touch the db. Is similar to how GraphQL unlock great tooling because is a specific…

Yeah, that would be nice to have.

Re: Ideas to improve the user and developer experiences of databases

#54

Post by sa46 »

theptip wrote: post #51
sa46 wrote: post #46

Earlier quoted context omitted.

I've used Datadog but if you need per query tracing from the database logs, I think the only way is to embed a trace ID in a SQL comment (or in an no-op column) like the grandparent showed. Datadog is tracing the request from the application akin to something like openTrace("Query Name", queryStr) db.Query(queryStr) closeTrace() The gap with that style of application-level tracing is that the database logs give no in…

I guess what I'm getting at is questioning why you need to look at the database logs (beyond what your APM system can pull from the DB's metrics). What are you getting from the DB logs that you can't get elsewhere?

Sure, off the top of my head:

- Explain plans for slow queries, via auto_explain in Postgres. You could get really fancy here and convert the pieces of an explain plan (Aggregate, ModifyTable, FullTableScan) into proper distributed traces. Hard to tell, but it looks like GCP might offer that.

- Various errors logs associated with queries that require more detail than a SQL state code. For example, lock timeouts.

- Debugging, especially root cause analysis when you're trying to figure out why something broke. The trace IDs help build up context.

Re: Ideas to improve the user and developer experiences of databases

#56

Post by bobbydreamer »

mamcx wrote: post #32

I'm worked on the side in a relational language ( https://tablam.org ) and have used FoxPro, that in a lot of ways is superior to any RDBMS of today. The ideas here are very good! But exist many other things that could have a greater impact: 1. We need an "wasm" for sql. SQL is not a good language to transpile to. ALL ORM ARE TRANSPILERS! The relational model allow to do so much with so little (you don't even need to…

My dad used to say that, FoxPro is the best in MSDos and visual Foxpro killed it and it was so fast in handling large amount of data.
Post reply on HN