Live data from Hacker News

Beyond Elk: Lightweight and Scalable Cloud-Native Log Monitoring

greptime.com

11–20 of 28 posts

Re: Beyond Elk: Lightweight and Scalable Cloud-Native Log Monitoring

#13

How does Greptime handle dynamic schemas where you don't know most of the shape of the data upfront? Where I work, we have maybe a hundred different sources of structured logs: Our own applications, Kubernetes, databases, CI/CD software, lots of system processes. There's no common schema other than the basics (timestamp, message, source, Kubernetes metadata). Apps produce all sorts JSON fields, and we have thousands…

Thanks for your question. GreptimeDB, like MongoDB, is schemaless. When ingesting data via OTEL or its gRPC SDKs, it automatically creates tables by inferring the schema and dynamically adds new columns as needed. Secondly, I prefer wide tables to consolidate all sources for easy management and scalability. With GreptimeDB's columnar storage based on Parquet, unused columns don't incur storage costs.

Thanks, that seems promising. So much of the documentation is schema-oriented, I didn't see that it supported dynamic schemas.

I find it interesting that Greptime is completely time-oriented. I don't think you can create tables without a time PK? The last time I needed log storage, I ended up picking ClickHouse, because it has no such restrictions on primary keys. We use non-time-based tables all the time, as well as dictionaries. So it seems Greptime is a lot less flexible?

Re: Beyond Elk: Lightweight and Scalable Cloud-Native Log Monitoring

#14

Earlier quoted context omitted.

Thanks for your question. GreptimeDB, like MongoDB, is schemaless. When ingesting data via OTEL or its gRPC SDKs, it automatically creates tables by inferring the schema and dynamically adds new columns as needed. Secondly, I prefer wide tables to consolidate all sources for easy management and scalability. With GreptimeDB's columnar storage based on Parquet, unused columns don't incur storage costs.

Thanks, that seems promising. So much of the documentation is schema-oriented, I didn't see that it supported dynamic schemas. I find it interesting that Greptime is completely time-oriented. I don't think you can create tables without a time PK? The last time I needed log storage, I ended up picking ClickHouse, because it has no such restrictions on primary keys. We use non-time-based tables all the time, as well as…

Yes, GreptimeDB requires a time index column for optimized storage and querying. It's not a constraint of a primary key, but just an independent table constraint.

Could you elaborate on why you find this inconvenient? I assumed logs, for example, would naturally include a timestamp.

Re: Beyond Elk: Lightweight and Scalable Cloud-Native Log Monitoring

#17
post #16

Am I the only one that got, "This article smells like it was written by an AI told to 'compare these two products'"? Something around the sentence structure just is offputting.

The author is not a native speaker; I promised it's not an AI article but with some minor reviews from AI :)

Re: Beyond Elk: Lightweight and Scalable Cloud-Native Log Monitoring

#19

I'm always skeptical toward software companies with an outdated year in the footer.

a "no brown M&Ms" razor!

From a website perspective, finding the current year can be challenging, but there's always a way to hack around it. For example, by parsing another website to get the year.

Re: Beyond Elk: Lightweight and Scalable Cloud-Native Log Monitoring

#20

I'm always skeptical toward software companies with an outdated year in the footer.

Thanks for pointing it out! The footer has been updated.

Thank you for your prompt attention to this matter. Until next year, then.
Post reply on HN