Live data from Hacker News

CouchDB 3.0

blog.couchdb.org

91–100 of 161 posts

Re: CouchDB 3.0

#91
post #59

Earlier quoted context omitted.

there's always one...as in there is always one who has to poke people for grammer. sighs.

Personally I wouldn't mind corrections on grammar. But I agree that the comments about grammar, even though in my opinion somewhat useful, still add noise to the discussion as a whole. I've been thinking about this and, I think having a "grammar patrol" that had the ability to alter other people's comments for grammar on a site like HN or Reddit might be useful. Maybe. Of course there is potential for abuse. So you'd…

That would actually be an interesting idea to pursue because I don't think it's even decidable and may require an AI/human to write back to understand. It depends on context that may not even be there. (Think of the absurdity of chain-comment-memes on Reddit, for example. The question may not even be answerable until lots of other people have replied. Now, you may not permit that as "grammar", but it speaks to the complexity of even defining grammar. Words are repurposed, etc. etc.)

Maybe... just maybe memes could be the key to AI /s

Re: CouchDB 3.0

#92
post #85
post #80

Earlier quoted context omitted.

Thank you for pointing at validation, I'll check it. It's not completely clear what is it possible to limit not only particular document but database, or how to handle conflict if document changed on pouch, but rejected on couchdb server. I'm not sure about current time but previously it was a problem that couchdb file grow until some limit on filesystem and couchdb just crashed. Start of the envoy readme: it's not b…

Also as far as database size, I don't believe there is a hard limit. I think you might be thinking of when MongoDB would silently corrupt databases larger than 2GB on it's 32-bit version.

As far as I remember it was a filesystem limit not couchdb limit. It was a problem that file always grow and couchdb crashed when limit exceeded. Can't find particular issue, but googling show some issues [1] that make me think that we should be very careful with db size.

[1] https://stackoverflow.com/questions/40752578/couchdb-views-c...

Re: CouchDB 3.0

#93
post #59

Earlier quoted context omitted.

it's = "it is" its = possessive form of "it" So should be "it wins for its unique..".

there's always one...as in there is always one who has to poke people for grammer. sighs.

In his original message right after "it's" he put "(I believe?)", implying he was unsure about the usage and inviting feedback so I responded to that. If I misread, my bad, but I thought he was specifically asking.

Re: CouchDB 3.0

#94
post #89
post #83

Earlier quoted context omitted.

Is it any documents that describes secure couchdb architecture? Most of the articles I find are limited to authentication and basic permissions.

What kind of document are you looking for here? There is [1], but yeah, that covers access controls. As do the MongoDB [2] and Postgres [3] documents. I feel like your thinking about Couch as exposing your entire PostgreSQL DB to the internet, whereas with couch, a common model is to have a single database per user. In the Postgres model, providing the end user with any direct access is a nightmare, because every oth…

> What kind of document are you looking for here? > There is [1], but yeah, that covers access controls. As do the MongoDB [2] and Postgres [3] documents.

Mongo and postgress usually is not accessible for clients only for backend. Security handled by backend mostly and there is a plenty of resources how to implement secure server side applications which discusses attack vectors and how to make secure apps. Thankfully to this thread I’ve got few good ideas, that may help to design secure couchdb architecture (such as remove _find endpoint) but I’ve not seen any in-depth document about couchdb.

> I feel like your thinking about Couch as exposing your entire PostgreSQL DB to the internet

No, why do you think so?

Re: CouchDB 3.0

#95
post #18

Reducing max document size from 4GB down to 8MB seems hyper-restrictive. For those interested, looks like the guts of CouchDB are going to be swapped out for FoundationDB. https://blog.couchdb.org/2020/02/26/the-road-to-couchdb-3-0-...

8MB is just the default, you can switch it back to 4GB if you want, but you won't have an easy time switching to 4.0 due to the 8MB limit imposed by FoundationDB.

It seems really odd to me that they are changing to an 8MB limit. "Document storage database" that can only hold 8MB per document. It can't hold full text output for a large documents... What am I missing here?

Re: CouchDB 3.0

#96
post #88
post #84

Earlier quoted context omitted.

I'm not clear what you mean by limiting "not only particular document but database". As far as a document changing in pouch and rejected on the server, that's one of two scenarios. 1) The client you wrote is bugged and generated bad data. This scenarios can occur just as easily using Postgres and an application server. What does your app server do if a client tries to send bad data? (Answer: Whatever you told it to d…

> I’m not clear what you mean by limiting "not only particular document but database". I’ve limited document size to 10mb and ratelimited updates to 10 per second. Client starts to update document with random data 10 requests per second. As far as I understand couch stores all versions at least some time. This means that this one client could fill space on my server 100mb/s. There is no such issues with postgress, an…

> I’ve limited document size to 10mb and rate-limited updates to 10 per second. Client starts to update document with random data 10 requests per second. As far as I understand couch stores all versions at least some time. This means that this one client could fill space on my server 100mb/s. There is no such issues with PostgreSQL, and no one allow clients execute raw queries on database without any application server. Document only 10mb but database is huge.

Ah! Now we are getting somewhere! Your concerned about someone filling your disk.

OK, let's modify your scenario a little. Instead of updating an existing document, they create a new document. This a malicious client, why do updates that'll get cleaned up in a few minutes when I can make it permanent?

So, CouchDB allows these writes, and now your disk is full.

What does Postgres with a custom API do? Allows these writes, and now your disk is full.

Your allowing 10MB documents because that makes sense for your application right? So your Postgres table is going to have a binary column or some other column meant to hold bulk data, and your API is going to accept it.

If it doesn't make sense, lower the max document size. Apply validations to limit what fields can be written to, and how big they can be. In Postgres this is called your "schema". Couch being "schemaless", it's now your validation function. Couch is no different from any other schemaless database such as Mongo, RethinkDB and FoundationDB in this regard.

Also your rate limiting here is weak. If I can post to your sever at 100Mb/s second, I can saturate a 1GB link with only 10 clients. Doesn't matter if you reject my posts, if I can send them to the server, I can DOS you pretty easily.

The main thing Postgres gives you here is that it requires you to define your schema upfront (unless you use JSON columns, in which case it joins the schemaless club above). Couch will happily let you not, in which case someone wants to write a record of their car maintenance into your recipe book app? Couch is good with that. But take a step back. what actually stops them from putting that in the "description" column of your Postgres recipe app? Not much. So you have to think about what's important. Do I actually need to make sure these are all the same "shape"? If so I need a validation function. If I can just shrug and say "garbage in, garbage out", then I just need controls around how much data they can insert, but hey, I needed that for Postgres anyway.

> Sorry for my ignorance, is it true that if I limit couch only to replication it will not be any not indexed lookups?

Correct (enough). The entirety of CouchDB is built around efficient replication. While it's not going to use a formal "index" getting all of the changes after a specific rev is an efficient operation.

Re: CouchDB 3.0

#98
post #96
post #88

Earlier quoted context omitted.

> I’m not clear what you mean by limiting "not only particular document but database". I’ve limited document size to 10mb and ratelimited updates to 10 per second. Client starts to update document with random data 10 requests per second. As far as I understand couch stores all versions at least some time. This means that this one client could fill space on my server 100mb/s. There is no such issues with postgress, an…

> I’ve limited document size to 10mb and rate-limited updates to 10 per second. Client starts to update document with random data 10 requests per second. As far as I understand couch stores all versions at least some time. This means that this one client could fill space on my server 100mb/s. There is no such issues with PostgreSQL, and no one allow clients execute raw queries on database without any application serv…

It’s trivial to limit number of created documents in postgres, couchdb or application server though validation, I’m talking about updating document not creating new. In posgres if I update 1mb document used space will not always grow. In couch db situation is different. In case of relation db you have application server with custom logic and validations, couchdb from other side is accessible from outsize.

My idea that it’s very hard to create safe couchdb based system and most recommendations limited to setup nginx proxy and authenticate users which is not enough.

Re: CouchDB 3.0

#99
post #95

Earlier quoted context omitted.

8MB is just the default, you can switch it back to 4GB if you want, but you won't have an easy time switching to 4.0 due to the 8MB limit imposed by FoundationDB.

It seems really odd to me that they are changing to an 8MB limit. "Document storage database" that can only hold 8MB per document. It can't hold full text output for a large documents... What am I missing here?

In CouchDB 4.0 the backend of CouchDB will be switching to FoundationDB, which will have a 8MB limit, so they're preemptively making the change. You can remove the limit now if you'd like.

Re: CouchDB 3.0

#100

Earlier quoted context omitted.

MongoDB is a an actual for-profit company, so of course it has marketing. CouchDB is an open source apache project. You are comparing apples to oranges.

Its not just CouchDB, I did a lot of searching a little while back and I generally just don't like how tightly they've wrapped up almost all of the results. If you go by searching alone its overwhelmingly MongoDB positive to such an extent that its hard to believe its organic. It's definitely impressive marketing but when I'm deciding on which tool to use that arguably works against them as opposed to for them.

So you are choosing software on it's technical merit and not just on popularity and hype!? What if there are no breaking changes every third month, what are you gonna do? Solve actual problems!? :P
Post reply on HN