Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Deno Queues

deno.com

Post reply on HN
Print view View original

141–150 of 168 posts

Re: Deno Queues

#141

Post by quickthrower2 »

skybrian wrote: post #32
lopatin wrote: post #17

> Leveraging public cloud infrastructure has traditionally demanded sifting through layers of boilerplate code and intricate configurations, often monopolizing a significant chunk of the developer’s time and energy. I don't buy this line of reasoning. At the end of the day we are building infrastructure that needs to be reliable. Spending 30 minutes to set up an SQS queue (proven technology) doesn't sound as bad as p…

I think this depends on who you are. As a hobbyist programmer, I don't use the big providers like AWS and Google because they seem rather complex? Maybe it's not so bad if you're used to them. I also like to make each project independent, as its own repo on GitHub. Ideally, a web app would be easy for anyone else to launch, as their their own independent web app, using a separate domain name, because I don't want to…

Be careful, simplicity seekers! Fly.io famous for outages, Vercel/Next.js v13 and app router saga. Simple is the stuff that ain't really changed for 10 years. C# + SQL Server. NodeJS + Postgres, etc. A bit more work at first but well worth it. Will pay off after a good 7 days when you hit the first snag of the new tech.

Re: Deno Queues

#142

Post by willsmith72 »

skybrian wrote: post #139
willsmith72 wrote: post #130

Earlier quoted context omitted.

If Deno only suits hobbyist programmers, it's failed. To pick up steam it's going to have to pull people away from AWS and Google who already know how to use them. As for making project's independent, so do I, but that's simple in other clouds too with infra as code or one of the various deployment frameworks (fly lets you do this, AWS copilot does this, terraform, serverless)

I think you might overestimate how much other people know about cloud infrastructure. I am very out-of-date and only vaguely know what these things do. I looked at Fly and they have nice docs and interesting ideas, but they’re also clear that they don’t do “fully managed” databases and I don’t want to be a DBA. I know of AWS as a huge pile of complexity that I’m not sure I want to get into? Using it directly seems to…

this is the serverless framework im talking about https://www.serverless.com/

but fair enough.

I do think the simplicity of deno will be a trap though. The benefit of a mature framework is you don't have to get lost in the weeds on obscure bugs or functionality with slightly niche use cases.

Re: Deno Queues

#143

Post by singron »

postalrat wrote: post #109
londons_explore wrote: post #101

Earlier quoted context omitted.

Is it not normally bad practice to have 2 tables with basically identical fields and move rows between them? Isn't that exactly what indexes were designed for?

If one table had millions of rows and the other only a few dozen and you are querying the few dozen often then it could make sense to have two tables. Different database treat indexes with sparse data differently.

In Postgres this effect is huge since there aren't primary indexes. All tuples for a table get mixed up together in the heap, so a large table will tend to having only 1 hot tuple per page. Additionally, every update and delete leaves a dead tuple to implement MVCC, and these are much easier to clean up with VACUUM on a small table.

I'm not as familiar with sqlite, but it might have a similar problem in this case since the primary clustered index is on rowid and not a value that correlates with whether the queue item is running.

Re: Deno Queues

#144

Post by rprwhite »

endisneigh wrote: post #110
dsherret wrote: post #71

Earlier quoted context omitted.

> Tying a language runtime to a specific KV interface which is tied to a specific hosted service is the opposite of forward thinking. This is not the case. The Deno runtime itself is not tied to the Deno Deploy hosting service. The KV feature in the Deno runtime can be used without the hosting service. You can read the details about how Deno KV works in the Deno runtime here: https://til.simonwillison.net/deno/deno-k…

This is not true, you cannot run your own foundationdb server and use the kv service without reimplementing yourself in fdb

This sentiment has been repeated in a few comments. But, why can’t the deno deploy implementation be reimplemented, by yourself, by running a foundationDB server with mvSQLite[1]? That shouldn’t require any changes to the code.

[1] https://github.com/losfair/mvsqlite

Re: Deno Queues

#145

Post by drdaeman »

londons_explore wrote: post #101
simonw wrote: post #67

I dug into the internals of the local, SQLite version of this just now and wrote up some notes here: https://til.simonwillison.net/deno/deno-kv#user-content-deno... The most interesting detail is probably the schema they're using for that: CREATE TABLE queue ( ts integer not null, id text not null, data blob not null, backoff_schedule text not null, keys_if_undelivered blob not null, primary key (ts, id) ); CREATE TA…

Is it not normally bad practice to have 2 tables with basically identical fields and move rows between them? Isn't that exactly what indexes were designed for?

Well, many SQL databases, although not SQLite, have table partitioning feature which does exactly this. The main benefit is that one can configure tablespace (in PostgreSQL terms; essentially the drives to store data) and storage parameters separately for each partition.

Indexes are just the tip of the iceberg (but sometimes just using partial indexes may do wonders), it’s ability to tweak stuff like fillfactor for a frequently updated small portion of the table (e.g. active sessions vs archived sessions) is that makes a lot of difference.

Re: Deno Queues

#146

Post by plaguuuuuu »

londons_explore wrote: post #102

> at least once semantics. > user code. A past life has taught me that users will never properly understand at-least-once semantics. Anytime you redeliver messages, you will get a flurry of user complaints and breakage. Either you do the impossible and invent a way to do exactly-once semantics. Or you should always redeliver 0.1% of all messages, just so that users don't come to depend on messages being delivered onc…

If messages need to be idempotent, I definitely recommend creating integration tests to ensure this from the sending side :-)

Re: Deno Queues

#147

Post by tmikaeld »

meowtimemania wrote: post #105
tmikaeld wrote: post #29

Why not instructions on how to use it with FoundationDB so that we can host it on our company infrastructure?

I think you can add your own implementation to Deno.openKv so that you can use your own companies infra.

That requires knowledge of rust and how deno work internally, it would probably be far from production ready…

Re: Deno Queues

#148

Post by baq »

tmikaeld wrote: post #29

Why not instructions on how to use it with FoundationDB so that we can host it on our company infrastructure?

How do you think they make money?

Re: Deno Queues

#149

Post by nsonha »

015a wrote: post #50
Exoristos wrote: post #21

Earlier quoted context omitted.

Node is not a language.

Since you're going to be pedantic, I will be too. Language is more than syntax. Language is the combination of Syntax and Meaning; its a system of communication. The word "Fooloofol" is syntactically correct English, but meaningless within the English language; and thus is not English. The english word "bark" is syntactically identical across more than one meaning; a dog barks next to the tree bark. Language isn't ju…

I agreed with it until you injected the irrelevant bit about HTML and CSS being programming languages. It is ARGUABLE true, but very unhelpful, you are not giving learners useful direction when you use a blanket term ("programming language") to refer to vastly different things that require different mindsets (js & other languages vs html, css, yml...) to master.

Re: Deno Queues

#150

Post by erlend_sh »

simonw wrote: post #67

I dug into the internals of the local, SQLite version of this just now and wrote up some notes here: https://til.simonwillison.net/deno/deno-kv#user-content-deno... The most interesting detail is probably the schema they're using for that: CREATE TABLE queue ( ts integer not null, id text not null, data blob not null, backoff_schedule text not null, keys_if_undelivered blob not null, primary key (ts, id) ); CREATE TA…

> I find this particularly interesting in terms of open source business models: they're baking a core feature into their framework which their SaaS platform is uniquely positioned to offer as a global-scale upgrade.

That’s quite novel, but I also find it a bit unnerving. They gotta commercialize of course, but they didn’t have to go fully closed. They could’ve just used a different license (BSL, PolyForm..) for the scaling layer.

Post reply on HN