Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Deno Queues

deno.com

Post reply on HN
Print view View original

161–168 of 168 posts

Re: Deno Queues

#161

Post by papichulo2023 »

zoogeny wrote: post #36

I think what Deno is aiming for here is actually very forward thinking. I'm using Go for the first time this year and some aspects of the language are very C like. Of the list of things that isn't C like (e.g. GC allowing one to return what look like stack allocated pointers from functions) there is the obvious inclusion of things like `map[string]string`. I bring this up because it struck me that inventing a languag…

Postgres connector actually supports encoding and decoding map[string]interface{} to json and jsonb data types, quite cool.

Re: Deno Queues

#162

Post by michaelcampbell »

postalrat wrote: post #109
londons_explore wrote: post #101

Earlier quoted context omitted.

Is it not normally bad practice to have 2 tables with basically identical fields and move rows between them? Isn't that exactly what indexes were designed for?

If one table had millions of rows and the other only a few dozen and you are querying the few dozen often then it could make sense to have two tables. Different database treat indexes with sparse data differently.

The one table model is exactly what "delayed_jobs" for ruby does (at least with postgres). Furthermore there's no broker; each worker does his own query for the next job/thing/action to take on, so as you scale with more workers it can get really contentiony on the one table.

Re: Deno Queues

#163

Post by michaelcampbell »

bobbyi wrote: post #119
londons_explore wrote: post #101

Earlier quoted context omitted.

Is it not normally bad practice to have 2 tables with basically identical fields and move rows between them? Isn't that exactly what indexes were designed for?

Materialized views handle this scenario, but sqlite doesn't have them

For a background action/job use case though, how often would you have to re-materialize the view?

Re: Deno Queues

#164

Post by mfateev »

xwowsersx wrote: post #138
tonyhb wrote: post #137

Earlier quoted context omitted.

Hot take: I don't think that people should even use queues any more. It's like using raw HTTP2, or managing threads. Use a workflow engine which abstracts queues for you so you can "just write code" without managing jobs/schedules/state/retries. https://www.inngest.com/blog/how-durable-workflow-engines-wo... (disclaimer: I'm biased as I'm an author of a workflow engine)

In the example in your post, the queue is abstracted away in some sense, but you aren't just "writing code". There's step.run(), step.sleep(), step.sleepUntil() sprinkled throughout. I'd say in something like Celery, which is explicitly about running jobs asynchronously backed by a message broker/queue, you really do just write code, but then call the function with .s().apply_async() and so on. Now, I'm not saying yo…

Check out temporal.io that fully abstract this. Disclaimer, I'm one of the founders.

Re: Deno Queues

#165

Post by xwowsersx »

mfateev wrote: post #164
xwowsersx wrote: post #138

Earlier quoted context omitted.

In the example in your post, the queue is abstracted away in some sense, but you aren't just "writing code". There's step.run(), step.sleep(), step.sleepUntil() sprinkled throughout. I'd say in something like Celery, which is explicitly about running jobs asynchronously backed by a message broker/queue, you really do just write code, but then call the function with .s().apply_async() and so on. Now, I'm not saying yo…

Check out temporal.io that fully abstract this. Disclaimer, I'm one of the founders.

hah, I was thinking about temporal as I was writing this. I have played with temporal pretty extensively.

Re: Deno Queues

#166

Post by 59nadir »

hueho wrote: post #133
vineyardmike wrote: post #84

Earlier quoted context omitted.

> If Google started adding Google Cloud specific primitives natively to Go would you call that forward thinking as well? Go actually ships with a quite forward thinking SQL interface. It's an abstract interface over a DB, and you just import the "driver" that powers it. The driver conforms to a standard interface, so all of them behave roughly the same. I think this is what everyone wants from Deno/etc - why can't th…

> Go actually ships with a quite forward thinking SQL interface SQL is low hanging fruit in this regard, because you just need to standardize the lowest common denominator flavors of SQL types for deserialization and then it's just juggling SQL queries around. JDBC for Java does the same as database/sql and it's from 1997. ODBC is from 1992.

I don't think GP is saying that this is forward thinking, revolutionary level stuff, but rather that it's generally speaking a better thing to do than ship specific implementations that wall people into your solutions.

Re: Deno Queues

#167

Post by 59nadir »

giraffe_lady wrote: post #97
paulddraper wrote: post #77

Earlier quoted context omitted.

> Tying a language runtime to a specific KV interface which is tied to a specific hosted service is the opposite of forward thinking. Yeah, I'm hella confused. Isn't Deno the Node.js replacement? But now it's a database as well? It's jumped the shark for sure.

I think people are hung up on a distinction between language and runtime that isn't that valuable and maybe doesn't even reflect normal use. For example erlang ships a persistent KV store, queue, relational database and much more as part of its standard library. I've never heard anyone complain about this or wish it were otherwise.

OTP isn't part of a cloud solution and there is zero lock-in to any kind of commercial offering as part of the batteries that are included in it.

Re: Deno Queues

#168

Post by 59nadir »

lopatin wrote: post #17

> Leveraging public cloud infrastructure has traditionally demanded sifting through layers of boilerplate code and intricate configurations, often monopolizing a significant chunk of the developer’s time and energy. I don't buy this line of reasoning. At the end of the day we are building infrastructure that needs to be reliable. Spending 30 minutes to set up an SQS queue (proven technology) doesn't sound as bad as p…

To extend this into more productive territory, there is no reason I should have to use either the Deno version or AWS; it should be an interface that can be implemented so that I can choose whatever makes sense, including my own implementation.
Post reply on HN