Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Deno Queues

deno.com

Post reply on HN
Print view View original

131–140 of 168 posts

Re: Deno Queues

#131

Post by BoorishBears »

AgentME wrote: post #127
BoorishBears wrote: post #116

Earlier quoted context omitted.

This instantly reminded me of Next.js, which is open source but has a special build format for serverless environments. The 1st party implementation is closed source: 3rd parties start on the back foot trying to implement alternatives and have to keep up with a 1st party that can move in lockstep. And sure enough, like every other time I see this kind of behavior: Deno was invested in by the CEO of Vercel. "Javascrip…

You can build a Next.js app and run it on a docker container or regular linux host almost anywhere. Vercel has some nice continuous deployment stuff built-in but I'm not sure how a Next.js app is locked into it at all.

This is often repeated but misguided.

Next.js and Vercel heavily push serverless deployment: 13 reworked the built-in API support to leverage Web Standards, which discarded interop with the a much larger server ecosystem in order to enable better edge support.

Serverless deploys require providers to support the Next.js Build API: https://nextjs.org/docs/pages/building-your-application/depl...

There is no open implementation of this API (unlike Remix for example)

This means projects like Open Next start from 0: https://open-next.js.org/

The end result is significantly fractured support for a headline feature of the framework and a lot of unnecessary pain (https://betterprogramming.pub/beware-of-next-js-on-aws-ampli...) trying to leverage it on any non-Vercel platform.

Re: Deno Queues

#132

Post by pests »

wmf wrote: post #100
pests wrote: post #98

I feel its odd for a compiler/language toolchain to have a cloud offering. I understand the motivations but 20% of the article was about their Deno Deploy product and how to calculate API costs... I understand its only when using their cloud and there is a local implementation, but I just couldn't imagine using a feature in LLVM, for example, where if you were to upload the binary to a specific cloud it would use a d…

These features pay for Deno. If they don't exist, Deno doesn't exist.

I agree projects need funding and developers need to be paid...

Deno was released 4 years before Deno Deploy and existed just fine.

Re: Deno Queues

#133

Post by hueho »

vineyardmike wrote: post #84
paxys wrote: post #38

Earlier quoted context omitted.

I disagree. Tying a language runtime to a specific KV interface which is tied to a specific hosted service is the opposite of forward thinking. In fact the tech industry has made a lot of progress away from vendor locked-in stacks, and this just reminds me of those. What is really the difference between Deno.KV being shipped as part of the runtime vs adding an `import KV` statement at the top of the file (which could…

> If Google started adding Google Cloud specific primitives natively to Go would you call that forward thinking as well? Go actually ships with a quite forward thinking SQL interface. It's an abstract interface over a DB, and you just import the "driver" that powers it. The driver conforms to a standard interface, so all of them behave roughly the same. I think this is what everyone wants from Deno/etc - why can't th…

> Go actually ships with a quite forward thinking SQL interface

SQL is low hanging fruit in this regard, because you just need to standardize the lowest common denominator flavors of SQL types for deserialization and then it's just juggling SQL queries around.

JDBC for Java does the same as database/sql and it's from 1997. ODBC is from 1992.

Re: Deno Queues

#134

Post by hueho »

simonw wrote: post #59
ranting-moth wrote: post #57

Pricing? I thought Deno was some sort of node.js replacement. What am I missing and can I use this either locally and/or self hosted without paying for it?

This is the same trick as the rest of Deno KV: the open source version uses SQLite, but when you deploy to their cloud product you get Foundation DB (proprietary) instead: > Since Queues are built on Deno KV, it uses SQLite when running locally and FoundationDB when running on Deno Deploy for maximum availability and throughput. I wrote a bit more about this pattern here: https://til.simonwillison.net/deno/deno-kv

Just as an side note, not relevant to anything in particular, Foundation DB itself is open-source (https://www.foundationdb.org/), but the integration layer used by Deno to make it the backend for Deno KV is not.

Although, from reading the Foundation DB docs and checking the Deno KV API, I honestly suspect it is a thin layer.

Self-hosting FDB is somewhat inscrutable though, so their value add is in not having to handle infrastructure while being backed by FDB.

Re: Deno Queues

#135

Post by aledalgrande »

BoorishBears wrote: post #131
AgentME wrote: post #127

Earlier quoted context omitted.

You can build a Next.js app and run it on a docker container or regular linux host almost anywhere. Vercel has some nice continuous deployment stuff built-in but I'm not sure how a Next.js app is locked into it at all.

This is often repeated but misguided. Next.js and Vercel heavily push serverless deployment: 13 reworked the built-in API support to leverage Web Standards, which discarded interop with the a much larger server ecosystem in order to enable better edge support. Serverless deploys require providers to support the Next.js Build API: https://nextjs.org/docs/pages/building-your-application/depl... There is no open impleme…

Yeah on the same note I developed a moderately complex app on Next but I hit a roadblock when I needed background job support, which is not natively supported (or at least at the time wasn't) on Vercel/other Next platforms and so it was never a priority for Next. Pushing serverless so hard also made deployments janky and production bugs weird when you tried to use things not supported by the underlying platform, AWS (don't remember the details now, but Node version was one of those).

Re: Deno Queues

#136

Post by giovannibonetti »

postalrat wrote: post #109
londons_explore wrote: post #101

Earlier quoted context omitted.

Is it not normally bad practice to have 2 tables with basically identical fields and move rows between them? Isn't that exactly what indexes were designed for?

If one table had millions of rows and the other only a few dozen and you are querying the few dozen often then it could make sense to have two tables. Different database treat indexes with sparse data differently.

Partial indexes solve exactly this problem. You add a WHERE condition that restricts the index to a subset of the whole table.

Re: Deno Queues

#137

Post by tonyhb »

lopatin wrote: post #17

> Leveraging public cloud infrastructure has traditionally demanded sifting through layers of boilerplate code and intricate configurations, often monopolizing a significant chunk of the developer’s time and energy. I don't buy this line of reasoning. At the end of the day we are building infrastructure that needs to be reliable. Spending 30 minutes to set up an SQS queue (proven technology) doesn't sound as bad as p…

Hot take: I don't think that people should even use queues any more. It's like using raw HTTP2, or managing threads.

Use a workflow engine which abstracts queues for you so you can "just write code" without managing jobs/schedules/state/retries.

https://www.inngest.com/blog/how-durable-workflow-engines-wo...

(disclaimer: I'm biased as I'm an author of a workflow engine)

Re: Deno Queues

#138

Post by xwowsersx »

tonyhb wrote: post #137
lopatin wrote: post #17

> Leveraging public cloud infrastructure has traditionally demanded sifting through layers of boilerplate code and intricate configurations, often monopolizing a significant chunk of the developer’s time and energy. I don't buy this line of reasoning. At the end of the day we are building infrastructure that needs to be reliable. Spending 30 minutes to set up an SQS queue (proven technology) doesn't sound as bad as p…

Hot take: I don't think that people should even use queues any more. It's like using raw HTTP2, or managing threads. Use a workflow engine which abstracts queues for you so you can "just write code" without managing jobs/schedules/state/retries. https://www.inngest.com/blog/how-durable-workflow-engines-wo... (disclaimer: I'm biased as I'm an author of a workflow engine)

In the example in your post, the queue is abstracted away in some sense, but you aren't just "writing code". There's step.run(), step.sleep(), step.sleepUntil() sprinkled throughout. I'd say in something like Celery, which is explicitly about running jobs asynchronously backed by a message broker/queue, you really do just write code, but then call the function with .s().apply_async() and so on. Now, I'm not saying you're wrong that we ought to abstract queues away, just pointing out that the abstraction leaks in your workflow example. Happy to hear where/how I'm wrong on this.

Re: Deno Queues

#139

Post by skybrian »

willsmith72 wrote: post #130
skybrian wrote: post #32

Earlier quoted context omitted.

I think this depends on who you are. As a hobbyist programmer, I don't use the big providers like AWS and Google because they seem rather complex? Maybe it's not so bad if you're used to them. I also like to make each project independent, as its own repo on GitHub. Ideally, a web app would be easy for anyone else to launch, as their their own independent web app, using a separate domain name, because I don't want to…

If Deno only suits hobbyist programmers, it's failed. To pick up steam it's going to have to pull people away from AWS and Google who already know how to use them. As for making project's independent, so do I, but that's simple in other clouds too with infra as code or one of the various deployment frameworks (fly lets you do this, AWS copilot does this, terraform, serverless)

I think you might overestimate how much other people know about cloud infrastructure. I am very out-of-date and only vaguely know what these things do.

I looked at Fly and they have nice docs and interesting ideas, but they’re also clear that they don’t do “fully managed” databases and I don’t want to be a DBA.

I know of AWS as a huge pile of complexity that I’m not sure I want to get into? Using it directly seems too low-level for me

(I’ve used Digital Ocean and it seems more my kind of thing, but it doesn’t scale down to zero for a website that gets no traffic.)

Looking at Terraform, it’s not obviously about solving any problem I care about, and wasn’t there a huge controversy about them?

Serverless is a buzzword. Isn’t Deno sort of serverless too?

I think of Deno Deploy as a step up from Netlify, which is fine for static websites. Or maybe a reincarnation of App Engine, which I quite liked back when it launched, but it seems to have lost its way.

Re: Deno Queues

#140

Post by tonyhb »

xwowsersx wrote: post #138
tonyhb wrote: post #137

Earlier quoted context omitted.

Hot take: I don't think that people should even use queues any more. It's like using raw HTTP2, or managing threads. Use a workflow engine which abstracts queues for you so you can "just write code" without managing jobs/schedules/state/retries. https://www.inngest.com/blog/how-durable-workflow-engines-wo... (disclaimer: I'm biased as I'm an author of a workflow engine)

In the example in your post, the queue is abstracted away in some sense, but you aren't just "writing code". There's step.run(), step.sleep(), step.sleepUntil() sprinkled throughout. I'd say in something like Celery, which is explicitly about running jobs asynchronously backed by a message broker/queue, you really do just write code, but then call the function with .s().apply_async() and so on. Now, I'm not saying yo…

It leaks in that the entire function becomes declarative, and state is colocated to a single function. You don't have multiple jobs for each attempt, passing indexes into each job queue. You don't have to worry about enqueueing each "step" as a separate job.

The code becomes "wait until this time", vs "enqueue this function with this state to run at this time via this message broker, which may enqueue other jobs in ways you can't see".

There's no real silver bullet that will let you run code in the future without specifying "when" that code should run — but workflow engines are by far the more productive of the two.

Post reply on HN