Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

How we run Firecracker VMs inside EC2 and start browsers in less than 1s

browser-use.com

Post reply on HN
Print view View original

121–130 of 251 posts

Re: How we run Firecracker VMs inside EC2 and start browsers in less than 1s

#121

Post by latchkey »

Reformedot wrote: post #118
latchkey wrote: post #112

Earlier quoted context omitted.

I never suggested reuse.

Oh my bad. You mean warm pools then. That works, yes, but you need to maintain that warm pool, which might not be ready if we receive a big burst of demand Keeping the browser open and warm is also a problem, not all customers require the same features. The same engineering required to fix that (modifying values with Chromium open), also fixes the post-chromium snapshot VM takes 20ms to start, browser around 300ms. P…

if people want custom features, then of course there is a cost to that. but if the majority of your customers are running on defaults, then there is a benefit. yes, it creates other issues, such as pool management, and if you do that wrong and you can't predict capacity well enough, then people get your "slow" path. but, overall, my experience is that the warm pools are extremely well regarded and not something that most people think of.

Re: How we run Firecracker VMs inside EC2 and start browsers in less than 1s

#122

Post by Reformedot »

johnsmith1840 wrote: post #120

Saving state before launch of the browser for quick startup is interesting but how do you configure it? I suppose you don't? Or post configure?

The state does not contain the browser config, since it's configured just before it starts running (and we currently snapshot before Chromium starts).

In our case, we prepare the environment, load files that we need later and then we create the state. Once we start, we instantly start Chromium with the config requested by the customer.

Re: How we run Firecracker VMs inside EC2 and start browsers in less than 1s

#123

Post by choilive »

embedding-shape wrote: post #21
losteric wrote: post #9

> Plain headless Chromium is easy to detect by websites with anti-bot measures. Plain headless Chromium avoided getting blocked by websites only 2% of the time, according to our stealth benchmark. > Our browsers avoid blocks 81% of the time on our stealth benchmark, and 84.8% on Halluminate BrowserBench, the highest of any provider. Seems very unethical, no? Who uses service providers like this? The whole point of an…

> Seems very unethical, no? Who uses service providers like this? The whole point of anti-bot measures is to get rid of bots - you are not wanted there. Unethical just because it does something someone else doesn't want? I guess it depends on why and what the intention is. I don't have time to sit 24/7 in front of a computer to get a ticket to some events, does that mean it's unethical for me to use my own bot so I c…

We use headless browser providers because the companies we interact with don't and won't create a proper API for us to use. Lots of legacy web apps/portals. Saves thousands of man hours.

Re: How we run Firecracker VMs inside EC2 and start browsers in less than 1s

#124

Post by charcircuit »

Reformedot wrote: post #107
sandGorgon wrote: post #55

have you tried running android browsers ? we run RL workloads using android browsers. We are having to maintain a fork of https://github.com/budtmo/docker-android/ and android chrome on top. We would rather use browser-use if it had that support. P.S. we do maintain our fork of a browser for rubric computation...but that is not relevant for this. The infrastructure is what we are looking for.

I've experimented with Android Browsers. The problem is that android VMs are super heavy compared to the resources needed to run just Chromium Startups are absurdly slow, isolation is harder, etc... Android bloat is insane, you need to run the entire Java VM to start the browser... It's also harder to fingerprint, and at scale that's something that we need for Browser Use Cool experiment but not yet production ready,…

>compared to the resources needed to run just Chromium

That Chromium is still running in a VM.

Re: How we run Firecracker VMs inside EC2 and start browsers in less than 1s

#126

Post by otterley »

latchkey wrote: post #112
Reformedot wrote: post #108

Earlier quoted context omitted.

Starting the VM itself takes 20ms with Firecracker, the slowest part is starting the browser. So there's no benefit on reusing the VM but not the browser. VM isolation is also important, customers can leave downloads and other files that should not be accessible for freshly created browsers on that same VM.

I never suggested reuse.

Following up like this--twice--is douchebaggy behavior. If someone misunderstands what you're saying, try elaborating or rephrasing instead. Help make the conversation more productive and less combative.

Re: How we run Firecracker VMs inside EC2 and start browsers in less than 1s

#127

Post by otterley »

latchkey wrote: post #121
Reformedot wrote: post #118

Earlier quoted context omitted.

Oh my bad. You mean warm pools then. That works, yes, but you need to maintain that warm pool, which might not be ready if we receive a big burst of demand Keeping the browser open and warm is also a problem, not all customers require the same features. The same engineering required to fix that (modifying values with Chromium open), also fixes the post-chromium snapshot VM takes 20ms to start, browser around 300ms. P…

if people want custom features, then of course there is a cost to that. but if the majority of your customers are running on defaults, then there is a benefit. yes, it creates other issues, such as pool management, and if you do that wrong and you can't predict capacity well enough, then people get your "slow" path. but, overall, my experience is that the warm pools are extremely well regarded and not something that…

> case in point, it took me saying the same thing twice, for you to catch on

Quit being a dick, Jon. It doesn't suit your stature.

At any rate, warm pools aren't cost free. If you overestimate demand, you'll waste too much money on idle resources.

Re: How we run Firecracker VMs inside EC2 and start browsers in less than 1s

#128

Post by latchkey »

otterley wrote: post #127
latchkey wrote: post #121

Earlier quoted context omitted.

if people want custom features, then of course there is a cost to that. but if the majority of your customers are running on defaults, then there is a benefit. yes, it creates other issues, such as pool management, and if you do that wrong and you can't predict capacity well enough, then people get your "slow" path. but, overall, my experience is that the warm pools are extremely well regarded and not something that…

> case in point, it took me saying the same thing twice, for you to catch on Quit being a dick, Jon. It doesn't suit your stature. At any rate, warm pools aren't cost free. If you overestimate demand, you'll waste too much money on idle resources.

Sure, you're right. I edited to remove that bit. Thanks for calling me out. I was getting frustrated for having felt like I was extremely clear in what I wrote and the person kept repeating something that I had clarified.

> At any rate, warm pools aren't cost free. If you overestimate demand, you'll waste too much money on idle resources.

Depends on how you're running your business. If it is your hardware, it isn't much of an expense at the benefit for having a product that makes your customers happy.

Re: How we run Firecracker VMs inside EC2 and start browsers in less than 1s

#129

Post by messh »

Reformedot wrote: post #82
messh wrote: post #62

Just use something like https://shellbox.dev instead of FireCracker inside ec2. Much simpler, boxes are up in a couple of seconds, and it is way cheaper.

It's not cheaper, slower startup, we lose full control and the environment is not optimized to run Chromium, so we also lose performance

Cheaper by a factor of 2-5 depending on your usage. Read here: https://shellbox.dev/#post-race-to-the-bottom

Startup is fast, less than 2 seconds if you pool connections.

Also, what do you mean by optimized environment for chrome. You can use whatever image you want, use an optimized one if there is such thing

Re: How we run Firecracker VMs inside EC2 and start browsers in less than 1s

#130

Post by otterley »

latchkey wrote: post #128
otterley wrote: post #127

Earlier quoted context omitted.

> case in point, it took me saying the same thing twice, for you to catch on Quit being a dick, Jon. It doesn't suit your stature. At any rate, warm pools aren't cost free. If you overestimate demand, you'll waste too much money on idle resources.

Sure, you're right. I edited to remove that bit. Thanks for calling me out. I was getting frustrated for having felt like I was extremely clear in what I wrote and the person kept repeating something that I had clarified. > At any rate, warm pools aren't cost free. If you overestimate demand, you'll waste too much money on idle resources. Depends on how you're running your business. If it is your hardware, it isn't m…

It depends on how you think about spare capacity on your hardware. They're an opportunity cost. Every idle cycle and bit of unallocated memory could be spent doing something else valuable. Consider the most extreme example of GPUs: CoreWeave was founded by repurposing nearly-useless GPUs their founders had on hand to mine Bitcoin (correction: Ethereum) that was obsoleted by ASICs and other specialized mining hardware (correction: transition to proof of stake).

Besides, they did say they were running on EC2, which charges by the second.

Post reply on HN