Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Show HN: Lightpanda, an open-source headless browser in Zig

github.com

Post reply on HN
Print view View original

111–120 of 141 posts

Re: Show HN: Lightpanda, an open-source headless browser in Zig

#111

Post by steeve »

zelcon wrote: post #86
frankgrecojr wrote: post #56

The hello world example does not work. In fact, no website I've tried works. It's usually always panics. For the example in the readme, the errors are: ``` ./lightpanda-aarch64-macos --host 127.0.0.1 --port 9222 info(websocket): starting blocking worker to listen on 127.0.0.1:9222 info(server): accepting new conn... info(server): client connected info(browser): GET https://wikipedia.com/ 200 info(browser): fetch http…

That's Zig for you. A ``modern'' systems programming language with no borrow checker or even RAII.

That has absolutely nothing to do with RAII or safety…

Re: Show HN: Lightpanda, an open-source headless browser in Zig

#112

Post by szundi »

JoelEinbinder wrote: post #7
fbouvier wrote: post #2

Author here. The browser is made from scratch (not based on Chromium/Webkit), in Zig, using v8 as a JS engine. Our idea is to build a lightweight browser optimized for AI use cases like LLM training and agent workflows. And more generally any type of web automation. It's a work in progress, there are hundreds of Web APIs, and for now we just support some of them (DOM, XHR, Fetch). So expect most websites to fail or c…

When I've talked to people running this kind of ai scraping/agent workflow, the costs of the AI parts dwarf that of the web browser parts. This causes computational cost of the browser to become irrelevant. I'm curious what situation you got yourself in where optimizing the browser results in meaningful savings. I'd also like to be in that place! I think your ram usage benchmark is deceptive. I'd expect a minimal bro…

Then came deepseek

Re: Show HN: Lightpanda, an open-source headless browser in Zig

#113

Post by concerndc1tizen »

benatkin wrote: post #64
blacksmith_tb wrote: post #49

Earlier quoted context omitted.

Not sure what "digital rights" that "manages"? I don't see it as an unreasonable suggestion that the tool shouldn't be set up out of the box to DoS sites it's scraping, that doesn't prevent anyone who is technical enough to know what they're doing to fork it and remove whatever limits are there by default? I can't see it as a "my computer should do what I want!" issue, if you don't like how this package works, change…

Digital Restrictions Management, then. Have it your way.

There are so many combative people on HackerNews lately who insist to misinterpret everything.

I really wonder if it's bots or just assholes.

Re: Show HN: Lightpanda, an open-source headless browser in Zig

#114

Post by concerndc1tizen »

tossandthrow wrote: post #44
gkbrk wrote: post #43

Earlier quoted context omitted.

Please don't. Software I installed on my computer needs to the what I want as the user. I don't want every random thing I install to come with DRM. The project looks useful, and if it ends up getting popular I imagine someone would make a DRM-free version anyway.

Where do you read DRM? Parent commenter merely and humbly asks the author of the library to make sure that it has sane defaults and support for ethical crawling. I find it disturbing that you would recommend against that.

> I find it disturbing

Oh no, someone on the internet found something offensive!

Re: Show HN: Lightpanda, an open-source headless browser in Zig

#115

Post by msoad »

nwienert wrote: post #58
fbouvier wrote: post #18

Earlier quoted context omitted.

The cost of the browser part is still a problem. In our previous startup, we were scraping >20 millions of webpages per day, with thousands of instances of Chrome headless in parallel. Regarding the RAM usage, it's still ~10x better than Chrome :) It seems to be coming mostly from v8, I guess that we could do better with a lightweight JS engine alternative.

Playwright can run webkit very easily and it's dramatically less resource-intensive than Chrome.

Does it work nicely on Linux? I'm very curious about this

Re: Show HN: Lightpanda, an open-source headless browser in Zig

#116

Post by audunw »

zelcon wrote: post #86
frankgrecojr wrote: post #56

The hello world example does not work. In fact, no website I've tried works. It's usually always panics. For the example in the readme, the errors are: ``` ./lightpanda-aarch64-macos --host 127.0.0.1 --port 9222 info(websocket): starting blocking worker to listen on 127.0.0.1:9222 info(server): accepting new conn... info(server): client connected info(browser): GET https://wikipedia.com/ 200 info(browser): fetch http…

That's Zig for you. A ``modern'' systems programming language with no borrow checker or even RAII.

If that kind of stuff is always preferable, the nobody would use C over C++, yet to this day many projects still do. Borrow checking isn’t free. It’s a trade-off.

I mean, you could say Rust isn’t a modern language because it doesn’t use garbage collection. But it’s a nonsensical statement. Different languages serve different purposes.

Besides, Zig is focusing a lot more on heavily integrating testing, debug modes, fuzzing, etc. in the compiler itself, which when put together will catch almost all of the bugs a borrow checker catches, but also a whole ton of other classes of bugs that Rust doesn’t have compile time checks for.

I would probably still pick Rust in cases where it’s absolutely critical to avoid bugs that compromise security.

But this project isn’t that kind of project. I’d imagine that the super fast compile times and rapid iteration that Zig provides is much more useful here.

Re: Show HN: Lightpanda, an open-source headless browser in Zig

#117

Post by cies »

concerndc1tizen wrote: post #113
benatkin wrote: post #64

Earlier quoted context omitted.

Digital Restrictions Management, then. Have it your way.

There are so many combative people on HackerNews lately who insist to misinterpret everything. I really wonder if it's bots or just assholes.

Indeed DRM is a very different thing from adhering to standards like `robots.txt` as a default out of the box (there could still be a documented option to ignore it).

Re: Show HN: Lightpanda, an open-source headless browser in Zig

#118

Post by concerndc1tizen »

cies wrote: post #117
concerndc1tizen wrote: post #113

Earlier quoted context omitted.

There are so many combative people on HackerNews lately who insist to misinterpret everything. I really wonder if it's bots or just assholes.

Indeed DRM is a very different thing from adhering to standards like `robots.txt` as a default out of the box (there could still be a documented option to ignore it).

- That's just like, your opinion, man

He was using DRM as a metaphor for restricted software. And advocating that software should do whatever the user wants. If the user is ignorant about the harm the software does, then adding robots.txt support is win-win for all. But if the user doesn't want it, then it's political, in the same way that DRM is political and anti-user.

Re: Show HN: Lightpanda, an open-source headless browser in Zig

#119

Post by internet_points »

bityard wrote: post #30
fbouvier wrote: post #2

Author here. The browser is made from scratch (not based on Chromium/Webkit), in Zig, using v8 as a JS engine. Our idea is to build a lightweight browser optimized for AI use cases like LLM training and agent workflows. And more generally any type of web automation. It's a work in progress, there are hundreds of Web APIs, and for now we just support some of them (DOM, XHR, Fetch). So expect most websites to fail or c…

Please put a priority on making it hard to abuse the web with your tool. At a _bare_ minimum, that means obeying robot.txt and NOT crawling a site that doesn't want to be crawled. And there should not be an option to override that. It goes without saying that you should not allow users to make hundreds or thousands of "blind" parallel requests as these tend to have the effect of DoSing sites that are being hosted on…

Yes! Having long time ago done some minor web scraping, I did not put any work at all into following robots.txt, simply because it seemed like a hassle and I thought "meh it's not that much traffic is it and boss wants this done yesterday". But if the tool defaulted to following robots.txt I certainly wouldn't have minded, it would have caused me to get less noise and my tool to behave better.

Also, throttling requests and following robots.txt actually makes it less likely that your scraper will be blocked, so even for those who don't care about the ethics, it's a good thing to have ethical defaults.

Re: Show HN: Lightpanda, an open-source headless browser in Zig

#120

Post by returnofzyx »

fbouvier wrote: post #107
returnofzyx wrote: post #102

Earlier quoted context omitted.

Hi. Can I embed this as library? Is there C API exposed? I can't seem to find any documentation. I'd prefer this to a CDP server.

Not now but we might do it in the future. It's easy to export a Zig project as a C ABI library.

Oh please do. I'm sure there are many people like me who want this.
Post reply on HN