Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Show HN: Tabstack – Browser infrastructure for AI agents (by Mozilla)

news.ycombinator.com

Post reply on HN
Print view View original

11–20 of 26 posts

Re: Show HN: Tabstack – Browser infrastructure for AI agents (by Mozilla)

#11

Post by jakelazaroff »

mossTechnician wrote: post #10
sippeangelo wrote: post #4

With all respect to Mozilla, "respects robots.txt" makes this effectively DoA. AI agents are a form of user agent like any other when initiated by a human, no matter the personal opinion of the content publisher (unlike the egregious automated /scraping/ done for model training).

I agree with you in spirit, but I find it hard to explain that distinction. What's the difference between mass web scraping and an automated tool using this agent? The biggest differences I assume would be scope and intent... But because this API is open for general development, it's difficult to judge the intent and scope of how it could be used.

What's difficult to explain? If you're having an agent crawl a handful of pages to answer a targeted query, that's clearly not mass scraping. If you're pulling down entire websites and storing their contents, that's clearly not normal use. Sure, there's a gray area, but I bet almost everyone who doesn't work for an AI company would be able to agree whether any given activity was "mass scraping" or "normal use".

Re: Show HN: Tabstack – Browser infrastructure for AI agents (by Mozilla)

#13

Post by 1shooner »

jakelazaroff wrote: post #11
mossTechnician wrote: post #10

Earlier quoted context omitted.

I agree with you in spirit, but I find it hard to explain that distinction. What's the difference between mass web scraping and an automated tool using this agent? The biggest differences I assume would be scope and intent... But because this API is open for general development, it's difficult to judge the intent and scope of how it could be used.

What's difficult to explain? If you're having an agent crawl a handful of pages to answer a targeted query, that's clearly not mass scraping. If you're pulling down entire websites and storing their contents, that's clearly not normal use. Sure, there's a gray area, but I bet almost everyone who doesn't work for an AI company would be able to agree whether any given activity was "mass scraping" or "normal use".

What is worse: 10,000 agents running daily targeted queries on your site, or 1 query pulling 10,000 records to cache and post-process your content without unnecessarily burdening your service?

Re: Show HN: Tabstack – Browser infrastructure for AI agents (by Mozilla)

#14

Post by jakelazaroff »

1shooner wrote: post #13
jakelazaroff wrote: post #11

Earlier quoted context omitted.

What's difficult to explain? If you're having an agent crawl a handful of pages to answer a targeted query, that's clearly not mass scraping. If you're pulling down entire websites and storing their contents, that's clearly not normal use. Sure, there's a gray area, but I bet almost everyone who doesn't work for an AI company would be able to agree whether any given activity was "mass scraping" or "normal use".

What is worse: 10,000 agents running daily targeted queries on your site, or 1 query pulling 10,000 records to cache and post-process your content without unnecessarily burdening your service?

I apprehend that you want me to say the first one is worse, but it's impossible with so few details. Like: worse for whom? in what way? to what extent?

If (for instance) my content changes often and I always want people to see an up-to-date version, the second option is clearly worse for me!

Re: Show HN: Tabstack – Browser infrastructure for AI agents (by Mozilla)

#15

Post by 1shooner »

jakelazaroff wrote: post #14
1shooner wrote: post #13

Earlier quoted context omitted.

What is worse: 10,000 agents running daily targeted queries on your site, or 1 query pulling 10,000 records to cache and post-process your content without unnecessarily burdening your service?

I apprehend that you want me to say the first one is worse, but it's impossible with so few details. Like: worse for whom? in what way? to what extent? If (for instance) my content changes often and I always want people to see an up-to-date version, the second option is clearly worse for me!

No, I've been turning it over in my mind since this question started to emerge and I think it's complicated, I don't have an answer myself. After all, the first option is really just the correlate to today's web traffic, it's just no longer your traffic. You created the value, but you do not get the user attention.

My apprehension is not with AI agents per se, it is the current, and likely future implementation: AI vendors selling the search and re-publication of other parties' content. In this relationship, neither option is great: either these providers are hammering your site on behalf of their subscribers' individual queries, or they are scraping and caching it, and reselling potentially stale information about you.

Re: Show HN: Tabstack – Browser infrastructure for AI agents (by Mozilla)

#16

Post by Findecanor »

sippeangelo wrote: post #4

With all respect to Mozilla, "respects robots.txt" makes this effectively DoA. AI agents are a form of user agent like any other when initiated by a human, no matter the personal opinion of the content publisher (unlike the egregious automated /scraping/ done for model training).

There is still a difference between "fetch this page for me and summarise" and "go find pages for me, and cross-reference". And what makes you think that all AI agents using Tabstack would be directly controlled in real time with a 1:1 correspondence between human and agent, and not in some automated way?

I'm afraid that Tabstack would be powerful enough to bypass some existing countermeasures against scrapers, and once allowed in its lightweight mode be used to scrape data it is not supposed to be allowed to. I'd bet that someone will at least try.

Then there is the issue of which actions and agent is allowed to do on behalf of a user. Many sites have in their Terms of Service that all actions must be by done directly by a human, or that all submitted content be human-generated and not from a bot. I'd suppose that an AI agent could find and interpret the ToS, but that is error-prone and not the proper level to do it at. Some kind of formal declaration of what is allowed is necessary: robots.txt is such a formal declaration, but very coarsely grained.

There have been several disparate proposals for formats and protocols that are "robots.txt but for AI". I've seen that at least one of them allow different rules for AI agents and machine learning. But these are too disparate, not widely known ... and completely ignored by scrapers anyway, so why bother.

Re: Show HN: Tabstack – Browser infrastructure for AI agents (by Mozilla)

#17

Post by ushakov »

> We don't spin up a full browser instance for every request (which is slow and expensive)

there's really no excuse for not spinning up a browser every request. a Firecracker VM boots ~50ms nowadays

> We respect robots.txt rules.

you might, but most companies in the market for your service don't want this

Re: Show HN: Tabstack – Browser infrastructure for AI agents (by Mozilla)

#20

Post by MrTravisB »

ushakov wrote: post #17

> We don't spin up a full browser instance for every request (which is slow and expensive) there's really no excuse for not spinning up a browser every request. a Firecracker VM boots ~50ms nowadays > We respect robots.txt rules. you might, but most companies in the market for your service don't want this

Regarding the browser instances: While VM boot times have definitely improved, accessing a site through a full browser render isn't always the most efficient way to retrieve information. Our goal is to get the most up-to-date information as fast as possible.

For example, something we may consider for the future is balancing when to implement direct API access versus browser rendering. If a website offers the same information via an API, that would almost always be faster and lighter than spinning up a headless browser, regardless of how fast the VM boots. While we don't support that hybrid approach yet, it illustrates why we are optimizing for the best tool for the job rather than just defaulting to a full browser every time.

Regarding robots.txt: We agree. Not all potential customers are going to want a service that respects robots.txt or other content-owner-friendly policies. As I alluded to in another comment, we have a difficult task ahead of us to do our best by both the content owners and the developers trying to access that content.

As part of Mozilla, we have certain values that we work by and will remain true to. If that ultimately means some number of potential customers choose a competitor, that is a trade-off we are comfortable with.

Post reply on HN