Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Launch HN: Context.dev (YC S26) – API to get structured data from any website

context.dev

Post reply on HN
Print view View original

51–60 of 102 posts

Re: Launch HN: Context.dev (YC S26) – API to get structured data from any website

#51

Post by TheYahiaBakour »

twosdai wrote: post #46
TheYahiaBakour wrote: post #32

Earlier quoted context omitted.

hi, no worries at all, it came off perfectly fine yes, we use residential proxies + all requests go are js-rendered, we maintain a caching layer which is 95%+ opted into by customers it's all included in the credit price, great value compared to alternatives, the business model does rely on scale and our margin gets better the more requests we serve (esp infra cost for k8 + browser fleet) to answer your e.g., yes pub…

What major edge cases have you found so far for the browser scraping method, its really complex like dealing with Auth, or pages that use pop ups, content blockers . etc...

if it needs auth, we don't do any type of logins at the moment, still figuring out how to do it without attracting the wrong type of customer (someone doing shady stuff)

popups / cookie banners are quite easy to get through with enough heuristics for 99%+ of cases

content-blockers are the same thing as the auth scenario, hard to figure out where to draw the line between building something people want and doing something shady

Re: Launch HN: Context.dev (YC S26) – API to get structured data from any website

#52

Post by TheYahiaBakour »

blast wrote: post #34
twosdai wrote: post #29

Are you using residential proxies? How do you handle websites that don't want to be scraped. EG if I start passing in Linkedin pages what is your expectation of the result that people would see per profile. EDIT: Congrats on the launch seriously hard work, just wanting to understand your scraping stance more. I've worked with a lot of tools on this, didn't mean for my initial comment to be adversarial.

Is "residential proxy" a euphemism for botnets or is there a difference? Genuinely curious.

- worst is "extension" based traffic routing where your provider is essentially tricking end users into routing traffic through - opt-in p2p networks can be described as botnets sometimes, i try to avoid these but not as bad - good enough dISP proxies can be classified as residential quality, work quite well, although people have argued w me on their quality - managed business/enterprise residential gateways are ideal, because businesses opt in and allow the traffic - few other options

Re: Launch HN: Context.dev (YC S26) – API to get structured data from any website

#53

Post by Barbing »

dataviz1000 wrote: post #40
TheYahiaBakour wrote: post #39

Earlier quoted context omitted.

Fascinating, this is definitely a path i was thinking about for a bit but decided against doing it primarily since it changes the business fundamentally

You are smart! Another YC startup went this path, scraping targeting endpoints, and they didn't get much traction and they have since pivoted. > We maintain a caching layer and avoid hammering websites. This made me think of it. It is another way to avoid hammering websites by using extremely targeted requests for the most part bypassing HTML, DOM, and JavaScript.

Why not much traction, was the benefit more to websites than businesses/scrapers?

Re: Launch HN: Context.dev (YC S26) – API to get structured data from any website

#54

Post by TheYahiaBakour »

Barbing wrote: post #53
dataviz1000 wrote: post #40

Earlier quoted context omitted.

You are smart! Another YC startup went this path, scraping targeting endpoints, and they didn't get much traction and they have since pivoted. > We maintain a caching layer and avoid hammering websites. This made me think of it. It is another way to avoid hammering websites by using extremely targeted requests for the most part bypassing HTML, DOM, and JavaScript.

Why not much traction, was the benefit more to websites than businesses/scrapers?

it counts as "reverse engineering" a website which is much worse for the website owner imo, then again im biased

Re: Launch HN: Context.dev (YC S26) – API to get structured data from any website

#55

Post by Barbing »

TheYahiaBakour wrote: post #37
korrectional wrote: post #36

Hey Yahia, I recently saw your startup on the YC registry and found it to be one of the most promising of the batch, so good luck with this! Do you mind if I ask how you managed to get your first users on Brand.dev?

hey, happy you like it, to be honest we had no real secret, it takes a ton of time and effort to earn developer trust i got my first customer after 1 month, and then just worked like hell to make him happy even though it was a very large company paying me $99/mo at the time, they're still a customer too since then, i just locked in on every customer until they were thrilled, and once we hit 50, word of mouth started…

Excellent way to run a business.

Re: Launch HN: Context.dev (YC S26) – API to get structured data from any website

#60

Post by SOLAR_FIELDS »

If you want to vibe something that gets you 70% of the way to this well funded startup in like 15 minutes just tell your LLM of choice to create a hook or override the web fetching skill to pipe the content through Mozilla’s readability extension that strips the DOM elements out deterministically, leaving only the content. You can then parse it however you want. Can be done entirely client side, in runtime, with a few JavaScript libraries
Post reply on HN