51–60 of 81 posts
Ah, I see it's pronounced more like "Krull" than "cruel".
lol - yeah. https://www.imdb.com/title/tt0085811/?ref_=tt_urv We've been thinking of ways to make the pronunciation a bit clearer, maybe a mascot or something. Open to ideas!
Earlier quoted context omitted.
Do you have an easy way to transform e.g. a typical Amazon product page into nicely structured data? Not that it's trying to be very video-gamey.
What should the structure look like. If it is CSV what are the columns, i.e., what specific data does it need include. Taking a quick look at the Amazon site these product pages appear to be enormous in size. Interestingly, the website requires a "viewport-width" header. Otherwise one gets directed to a CAPTCHA. The product page I checked already has some structered data in the form of JSON, including keys such as "t…
Earlier quoted context omitted.
I'll make this same offer for Splitgraph :) If you feel like writing a Postgres FDW then we can add it to the engine on the backend, so that anyone with a Postgres client could connect to postgres://data.splitgraph.com:5432 and SELECT from a table backed by crul (either "mounted" for live querying, and/or ingested once/periodically for subsequent querying). The user just needs to provide parameters for the table; it'…
How did you integrate with all those services (the third party service APIs, not asking about the DBs)? I've seen a bunch of sites do this and I'm curious if there's some open source library I should be using, so that I don't need to write from scratch each time I'd like to integrate with another service. Edit: just saw "airbyte" in a bunch of places, which I assume answers my question. So updated question: airbyte w…
Earlier quoted context omitted.
How did you integrate with all those services (the third party service APIs, not asking about the DBs)? I've seen a bunch of sites do this and I'm curious if there's some open source library I should be using, so that I don't need to write from scratch each time I'd like to integrate with another service. Edit: just saw "airbyte" in a bunch of places, which I assume answers my question. So updated question: airbyte w…
We hand wrote a number of integrations, sometimes it was a simple as reusing a schema with slightly different values, we are also using the awesome https://www.benthos.dev/ !
Earlier quoted context omitted.
Yeah the default domain throttle policy is 1 req per second per domain. Configurable through domain policies https://www.crul.com/docs/features/domain-policies - although currently an enterprise feature. We found that it becomes too easy to break API request limits or spam a website otherwise. However if you rerun that query it should load pretty instantly due to the caching layers, so the actual querying/filtering o…
> although currently an enterprise feature. Wait, so we literally can't go faster than 1 req/s unless we pay? I have to say I'm pretty disappointed :/
Don't spam APIs. That said, if you're determined to do so, there's not much this or any other tool can do to stop you from trying.
Screenshot: https://www.dropbox.com/s/i254xxbdenmqde1/screenshot%202023-...
I just signed up and the email confirmation ended up at http://links.outseta.com/ls/click?upn=xKo-2FU5fxLX67yddEnUva... which I get a page that says "Malware and Phishing This site is blocked because it is a known security threat. Please contact your network administrator to gain access." I can't tell if the message is from Brave or from the ISP though.
Can it get text of a web page given just URL, like how pocket app saves the text or the link.
open https://news.ycombinator.com/item?id=34970917 || filter "(attributes.class == 'toptext')" || table innerText
