Viewing profile — portInit

Appreciate you taking the time to play around with crul and share your thoughts! They're incredibly valuable. 1. Although it has limitations, were you able to try the normalize com…

We certainly have, it is admittedly quite new to both of us, so we have been exploring the best way to introduce something like this, as well as tying up technical bits. The open c…

Thank you! Will make this more clear.

Thank you! The query below would get you the full page text, although it likely wont be too legible. I'll read up on readability JS some, it looks quite magic and could possibly be…

I'm having trouble replicating. Although I did get a prompt from Safari to allow downloads from crul.com. The screenshot looks like it is redirecting from the button, is that what …

Yeah exactly. Living up to your username :) nice find! Note this is a global default, unlike domain policies which are associated with a domain.

Thanks for the heads up. Could you share what browser you are using? And are you downloading from https://www.crul.com/account ?

You would need to do a bit filtering to get the exact text you need. For example, to get the text for this post you could run: open https://news.ycombinator.com/item?id=34970917 ||…

We hand wrote a number of integrations, sometimes it was a simple as reusing a schema with slightly different values, we are also using the awesome https://www.benthos.dev/ !

That's unfortunate. A quick look suggests ISP but maybe not with the hotspot. Will do a little more digging.

Thank you! Docker image has everything for functionality, there's just an update check that pings our end to check for updates.

Sorry to hear that - we do need to think about this. It's our first pass at product tiers and features and we may need to adjust. Scheduling and Domain Policies were the main featu…

We'll have to see how close our current postgres integration is! Would like to understand more and will reach out.

Thanks for this, we're still trying to figure out these details ourselves. Related to the quote, we've seen interest for API data wrangling, where prepackaged data feeds can be cum…

Ah! We didn't quite get an html table command in this release but it will be in the next one. Here's a query that shows an option, but the table command will be far more straightfo…

Thank you! Really means a lot to us

Agreed, caching does come with its own set of quirks and mind-numbing bugs, crul does have a caching override flag at the command/stage level which alleviates some of this: https:/…

Yes that is possible, although we currently have the scheduler set as an enterprise feature. We should look into a free trial, I enjoyed the flow of the EasyDataTransform installat…

Would love to chat and try a few use cases together! At first glance I see you can drag in csv files, which could be generated by crul and either manually downloaded or scheduled a…

Yeah the default domain throttle policy is 1 req per second per domain. Configurable through domain policies https://www.crul.com/docs/features/domain-policies - although currently…

Thank you and your comment really warms our hearts. It's been fun building in the "cave" but comes with self doubt. We've built using a microservice architecture to allow us to sca…

This makes sense and I am curious about this. Was there consistency between those 1k client sites or were they all rather different? Mind if I reach out?

lol - yeah. https://www.imdb.com/title/tt0085811/?ref_=tt_urv We've been thinking of ways to make the pronunciation a bit clearer, maybe a mascot or something. Open to ideas!

It's a tricky question. Part of it is looking at APIs as the main source of data for scheduled queries/data feeds. Crul sort of operates as a text only browser when interacting wit…

At this point we're considering it a foundational concept to build around - web content changes, so our best option currently is to make the query as easy as possible to change, an…