Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Show HN: DataFuel.dev – Turn websites into LLM-ready data

datafuel.dev

Post reply on HN
Print view View original

1–10 of 37 posts

Show HN: DataFuel.dev – Turn websites into LLM-ready data

#1

Post by sachou »

Just launched DataFuel.dev on Product Hunt last Sunday, and I landed in the top 3!

I built this API after working on an AI chatbot builder.

Scraping can be a pain, but we need clean markdown data for fine-tuning or doing RAG with new LLM models.

DataFuel API helps you transform websites into LLM-ready data. I've already got my first paying users.

Would love your feedback to improve my product and my marketing!

Show HN: DataFuel.dev – Turn websites into LLM-ready data
datafuel.dev

Re: Show HN: DataFuel.dev – Turn websites into LLM-ready data

#4

Post by sachou »

zerop wrote: post #2

Great, congrats on your launch. 1. Does it take care of Bot detection. Most sites will have it. 2. Is this something similar to Firecrawl - https://www.firecrawl.dev/

Yes, it has an extensive proxy IP and retry system in place to bypass bot detection.

I’m also trying to gather more feedback to identify the killer feature:

- Adding vectorization to Pinecone out of the box? - Adding multiple integrations like n8n, etc.?

Any crucial pain points to avoid?

Re: Show HN: DataFuel.dev – Turn websites into LLM-ready data

#5

Post by sachou »

SalmonSnarker wrote: post #3

Projects like this should respect that if a site's robots.txt contains a long list of Disallow entries for other AI scrapers that they are probably not welcome to scrape either. (of course this project doesn't do that)

Good point! Thanks for the feedback.

Let me add that to my todos.

Re: Show HN: DataFuel.dev – Turn websites into LLM-ready data

#6

Post by keyle »

sachou wrote: post #5
SalmonSnarker wrote: post #3

Projects like this should respect that if a site's robots.txt contains a long list of Disallow entries for other AI scrapers that they are probably not welcome to scrape either. (of course this project doesn't do that)

Good point! Thanks for the feedback. Let me add that to my todos.

It boggles my mind that you would launch without that as a prime directive.

Re: Show HN: DataFuel.dev – Turn websites into LLM-ready data

#9

Post by tesrx »

keyle wrote: post #6
sachou wrote: post #5

Earlier quoted context omitted.

Good point! Thanks for the feedback. Let me add that to my todos.

It boggles my mind that you would launch without that as a prime directive.

OP just graciously accepted that feedback, no need to be condescending :)
Post reply on HN