Show HN: I made a tool to clean and convert any webpage to Markdown

#1

Post by asadalt » Sun, Apr 14, 2024, 7:03 PM UTC

My partner usually writes substack posts which I then mirror to our website’s blog section.

To automate this, I made a simple tool to scrape the post and clean it so that I can drop it to our blog easily. This might be useful to others as well.

Oh and ofcourse you can instruct GPT to make any final edits :D

Show HN: I made a tool to clean and convert any webpage to Markdown
markdowndown.vercel.app

Re: Show HN: I made a tool to clean and convert any webpage to Markdown

#2

Post by [deleted] » Sun, Apr 14, 2024, 7:06 PM UTC

[deleted]

Re: Show HN: I made a tool to clean and convert any webpage to Markdown

#3

Post by LeonidBugaev » Sun, Apr 14, 2024, 7:26 PM UTC

One of the cases when AI not needed. There is very good working algorithm to extract content from the pages, one of implementations: https://github.com/buriy/python-readability

Re: Show HN: I made a tool to clean and convert any webpage to Markdown

#4

Post by asadalt » Sun, Apr 14, 2024, 7:28 PM UTC

LeonidBugaev wrote: post #3

One of the cases when AI not needed. There is very good working algorithm to extract content from the pages, one of implementations: https://github.com/buriy/python-readability

oh AI is optional here. I do use readability to clean the html before converting to .md.

Re: Show HN: I made a tool to clean and convert any webpage to Markdown

#5

Post by ChadNauseam » Sun, Apr 14, 2024, 7:33 PM UTC

This is awesome. I kind of want a browser extension that does this to every page I read and saves them somewhere.

Re: Show HN: I made a tool to clean and convert any webpage to Markdown

#6

Post by cratermoon » Sun, Apr 14, 2024, 7:34 PM UTC

I've found htmltidy [1] and pandoc html->markdown sufficiently capable.

1 http://www.html-tidy.org/

2 https://pandoc.org/

Re: Show HN: I made a tool to clean and convert any webpage to Markdown

#7

Post by ElFitz » Sun, Apr 14, 2024, 7:46 PM UTC

ChadNauseam wrote: post #5

This is awesome. I kind of want a browser extension that does this to every page I read and saves them somewhere.

Wouldn’t that describe Pocket, Readwise Reader, Matter, or another one of those many apps?

Edit: read too fast. Didn’t notice the automatic and systematic aspects.

Re: Show HN: I made a tool to clean and convert any webpage to Markdown

#8

Post by ZunarJ5 » Sun, Apr 14, 2024, 7:50 PM UTC

ChadNauseam wrote: post #5

This is awesome. I kind of want a browser extension that does this to every page I read and saves them somewhere.

Wallabag + Obsidian + Wallabag Browser Ext. It's a manual trigger but its great.

Re: Show HN: I made a tool to clean and convert any webpage to Markdown

#9

Post by midzer » Sun, Apr 14, 2024, 7:52 PM UTC

"Failed to Convert. Either the URL is invalid or the server is too busy. Please try again later."

Re: Show HN: I made a tool to clean and convert any webpage to Markdown

#10

Post by julienreszka » Sun, Apr 14, 2024, 7:54 PM UTC

I think it was hugged to death