I'm trying to OCR a very large book: 45 volumes of ~500 pages each. The digitization has been done (not very good but not too bad either), but the pages have comments in the margin and lots of footnotes. Just doing plain OCR doesn't really work because the notes in the margin and the footnotes get mingled with the text, which results in gibberish. But, when sent to Google Vision API, each page results in a json file…
Are the footnotes in a different font or fontsize? If so, then the bounding box for footnote words should be smaller. Perhaps that can help with categorization.
Ask HN: What are you working on? (October 2024)
931–940 of 1001 posts
Re: Ask HN: What are you working on? (October 2024)
#932I am working on Shepherd.com and trying to reimagine book discovery online. I just launched the "best books of 2024," where I ask readers and authors to share their 3 favorite reads of the year and make it fun to navigate them by different factors (genres, topics, book club reads, audiobooks, and more coming) -> https://shepherd.com/bboy/2024 Slowly getting more in place as we grow :)
One thing I always wonder when thinking about creating something with books is, where does one get all the Metadata fir the books? How do you do that?
Re: Ask HN: What are you working on? (October 2024)
#933I am working on Shepherd.com and trying to reimagine book discovery online. I just launched the "best books of 2024," where I ask readers and authors to share their 3 favorite reads of the year and make it fun to navigate them by different factors (genres, topics, book club reads, audiobooks, and more coming) -> https://shepherd.com/bboy/2024 Slowly getting more in place as we grow :)
That looks pretty cool! Keep up the good work. One thing I always wonder when thinking about creating something with books is, where does one get all the Metadata fir the books? How do you do that?
The metadata system is a nightmare.
To start, we did everything manually.
We source the book cover from the publisher, create author entries, etc.
Eventually, we started paying Nielsen a lot of money to use their Book metadata API. It is "ok," and they don't update it often. But it helps us automatically pull in an author's name, book title, genres, and age-group.
We still manually source the book covers as they only have super small and blurry covers. And we screen every book we add to make sure the data is correct.
What is especially frustrating? - Author names are text and not linked in any way. So we have to decide what is a slightly different name but the same author and what is a different author. - The BISAC genre standard is full of errors and abuse by publishers. For example, they might tag Dune as "AI" which marks it also as being nonfiction because they don't know how the BISAC standard they created works :). - It lists book editions and has no concept that all book editions belong to one book. - No real concept of a book series. - Terrible book descriptions where publishers put in all kinds of reviews and nonsense that we need to figure out how to scrub eventually. They also abuse weird symbols to make it stand out.
It requires a lot of work to fix and manage all of this.
I am about to redo the entire topic/genre system due to some of these problems (this winter).
I am hoping to build a database of all books to use with new features in 2025. I don't know what we are going to do here. We might license a full DB of books from Ingrams (expensive) or Bowker. I liked Ingrams, but Bowker didn't email me back for months and gave me a lot of worry about working with them in the future. I might just do the best I can or break down and use Amazon's API (lots of stipulations in using it).
Some thoughts here of what we've built so far to manage this: https://build.shepherd.com/p/a-big-focus-for-2024-improving-...
Hit me up any time to chat (ben@shepherd.com).
Re: Ask HN: What are you working on? (October 2024)
#934I've noticed that my son spends way too much time on YouTube or playing Minecraft and one of the few offline activities he enjoys doing on his own is coloring. And since he comes to me every time he wants a new coloring book and we spend about 10 minutes together searching for each picture, I made a website with a collection of coloring books for him. The site is very simple, but to be honest, I haven't had so much f…
Re: Ask HN: What are you working on? (October 2024)
#935On the tech side, I built https://svanq.com, a Q&A platform where people ask questions, and others respond with short video clips instead of text. It’s also available on Android: Svanq App- https://play.google.com/store/apps/details?id=com.svanqapp&h...
I'm currently working on another app where users can ask questions with multiple-choice options. Users earn points for answering, which they can later redeem for cash or digital rewards. Previously, I've created social networking apps like Frendsdom and Circleshouts—I love creating social, fun, and unique apps.
I'm also on the lookout for side gigs like technical project management, Freelancing, CTO, partnerships, or any projects that can make a meaningful impact.
If you’re like-minded, let’s connect!
Re: Ask HN: What are you working on? (October 2024)
#936I'm trying to OCR a very large book: 45 volumes of ~500 pages each. The digitization has been done (not very good but not too bad either), but the pages have comments in the margin and lots of footnotes. Just doing plain OCR doesn't really work because the notes in the margin and the footnotes get mingled with the text, which results in gibberish. But, when sent to Google Vision API, each page results in a json file…
Can you talk about what book you’re trying to digitize?
Re: Ask HN: What are you working on? (October 2024)
#937I've noticed that my son spends way too much time on YouTube or playing Minecraft and one of the few offline activities he enjoys doing on his own is coloring. And since he comes to me every time he wants a new coloring book and we spend about 10 minutes together searching for each picture, I made a website with a collection of coloring books for him. The site is very simple, but to be honest, I haven't had so much f…
Re: Ask HN: What are you working on? (October 2024)
#938I've been building a chrome extension called Skipper [1] that helps people to organize their browser tabs with AI. For my whole career so far I've been applying ML (as they called it back then) / AI to various domains like drug discovery and cybersecurity. Both were fun but, man, it feels really different to build a consumer app. It's just very exciting to be able to develop something, push it, and get compliments/co…
Re: Ask HN: What are you working on? (October 2024)
#939I've noticed that my son spends way too much time on YouTube or playing Minecraft and one of the few offline activities he enjoys doing on his own is coloring. And since he comes to me every time he wants a new coloring book and we spend about 10 minutes together searching for each picture, I made a website with a collection of coloring books for him. The site is very simple, but to be honest, I haven't had so much f…
Re: Ask HN: What are you working on? (October 2024)
#940For me the most important issue that needs to be solved right now is the increasing urban sprawl and the car dependent neighborhoods. It causes social isolation. Maintaining infrastructure like roads and electricity, is causing a strain on the economy for local municipalities. Not to mention the disastrous effect car based transportation has on the environment. I am a fullstack developer living in Norway. Last year I…
I love this! There is similar inspiration in the US with non-profits such as KidSafeSF [1] in San Francisco or Families for Safe Streets [2]. I also have a fullstack background and have taken an approach to work on the street design software itself. Although AutoCAD is used by professionals, it's overkill for most street design projects and difficult for laypeople to use. I've been hacking away at a project called 3D…