Live data from Hacker News

Building an (almost) fully self-hosted, sandboxed, agentic software factory

blog.jakesaunders.dev

71–75 of 75 posts

Re: Building an (almost) fully self-hosted, sandboxed, agentic software factory

#71
post #65

Earlier quoted context omitted.

My favorite real-world example that I worked on: I created a YouTube documentary series about corruption in a small town in Illinois. This required downloading thousands of hours of government meeting videos from YouTube, transcribing + chunking + embedding, summarizing meeting segments, running a couple passes of validation, then searching for interesting storylines. I also scraped thousands of public documents whic…

Do you have a writeup of how you did it, which tools you used? I have a similar project on a much more mundane topic, where I want to build a searchable index of Youtube videos, blog posts, etc. based on their content.

I've been meaning to write something about how I did it, and have been putting it off for a long time. I wrote all the tools myself - by that, I mean that everything was a web app running in my browser.

My advice would be to first set up an automation for yt-dlp to pull the media, then use Whisper to build a transcription pipeline. Chunk the transcript based on desired result granularity, then store embeddings in local Qdrant. It helps to use an orchestrator to handle all of this - my current recommendation is Dagster (dagster.io).

Re: Building an (almost) fully self-hosted, sandboxed, agentic software factory

#72
post #64

Earlier quoted context omitted.

why do people find comments like these necessary? What did you gain by making this comment?

In principle I’d agree usually, but I think this specific case, it’s reasonable to highlight that the “experts get value” argument doesn’t really add much to the discourse anymore. We know this and you just need to look at what’s happening with LLMs in maths to see it’s true. It doesn’t add anything to the verification point however.

While this may be a common understanding in this community, it's worth repeating so that this community can tell everyone else, as I don't think most people understand this currently.

Re: Building an (almost) fully self-hosted, sandboxed, agentic software factory

#74
post #64

Earlier quoted context omitted.

why do people find comments like these necessary? What did you gain by making this comment?

In principle I’d agree usually, but I think this specific case, it’s reasonable to highlight that the “experts get value” argument doesn’t really add much to the discourse anymore. We know this and you just need to look at what’s happening with LLMs in maths to see it’s true. It doesn’t add anything to the verification point however.

But also, it is a post that is self promoting their personal twitter page; there should be a high threshold of doing this, and regurgitating some trite analogy should fall below the bar.

Re: Building an (almost) fully self-hosted, sandboxed, agentic software factory

#75

Neat stuff. I use Hermes with the $20/mo ChatGPT Codex integration, myself, as well as Qwen running locally on a 64GB M1 Max Macbook Pro to help save my ever-quickly shrinking Codex credits. It's been wonderful for stuff like, say, giving Hermes an account on my Home Assistant server (after taking a backup) and having it make a bunch of configurations, or draw conclusions based on historical sensor data (trying to de…

That’s a great use of Hermes. Had it concluded if your A/C is undersized? Before this little experiment I generally used it as a very specific search engine for houses, ‘Find me a house in the country but close to amenities with a workshop barn but close enough on the train to Manchester’. It’s great for this! I’ve never been on the Hermes subreddit I’m gonna go check it out. I agree with the commenter below, it’s fi…

Its conclusion-drawing required a lot more guidance than I expected, but it concluded that my given my house's age (1940s with ducting added later) the A/C is about correct, and was able to tell what part of the house various rooms were in given their solar loading and the temperature differential across them on a sunny day vs a cloudy day.
Post reply on HN