Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Using an open model feels surprisingly good

matthewsaltz.com

Post reply on HN
Print view View original

151–157 of 157 posts

Re: Using an open model feels surprisingly good

#151

Post by theshrike79 »

scorpioxy wrote: post #63
Aquitard wrote: post #50

Earlier quoted context omitted.

This is what I've found and would likely be the consenses of the HN community. This has lowered the barrier of entry for many non-SWEs. I'm primarily a data scientist myself in the environmental sector with limited front end development. My partner proposed an idea to help manage her horses and over the course of several weeks we fleshed out an android app that would enable/assist her with horse care. I spent a few d…

Question if you don't mind. Would you have considered outsourcing the application you mentioned? As in, paying someone else to do it? To me it is great that LLMs are allowing more people to use computers and software the way they were meant to. But the people that are using them this way, in my opinion, wouldn't have commissioned anybody to do it anyway. They'd just live with whatever process/pain they have. So jumpi…

If the choices are: Use excel/sheets or something massive in a janky way, pay someone to build it for me (massive cost) or DIY with LLM

The "pay someone to build it" is definitely the one I wouldn't touch, or even consider.

If it's a good idea, they'll already have the code and will run away and make their own business with it. If it's a bad idea, I've wasted a bunch of money.

And in both cases I'd need to spend an inordinate amount of time explaining the terminology and use-cases to the contractor, book meetings with them during office hours to talk about requirements and get demos etc.

With an LLM I can be walking the dog late at night, get an idea, and within 2 minutes I'll have Claude working on it via my phone.

Then I come back home and maybe test whether it worked while watching TV.

If I had fuck you money, I might have a contractor on call that would agree to workflows like that, but I'm just a middle class software dev so I don't :D

Re: Using an open model feels surprisingly good

#152

Post by theshrike79 »

somenameforme wrote: post #136
scorpioxy wrote: post #63

Earlier quoted context omitted.

Question if you don't mind. Would you have considered outsourcing the application you mentioned? As in, paying someone else to do it? To me it is great that LLMs are allowing more people to use computers and software the way they were meant to. But the people that are using them this way, in my opinion, wouldn't have commissioned anybody to do it anyway. They'd just live with whatever process/pain they have. So jumpi…

A practical issue is that a lot of big software is used to solve small problems. For instance with his example, in the past perhaps instead of creating a little custom app it would have been a series of excel sheets with some other third parties tool as needed. And I think this is a very common use case. At most/all companies there tend to be convoluted processes to do relatively simple things, often enriching the bi…

This is _exactly_ it.

The people who were twisting Excel+VBA into massive "applications" in-house should be getting Claude licenses and guidance how to do that with a custom application instead.

We've done that internally already with a few very successful use-cases and a bunch of "this is nice" -level things. I think one case saved us 4 figures a month when we built a bespoke service that does just the things we need and could replace a SaaS that had 420 features - of which we used 2.

Re: Using an open model feels surprisingly good

#153

Post by theshrike79 »

scorpioxy wrote: post #80
Aquitard wrote: post #76

Earlier quoted context omitted.

To answer your question probably not unless I knew the person. I could argue the case that I would have outsourced the android UI scope if I was motivated enough to develop it and kept the backend to myself. I could then tinker with the front end and build upon it. However I wouldn't outsource now if I knew the person would just use LLM anyway. I'm the type that would rather build something myself than purchase off t…

Thank you for answering. I agree with you that, if used well, LLMs can be useful tools and increase efficiency. Unfortunately, what I am increasingly seeing is an increase in confidence but no increase in knowledge or understanding. I hope this is just a side-effect of the hype and corresponding bubble and not the future trajectory of knowledge work.

But it does increase knowledge and understanding.

Not in a deep deep level like "this is the optimal way to manage low level memory access in an Android application" but on a higher level of "It's possible to do a custom Android app in a week that's of decent quality for a small userbase".

Every piece of software doesn't need to be targeted for billion user hockey stick growth and an IPO as a target. It's perfectly fine to build something that's just right for 10 or 20 people. Or just for your immediate family.

For example I have a quiz tool I built with LLM assistance for Christmas that mine and my siblings kids use every December, they get clues appropriate for their level and rewards when they solve the daily task.

It's fine, has custom handling for every user etc. But also something I'd never bothered to build by hand from scratch nor something I'd pay someone to build for me. It was a fun "hmm, I wonder if..." project I started a few years ago on a late evening in November.

Re: Using an open model feels surprisingly good

#154

Post by theshrike79 »

lukeschlather wrote: post #52
aprilnya wrote: post #35

Earlier quoted context omitted.

With OpenRouter this is pretty easy to see - each model page has various providers listed along with an icon marking providers with zero data retention, and the price. You can also set up an account-wide "guardrail" to prevent routing to non-ZDR providers.

OpenAI and Anthropic also claim they don't retain data. I'm not sure what it would take for me to trust any of these inference companies, and why I should trust them more than the big ones.

I trust both OpenAI And Anthropic, they have literal billions to lose if someone finds out they've been training with in-house corporate data.

Just the GDPR/CCPA fines alone WILL bankrupt both companies.

Re: Using an open model feels surprisingly good

#155

Post by mayank »

pimeys wrote: post #141
mayank wrote: post #22

Earlier quoted context omitted.

Oddly enough, I did the exact same thing this past weekend, for a couple hundred usd in Fable overage. It was truly remarkably easy to build and package it exactly to my whims, in my case as a single docker container with a process reaper that runs llama, my Go code, tts, chat harness, browser in xvfb, and even a mailer daemon. With Gemma, it even runs on a RPi 5. What’s absolutely wild to me is that over a couple ho…

Funny. I use Fable a lot at work, but this one I paid from my own pocket and coded it with GLM 5.2. Paid 20 euros in total.

Close to $500 in overage for me :) I’m going to give GLM a shot next.

Re: Using an open model feels surprisingly good

#156

Post by irishcoffee »

anuramat wrote: post #148
irishcoffee wrote: post #43

Earlier quoted context omitted.

… to hack. For fun. To play around. Why do anything?

what's there to hack that you couldn't do with $3 in openrouter credits?

There is value in learning how to launch a model and develop an environment with which to interact with it. This understanding is basically a pillar of a “hacker” ethos.

But if you just wanna outsource understanding for 3 bucks in tokens, that’s cool too.

Re: Using an open model feels surprisingly good

#157

Post by anuramat »

irishcoffee wrote: post #156
anuramat wrote: post #148

Earlier quoted context omitted.

what's there to hack that you couldn't do with $3 in openrouter credits?

There is value in learning how to launch a model and develop an environment with which to interact with it. This understanding is basically a pillar of a “hacker” ethos. But if you just wanna outsource understanding for 3 bucks in tokens, that’s cool too.

do you think typing `make` into a terminal is some sort of an insane intellectual achievement?
Post reply on HN