Earlier quoted context omitted.
git still works pretty well, I just wouldn't try to use it as a bank account.
git blockchains are famous for being easy to fork (-;
You can fork a blockchain by building a ecosystem with a different consensus.
AI companies are pivoting from creating gods to building products
161–170 of 210 posts
Re: AI companies are pivoting from creating gods to building products
#162Companies keep going at it the wrong way. Instead of saying "We have AI, let's find products we can make out of AI!" they should be saying, "What products do people want, let's use whatever tools we have (including maybe AI) to make them." The idea that a company is an AI company should be as ridiculous as a company being a Python company. "We are Python-first, have Python experts, and all of our products are made wi…
People want a faster horse
Re: AI companies are pivoting from creating gods to building products
#163Companies keep going at it the wrong way. Instead of saying "We have AI, let's find products we can make out of AI!" they should be saying, "What products do people want, let's use whatever tools we have (including maybe AI) to make them." The idea that a company is an AI company should be as ridiculous as a company being a Python company. "We are Python-first, have Python experts, and all of our products are made wi…
I only partially agree with this. Having spent a lot of time in the “find a problem then the solution” way of working, I’ve found the solutions are often too tame and lack innovation. When you’re truly bring novel new value to things, sometime you need to say “we can do this cool thing, but don’t know what that means”. Simply knowing that capability opens you up to better sets of solutions.
What's wrong with tame and lack of innovation? Sometimes people just need to get things done. There are lots of businesses with basic needs that aren't being met.
Re: AI companies are pivoting from creating gods to building products
#164> But when developers put AI in consumer products, people expect it to behave like software, which means that it needs to work deterministically. If your AI travel agent books vacations to the correct destination only 90% of the time, it won’t be successful. This is the fundamental problem that prevents generative AI from becoming a "foundational building block" for most products. Even with rigorous safety measures i…
The AI travel agent is trivial to solve though. It's the same as the human travel agent. Put the plan and pricing together, then give it to the user to sign and accept. Do it in an app, do it in an email, do it on a piece of paper, whatever floats your boat, but give them something they can review and accept instead of trying to do everything verbally or in a basic chat interface.
I'm not disagreeing with the "needs to work deterministically" -- there is a need for that, but this is a poor example. "Hey robot, plan a trip to Mexico" might still save me time overall if done right, and that has value.
Re: AI companies are pivoting from creating gods to building products
#165> But when developers put AI in consumer products, people expect it to behave like software, which means that it needs to work deterministically. If your AI travel agent books vacations to the correct destination only 90% of the time, it won’t be successful. This is the fundamental problem that prevents generative AI from becoming a "foundational building block" for most products. Even with rigorous safety measures i…
I have a question for folks working heavily with AI blackboxes related to this - what are methods that companies use to test the quality of outputs? Testing the integration itself can be treated pretty much the same as testing around any third-party service, but what I've seen are some teams using models to test the output quality of models... which doesn't seem great instinctively
Take this with a grain of salt because I haven't done it myself, but I would treat this the same as testing anything that uses some element of random.
If you're writing a random number generator, that generates numbers between 0 and 100. How would you test it? Throw your hands up in the air and say nope, can't test it, it's not deterministic! Or maybe you can just run it 1000 times and make sure all the numbers are indeed between 0 and 100. Maybe count up the number frequencies and verify its uniform. There's lots of things you can check for.
So do the same with your LLMs. Test it on your specific use-cases. Do some basic smoke tests. Are you asking it yes or no questions? Is it responding with yes or no? Try some of your prompts on it, get a feel for what it outputs, write some regexes to verify the outputs stay sane when there's a model upgrade.
For "quality" I don't think there's a substitute than humans. Just try it. If the outputs feel good, add your unit tests. If you want to get scientific, do blind tests with different models and have humans rate them.
Re: AI companies are pivoting from creating gods to building products
#166Instead of pivoting, can this behaviour be explained by trying lots of different things and then iterating on the ones that show promise? It's all well and good to say "Make something people want" but for anything that people want usually one of three things is true 1. Someone else is already making it. 2. Nobody knows how to make it. 3. Nobody knows that people want it. People experimenting with 2 and 3 will have a…
That's exactly what these companies are doing. They're trying a lot of different ideas, and seeing what sticks. The problem is that they're annoying users and causing distrust.
Re: AI companies are pivoting from creating gods to building products
#167> But when developers put AI in consumer products, people expect it to behave like software, which means that it needs to work deterministically. If your AI travel agent books vacations to the correct destination only 90% of the time, it won’t be successful. This is the fundamental problem that prevents generative AI from becoming a "foundational building block" for most products. Even with rigorous safety measures i…
> If your AI travel agent books vacations to the correct destination only 90% of the time, it won’t be successful. Well, I don't agree. I think there are ways to make this successful, but you have to be honest about the limitations you're working it with and play to your strengths. How about an AI travel agent that gets your itineraries at a discount with the caveat that you be ready for anything. Like old, cheap sta…
We've had good, free (non ai) media recommendation tools in the past and they got killed by licensing agreements.
AI is creating a post-scarcity content economy where quality is going to be the only driver of value.
If you are the rights holder of any premium human created media content you are not going to let a 'cheap' AI tool get access to recommend it out to people.
Re: AI companies are pivoting from creating gods to building products
#168Earlier quoted context omitted.
How do you know that? Are you a doctor?
Ask chatgpt -> get advice-> double check on Google- > try advice -> my medical issue is solved This has happened to me 3-4 times, hasn't sent me wrong yet. Meanwhile I've had doctors misdiagnose me or my wife a bunch of times in my life. Doctors may have more knowledge but they barely listen, and often don't keep up with the latest stuff.
Hope it will never hallucinate on you. Doctors will start to need to warn against DoctorGPT MD now, not just Doctor Google MD.
Maybe I'm a low-risk guy but I would never follow a medical solution spit out by an LLC. First, I might be explaining myself badly, hiding important factors that a human person would spot immediately. Then, yeah, the hallucinations issue, and if I have to double check everything anyway, well, just trust a (good) professional.
Re: AI companies are pivoting from creating gods to building products
#169Companies keep going at it the wrong way. Instead of saying "We have AI, let's find products we can make out of AI!" they should be saying, "What products do people want, let's use whatever tools we have (including maybe AI) to make them." The idea that a company is an AI company should be as ridiculous as a company being a Python company. "We are Python-first, have Python experts, and all of our products are made wi…
To be fair, astral is the python company and thank god they are.
I love ruff and uv
Re: AI companies are pivoting from creating gods to building products
#170Earlier quoted context omitted.
I only partially agree with this. Having spent a lot of time in the “find a problem then the solution” way of working, I’ve found the solutions are often too tame and lack innovation. When you’re truly bring novel new value to things, sometime you need to say “we can do this cool thing, but don’t know what that means”. Simply knowing that capability opens you up to better sets of solutions.
What's wrong with tame and lack of innovation? Sometimes people just need to get things done. There are lots of businesses with basic needs that aren't being met.
Especially amongst programmers the feeling of constantly pushing boundaries and grinding to a new unexplored territory is far more interesting than working on practical “tame” solutions. See for instance the amount of people who spend tons of time hacking at and opening up platforms for GNU/Linux that are poor fits for the OS. It would make far more practical sense for say Marcan to spend his time working on the Linux kernel more directly than playing around with Apple devices that are made to burn out in a year and become ewaste. However, he feels more satisfaction working on that “new frontier”, regardless of its practicality.