Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Uber’s COO says it’s getting harder to justify money spent on tokenmaxxing

businessinsider.com

Post reply on HN
Print view View original

131–140 of 360 posts

Re: Uber’s COO says it’s getting harder to justify money spent on tokenmaxxing

#131

Post by mike_hearn »

99954bb63ccc wrote: post #108
izanton wrote: post #29

What if... we stop for a moment, and then, after thinking for a moment, we stop hammering nails with a microscope, and stop using token usage as a metric of productivity? I know it's sounds stupid, but what if

I feel like individually, if you sat down with literally any reasonable person on the planet they would arrive at and/or agree with the tenor here. I'd be curious to hear from people well versed in group psychology/dynamics and/or just a lot of leadership/people experience: what leads people to this type of thinking once they get in a group setting? It just... seems endemic at this point. Obviously nobody here is goi…

It's because the average organization has lots of people who don't care about their own productivity and won't adopt new tools or processes unless forced to. This is true of most new tech - lots of workers had to be forced into using computers - but AI also has some other bumps to cross like lots of people who tried early models and then wrote them off, not realizing how fast they'd improve. And most orgs have no infrastructure or processes for allocating individuals token budgets, and most employees have no experience of properly deploying budgets.

Roll it all together and saying "just use it dammit" has some obvious advantages:

1. It's clear.

2. It's simple.

3. It eliminates all excuses employees might come up with for not using it.

The people at the top of these companies aren't stupid. They might have miscalculated how many tokens people can actually use, but that's very hard to calculate because usage is opaque and tools/processes change on a nearly weekly basis. They will eventually build out processes, tools, social conventions and performance metrics that take into account efficiency of token usage. But this is hard! Most managers aren't really assessed on the precise productivity of their teams, for instance, because productivity is often poorly defined.

Re: Uber’s COO says it’s getting harder to justify money spent on tokenmaxxing

#132

Post by BonoboIO »

mrkeen wrote: post #68

I always used to wonder this about software stacks even prior to LLMs, but it seems more relevant now somehow: When will Uber (or your favourite company) be 'done'? They've been writing software for 16 years. They match drivers to passengers. More software isn't going to increase the chance that I seek them out instead of taking a bus or train. Will their software be finished in 20 years? 80?

There is always a rewriting around the corner

Re: Uber’s COO says it’s getting harder to justify money spent on tokenmaxxing

#133

Post by fredley »

7777777phil wrote: post #9

As soon as tokens stop stop being subsidized, heavy agentic use will become as least as expensive than paying an (entry level) employee. When this happens many companies will trade off havy tolen usage for (maybe a bit slower, bit less accurate) employees again.

I think if local models catch up with current SOTA then that might not happen. Either way, I'm don't think the long-term for OAI, Anthropic etc. really holds up.

Re: Uber’s COO says it’s getting harder to justify money spent on tokenmaxxing

#134

Post by nradov »

swed420 wrote: post #120
Lalabadie wrote: post #92

Earlier quoted context omitted.

You're now in the last frame of the comic, getting thrown out the window.

Maybe it's time we adopt/design an economic system that isn't so easily co-opted by counterproductive prisoner's dilemmas.

What would such an economic system look like?

Re: Uber’s COO says it’s getting harder to justify money spent on tokenmaxxing

#135

Post by mmastrac »

I am certain that the max sustainable boost from AI use -- with code review and otherwise all-in -- is approximately 20% with the appropriately skilled senior engineering talent, and the token budget for any engineer should not exceed that.

I do not believe that engineers who are tokenmaxxing are truely productive and I have not seen any evidence whatsoever (perhaps the opposite).

I've personally found that with the right flow and codebase knowledge, that's achievable with sustainable levels of effort.

Re: Uber’s COO says it’s getting harder to justify money spent on tokenmaxxing

#136

Post by Groxx »

iLoveOncall wrote: post #109
SoftTalker wrote: post #102

Earlier quoted context omitted.

Can you provide an example? What is different about running Uber services in Chicago vs. Indianapolis?

For example in Seattle you pay county fees, and then state fees, and then maybe special fees if you were picked up in the airport. I took a ride from SEATAC to my hotel in downtown Seattle and besides the ride itself, there were 5 other items on the bill, 4 of which are specific to the place I used Uber. Then I had the return trip from my hotel to SEATAC, on this one I got EIGHT items on the bill, on top of the ride…

Uber has also been public transit: https://www.theguardian.com/cities/2019/jul/16/the-innisfil-... (like actually public transit, not "lol they reinvented busses again" (though, lol, yes that too))

Essentially every single airport in the world is custom UI and custom walking path guides and pickup instructions, and rules for where pickups/dropoffs/etc can occur can change multiple times in a day, much to everyone's enjoyment. They're almost all private property, and are so valuable that whatever they want is what they get.

And food. Most/~all? major brands get custom integrations.

Hundreds (iirc) of identity verification providers, most or all custom, and constantly weighed against cost and accuracy because it ain't cheap and it ain't good but it is far better than none (both legally and ethically).

No idea how many payment sources they accept, but it's definitely a lot more than anyone who hasn't lived on 5+ continents thinks.

And remember that this is all international. So scale is huge and law changes are constant and frequently conflicting. Darn near every useful feature is illegal somewhere, at some time, for both good and bad reasons.

---

This is not at all to say I think Uber is efficient, clearly it is not. Not by an enormous margin. But there is a legitimate need for truly absurd complexity, because the world is not consistent. You see similar things happen anywhere [thing] tightly interacts with humans.

Re: Uber’s COO says it’s getting harder to justify money spent on tokenmaxxing

#137

Post by MajorBee »

SoftTalker wrote: post #102
goldenarm wrote: post #85

Earlier quoted context omitted.

Most of the codebase is custom integrations for local markets. You can systematize some of it but most of the complexity comes from there.

Can you provide an example? What is different about running Uber services in Chicago vs. Indianapolis?

There's an excellent HN thread that talks about this very question (that comes up on HN every now and then - what _does_ company X do that needs so many engineering resources?): https://news.ycombinator.com/item?id=25375921

TL;DR: Managing a taxi service (that's what Uber is in my mind, not whatever "ride share" means) that spans cities and states, never mind countries, is extremely complicated. To their credit, Uber manages to make it look simple to the end user, prompting such comments as "meh it's just a few screens how hard could it be", which is triumph of product engineering as far as I am concerned.

Related: this blog from Uber talks about the problem of serving market-specific configuration data at scale: https://www.uber.com/us/en/blog/how-we-unified-configuration...

Re: Uber’s COO says it’s getting harder to justify money spent on tokenmaxxing

#138

Post by loeg »

tquinn35 wrote: post #130
mchusma wrote: post #126

I actually do think token maxing is good, but they should have limited it per user. I find it reallly hard to get people to max out the Claude $100 plan, let alone the $200 plan. I understand the enterprise plans are different and more expensive, which is how you get these kinds of issues. But encouraging people to try things with AI is very important, and some amount of token maxing is importsnt.

Who’s it important for?

The business. Employees are hesitant to learn new tools that are very different from what they are used to, so if your business believes that AI is a productivity multiplier, it behooves it to incentivize individual employees to learn to use the tool.

Re: Uber’s COO says it’s getting harder to justify money spent on tokenmaxxing

#139

Post by stusmall »

izanton wrote: post #29

What if... we stop for a moment, and then, after thinking for a moment, we stop hammering nails with a microscope, and stop using token usage as a metric of productivity? I know it's sounds stupid, but what if

That was a fun thought experiment while I waited for my ralph wiggum to finish running. Now thinking is over and back to the vibe

Re: Uber’s COO says it’s getting harder to justify money spent on tokenmaxxing

#140

Post by treis »

devin wrote: post #110
treis wrote: post #95

Earlier quoted context omitted.

Please. These are the same people that force their employees to use Microsoft teams because slack is $5 an employee a month. They're not going to sit idly by while employees burn thousands a month in tokens.

It depends on which people you're referring to. The allocation toward AI budget has been so massive that I think a lot of businesses are way behind on trying to assess value for dollar for the AI-related crud they're shelling out for.

Everyone is feeling it out but the vast majority of spend has been subscription based. Some outliers may have used a massive amount of tokens but companies didn't pay for that.

That VC funded gravy train is likely coming to an end. But fortunately there are also reasonably efficient models now so that the tokenmaxxers can still make the (much cheaper) tokens go brrrr.

Post reply on HN