Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The Waymo effect: how AI is quietly making research less collaborative

researchagenda.news

Post reply on HN
Print view View original

171–180 of 308 posts

Re: The Waymo effect: how AI is quietly making research less collaborative

#171

Post by noir_lord »

01100011 wrote: post #93
Frost1x wrote: post #32

I work at an intersection of tech, applied research, and science. Something I’ve noticed in collaboration that does occur is an increased confidence in people outside their domains to say things with conviction. I have people who have limited experience with software pushing out layers and layers of abstracted code that’s fairly sophisticated but often misguided in intent who will say what they’re doing is correct, w…

Idk, seems like overconfidence outside your domain really got going in the '00s and is largely perpetrated by SWEs. Something about making a lot of money and living in a world of abstractions really seems to fuel our sense of overconfidence in our abilities in other areas.

It's not just SWE's we just see it more in that pool because we mostly (we are on HN after all) swim in that pool.

I've eaten dinner with Doctors (medical) where they've confidently spoken about a subject I am more experienced in than them and they've been hilariously far off the mark, I just nod and ask for the gravy - not picking a fight over a meal.

Prof. Nathan Ballantyne coined the term Epistemic Trespassing to describe it, it is rampant and LLM's just act as an amplifier.

It's like people have a collective fear of just admitting they don't know something, there are a vast number of disciplines/topics I'm not competent or knowledgeable enough to have a meaningful opinion about even if I find the topic interesting, it's just been smart enough to realise that in the end.

Worst case, find the person who does know and ask then both of you might know (assuming you understand the answer of course which isn't guaranteed, I like physics, I'd understand about one word in ten if a physicist actually described it the way they would to another physicist).

Re: The Waymo effect: how AI is quietly making research less collaborative

#172

Post by bbmatryoshka »

alcomatt wrote: post #90
simianwords wrote: post #81

Earlier quoted context omitted.

I think it is amazing and it will democratise information. A lot of times the concept ends up being simple with a lot of jargon exists as a gatekeeping method. Well of course the incumbents wouldn't like that rando from outside gets to understand and speak about things they worked on for many years. Sure he doesn't get it right 100% but he's in the correct direction.

I very much doubt it will do that because access to that information was already available to anyone who wanted to know. It was not a secret. All AI did was to provide it via single interface, for a fee - which strangely do not go back to the original source of the knowledge, but to the AI company these days. This is not democratisation of information, it's commodification of it.

you can say the same of google, or libraries, information technology is the technology of moving and rearranging information, not of creating it

Re: The Waymo effect: how AI is quietly making research less collaborative

#173

Post by datsci_est_2015 »

tacomonstrous wrote: post #128
fidotron wrote: post #89

Earlier quoted context omitted.

Physics, and to a large degree pure mathematics, has attracted a critical mass of the personality type that needs to feel like the smartest person in the room in any given situation. It has given them a slightly notorious reputation in academia.

As a mathematician, this is far from my experience of actual professional mathematicians, because they encounter people much smarter than them quite often. Maybe more true among graduate students, however.

This thread started by referencing grad students, and I can vouch for it from personal experience with physics grad students.

Re: The Waymo effect: how AI is quietly making research less collaborative

#174

Post by app134 »

huurtehoog wrote: post #79
zzzeek wrote: post #70

Earlier quoted context omitted.

it's hard to gauge this take without specifics. My experience with LLMs tends to be towards the opposite concept; LLMs dissuading me of hunches and notions I have about things (where I have no particular expertise; societal-level things), saving me and others time and strife having an argument about something they were actually right about all along, as my internal doubts that I'm too embarrassed to bring up (because…

LLM tools tend confirm what you come to them with, be it unfounded conviction or doubt. I find them quite useless to tell me things I don't know, as verifying what they say shows that any non-trivial claims they make are usually oversimplified or plain wrong. Anything more than using them to just pointing me to media and literature is usually a waste of time.

That's interesting. Lately I have found the exact opposite to be true; the frontier models are quite disagreeable and require some convincing

Re: The Waymo effect: how AI is quietly making research less collaborative

#175

Post by chunky1994 »

fidotron wrote: post #111
neutronicus wrote: post #98

Earlier quoted context omitted.

From a one-time physics grad student, I do think there advantages to being ... like this. The flip side of being confidently wrong sometimes is that we treat understanding new systems as something approachable. So, yeah, it's annoying that we don't stay in our lane, but, from my boss's perspective, I'm the only one who's actually willing to just read code, investigate, possibly read a paper, and figure out what's goi…

Like your sibling comment I saw this mainly with physics people doing data science: > Eventually you have to understand something. And part of that process is being wrong and annoying people. There is nothing more frustrating as a colleague than having a smart person walk in, insist you're all wrong, and then burn all the energy and goodwill in the organisation having to rebuild absolutely everything from the princip…

But there is a difference between not listening to experience and examining whether the resulting implementation/decisions based on the experience are actually valid. I doubt that someone smart (as you put it) would deliberately want things redone because they don't understand it, that's a trait borne of arrogance not intelligence.

At least in my experience (as a physics grad), me and all my fellow grads are much more likely to apply Chesterton's fence and then make decisions than not doing that and steamrolling existing decisions.

Re: The Waymo effect: how AI is quietly making research less collaborative

#176

Post by ricksunny »

teiferer wrote: post #161
meowface wrote: post #4

This article is written by an LLM, by the way. ("Quietly" is... as Claude might put it... "often the quiet tell".)

Do you have more evidence than this? I'd honestly like to know. I don't like AI slop like everybody else, but I'm also growing skeptical of the very fast determinations that sth is supposedly made by LLM just because it uses some phrase that Claude also uses. Models are trained on text written by human writers and if human text resembles it, this goes both ways. I have looked at the authors work from the pre-AI era a…

"every hour spent aligning with a co-author is an hour not spent producing output that is legible to an evaluation system." use of the word 'legible'.

"We defund the corridor and then wonder where the corridor conversations went." this is a claude-ist construction.

"This is an old worry wearing new clothes" this is a very claude-ist construction.

(rather ironically... as @embedding-shaep pointed out earlier): "Outsource the writing and you have not accelerated the thinking; you have skipped it." 'not X but punchy-Y' (not to be confused with 'not X but profound-Y'.

I'm not saying the author shouldn't have used an LLM - once in a while I suppose it gets used to good effect. But I wouldn't kid myself that the writing is certainly all human through-and-through.

[Incidentally, in case anyone's wondering where many of these claude-isms come from, just search within lesswrong.com .]

Re: The Waymo effect: how AI is quietly making research less collaborative

#177

Post by saghm »

neutronicus wrote: post #98
fidotron wrote: post #89

Earlier quoted context omitted.

Physics, and to a large degree pure mathematics, has attracted a critical mass of the personality type that needs to feel like the smartest person in the room in any given situation. It has given them a slightly notorious reputation in academia.

From a one-time physics grad student, I do think there advantages to being ... like this. The flip side of being confidently wrong sometimes is that we treat understanding new systems as something approachable. So, yeah, it's annoying that we don't stay in our lane, but, from my boss's perspective, I'm the only one who's actually willing to just read code, investigate, possibly read a paper, and figure out what's goi…

You can get pretty much all the same benefits with additional upside and fewer downsides by just showing curiosity and eagerness to learn from others' expertise. Yes, on the internet you might get better answers by being confidently wrong rather than asking for help, but when talking to other humans directly you'll get a lot more people who will just try to end the conversation as quickly as they can if you're arrogant, whereas people tend to respond really well to respecting their intelligence and experience.

I've personally found that doing the opposite of what you say and being willing to look like the dumbest person in the room by asking whatever questions I need to for my own understanding often ends up working out quite well; people are often wonderfully willing to share their knowledge with me even if I feel like I'm asking something very basic as long as I'm actually being friendly and humble, and there are plenty of times where there have been others who told me they have same question but didn't feel comfortable asking.

In some ways, it's similar to when I played bass in two different bands in college. In one of them, I was probably the most talented musician in the group, and it was fairly boring for me to practice with them, but we needed it as a group, whereas the other was with a friend of mine who was an insanely talented guitarist possibly more talented in music than pretty much anyone else I ever met. I loved practicing in the second one because I'd learn so much from him just by exposure and getting to pick his brain, but I imagine it got boring for him sometimes as well having to wait for me to "catch up" to what was easy for him. Being the smartest one in the room feels like it would get old after a while, but having smarter people around to learn from is endlessly engaging.

Re: The Waymo effect: how AI is quietly making research less collaborative

#178

Post by chunky1994 »

Frost1x wrote: post #32

I work at an intersection of tech, applied research, and science. Something I’ve noticed in collaboration that does occur is an increased confidence in people outside their domains to say things with conviction. I have people who have limited experience with software pushing out layers and layers of abstracted code that’s fairly sophisticated but often misguided in intent who will say what they’re doing is correct, w…

But isn't there value to refuting what Claude is saying if it deserves to be refuted?

The way I see it, if I was repeatedly getting sophisticated but subtly wrong arguments about my work that require me to understand why said argument is wrong, that's essentially drilling down to specifics of what precisely needs to be true to solve the problems I am aiming to solve. There is enormous value to this precision, isn't there?

Re: The Waymo effect: how AI is quietly making research less collaborative

#179

Post by pastel8739 »

Frost1x wrote: post #32

I work at an intersection of tech, applied research, and science. Something I’ve noticed in collaboration that does occur is an increased confidence in people outside their domains to say things with conviction. I have people who have limited experience with software pushing out layers and layers of abstracted code that’s fairly sophisticated but often misguided in intent who will say what they’re doing is correct, w…

What do you suggest these people do instead? I’ve been frustrated by this recently: I pivoted to a new subfield and am working on stuff that I would love to dive deep into and really learn what is going on so I can speak intelligently about the tradeoffs etc. But that would take a long time, and I have tasks that I should get done. So I have found myself working with a pretty vague understanding that, when pressed by coworkers, quickly finds its limits. Then I go back and try to deepen my understanding enough to cover those limits. But because I’m not working with each detail of the problem, implementing line by line with time to think about what’s happening, there just isn’t time for me to learn this unfamiliar topic. But I would love to, and I would enjoy the work much more if I could. So what am I supposed to do?

Re: The Waymo effect: how AI is quietly making research less collaborative

#180

Post by giva »

huurtehoog wrote: post #79
zzzeek wrote: post #70

Earlier quoted context omitted.

it's hard to gauge this take without specifics. My experience with LLMs tends to be towards the opposite concept; LLMs dissuading me of hunches and notions I have about things (where I have no particular expertise; societal-level things), saving me and others time and strife having an argument about something they were actually right about all along, as my internal doubts that I'm too embarrassed to bring up (because…

LLM tools tend confirm what you come to them with, be it unfounded conviction or doubt. I find them quite useless to tell me things I don't know, as verifying what they say shows that any non-trivial claims they make are usually oversimplified or plain wrong. Anything more than using them to just pointing me to media and literature is usually a waste of time.

I find quite useful to write down my understanding/opinion and ask an LLM to find errors and write objections to it.
Post reply on HN