Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

I'd rather read the prompt

claytonwramsey.com

Post reply on HN
Print view View original

161–170 of 880 posts

Re: I'd rather read the prompt

#161

Post by cortesoft »

oncallthrow wrote: post #120
cortesoft wrote: post #82

Earlier quoted context omitted.

Because you want to pass on knowledge? I am not saying there aren't ANY situations where a teacher cares about what their students think, but the example given isn't really one of those times. The question is not one that has many opportunities for original thought; it is a basic question that everyone who knows the answer will answer similarly. The entire purpose is to ascertain if the person understands what was ta…

How do you know if you have passed on your knowledge without knowing what your students think/know?

Sure, but my contention was more with the word “original”, because they aren’t really original thoughts. The teacher just wants to make sure the student’s thoughts contain the information they are teaching. The teacher isn’t looking for actual original thought in this test.

Re: I'd rather read the prompt

#162

Post by musicale »

throwawaysleep wrote: post #125
jsheard wrote: post #74

I'm far from the first to make this observation but LLMs are like anti-compression algorithms when used like that, a simple idea gets expanded into a bloated mess by an LLM, then sent to someone else who runs it through another LLM to summarize it back to something approximating the original prompt. Nobody benefits aside from Sam Altman and co, who get to pocket a cool $0.000000001 for enabling this pointless exercis…

Depends on what you are looking for. I’ve turned half baked ideas into white papers for plenty of praise. I’ve used them to make my Jira tickets seem complicated and complete. I’ve used them to get praised for writing comprehensive documentation. Part of my performance review is indirectly using bloat to seem sophisticated and thorough.

> comprehensive documentation

Documentation is an interesting use case. There are various kinds of documentation (reference, tutorial, architecture, etc.) and LLMs might be useful for things like

- repetitive formatting and summarization of APIs for reference

- tutorials which repeat the same information verbosely in an additive, logical sequence (though probably a human would be better)

- sample code (though human-written would probably be better)

The tasks that I expect might work well involve repetitive reformatting, repetitive expansion, and reduction.

I think they also might be useful for systems analysis, boiling down a large code base into various kinds of summaries and diagrams to describe data flow, computational structure, signaling, etc.

Still, there is probably no substitute for a Caroline Rose[1] type tech writer who carefully thinks about each API call and uses that understanding to identify design flaws.

[1] https://folklore.org/Inside_Macintosh.html?sort=date

Re: I'd rather read the prompt

#163

Post by staunton »

sebzim4500 wrote: post #62
xmorse wrote: post #20

I am thinking about creating a proof-of-writing signature. Basically an editor with an "anti-cheat", you can't paste text into it. It signs your text with a public key.

We're going to invent kernel level anticheat for text editors rather than just do in person exams.

And what will we do next after that gets cheated?

Re: I'd rather read the prompt

#164

Post by ortusdux »

necovek wrote: post #54

I've already asked a number of colleagues at work producing insane amount of gibberish with LLMs to just pass me the prompt instead: if LLM can produce verbose text with limited input, I just need that concise input too (the rest is simply made up crap).

https://marketoonist.com/2023/03/ai-written-ai-read.html

Re: I'd rather read the prompt

#165

Post by RicoElectrico »

> Either the article is so vapid that a summary provides all of its value, in which case, it does not merit the engagement of a comment, or it demands a real reading by a real human for comprehension, in which case the summary is pointless.

There's so much bad writing of valuable information out there. The major sins being: burying the lede, no or poor sectioning, and just generally verbose.

In some cases, like in EULAs and patents that's intentional.

Re: I'd rather read the prompt

#166

Post by latentsea »

nathants wrote: post #32

the solution is obvious. stop grading the result, and start grading the process. if you can one-shot an answer to some problem, the problem is not interesting. the result is necessary, but not sufficient. how did you get there? how did you iterate? what were the twists and turns? what was the pacing? what was the vibe? no matter if with encyclopedia, google, or ai, the medium is the message. the medium is you interac…

> reviewers are scrubbing through video in an editor. evaluating the journey, not the destination.

Let's be real... Multi-modal LLMs are scrubbing through the journey :P

Re: I'd rather read the prompt

#167

Post by helloplanets »

robertlagrant wrote: post #107
matthewdgreen wrote: post #36

Earlier quoted context omitted.

It's challenging. Assignments (and particularly programming assignments) were by far the larger and more difficult part of my CS degree, and also the place where I learned the most. I cannot imagine losing that portion of my education and just replacing it with a few exams.

I wonder if you could invent a teaching language so the LLM wouldn't know about it. A little drastic, but still.

Pretty sure you could just give an LLM the given docs / course material and it would be able to write the language to a reasonable standard. Especially if it had sensible error messages.

Re: I'd rather read the prompt

#168

Post by amclennon »

ctkhn wrote: post #85

> The model produces better work. Some of my peers believe that large language models produce strictly better writing than they could produce on their own. Anecdotally, this phenomenon seems more common among English-as-a-second-language speakers. I also see it a lot with first-time programmers, for whom programming is a set of mysterious incantations to be memorized and recited. AI usage is a lot higher in my work e…

The "aha" moment for me came when I started writing a ticket for a junior engineer to work on. However, to satisfy my own curiosity, I gave the ticket to Cursor, and was able to get 90% of the way there (implementing a small feature based on a Figma design).

As it turns out, a well written ticket makes a pretty good input into an LLM. However, it has the added benefit of having my original thought process well documented, so sometimes I go through the process of writing a ticket / subtask, even if I ended up giving it to an AI tool in the end.

Re: I'd rather read the prompt

#169

Post by _qswe »

palata wrote: post #130
_qswe wrote: post #78

Earlier quoted context omitted.

I think the author has a fair take on the types of LLM output he has experience with, but may be overgeneralizing his conclusion. As shown by his example, he seems to be narrowly focusing on the use case of giving the AI some small snippet of text and asking it to stretch that into something less information-dense — like the stereotypical "write a response to this email that says X", and sending that output instead o…

> the exact inverse of how I most often use AI, which is to throw a ton of information at it in a large prompt It sounds to me that you don't make the effort to absorb the information. You cherry-pick stuff that pops in your head or that you find online, throw that into an LLM and let it convince you that it created something sound. To me it confirms what the article says: it's not worth reading what you produce this…

You're making a weirdly uncharitable assumption. I'm referring to information which I largely or entirely wrote myself, or which I otherwise have proprietary access to, not which I randomly cherry-picked from scattershot Google results.

Synthesizing large amounts of information into smaller more focused outputs is something LLMs happen to excel at. Doing the exact same work more slowly by hand just to prove a point to someone on HN isn't a productive way to deliver business value.

Re: I'd rather read the prompt

#170

Post by palata »

oncallthrow wrote: post #37

LLM cheating detection is an interesting case of the toupee fallacy. The most obvious ChatGPT cheating, like that mentioned in this article, is pretty easy to detect. However, a decent cheater will quickly discover ways to conduce their LLM into producing text that is very difficult to detect. I think if I was in the teaching profession I'd just leave, to be honest. The joy of reviewing student work will inevitably b…

> there is 0 way of telling if the work is real or not

Talk to the student, maybe?

I have been an interviewer in some startups. I was not asking leetcode questions or anything like that. My method was this: I would pretend that the interviewee is a new colleague and that I am having coffee with them for the first time. I am generally interested in my colleagues: who are they, what do they like, where do they come from? And then more specifically, what do they know that relates to my work? I want to know if that colleague is interested in a topic that I know better, so that I could help them. And I want to know if that colleague is an expert in a topic where they could help me.

I just have a natural discussion. If the candidate says "I love compilers", I find this interesting and ask questions about compilers. If the person is bullshitting me, they won't manage to maintain an interesting discussion about compilers for 15 minutes, will they?

It was a startup, and the "standard" process became some kind of cargo culting of whatever they thought the interviews at TooBigTech were like: leetcode, system design and whatnot. Multiple times, I could obviously tell in advance that even if this person was really good at passing the test, I didn't think it would be a good fit for the position (both for the company and for them). But our stupid interviews got them hired anyway and guess what? It wasn't a good match.

We underestimate how much we can learn by just having a discussion with a person and actually being interested in whatever they have to say. As opposed to asking them to answer standard questions.

Post reply on HN