can i automate the process of answering this pr questions too?
Show HN: A GitHub Action that quizzes you on a pull request
21–30 of 35 posts
Re: Show HN: A GitHub Action that quizzes you on a pull request
#22I would probably be putting devs on a pip or firing them if they failed these quizzes often...understanding your own prs is the bare fucking minimum, even without AI help.
What makes you think the AI can instead generate the correct answers to double check the developer's answers?
Re: Show HN: A GitHub Action that quizzes you on a pull request
#23Nice! A quiz to ensure you understand your vibe code.
Re: Show HN: A GitHub Action that quizzes you on a pull request
#24See, I think this is a good idea even for reviewing non-agentic human-written PRs! We've got a huge LGTM problem where people approve PRs they clearly don't understand. Recently we had a bug in some code of an employee that got laid off. The people who reviewed it are both still with the company, but neither of them could explain what the code did. That triggered this angry tweet https://x.com/donatj/status/194559338…
The only way I’ve ever seen engineers care about PR’s is if the software or product is tied directly to their paycheck. If uptime or bugs directly impact a quarterly bonus, or result in a layoff / getting fired, they spend a lot more time reviewing PR’s. Furthermore, the work and its estimate is expanded to include enough time for the team to thoroughly review the change.
Unless someone is getting fired for bad code the “lgtm” culture will never die.
Re: Show HN: A GitHub Action that quizzes you on a pull request
#25What a fun world we devs now live in.
Remember non-devs are affected just as much by this "new world". Perhaps even worse because they don't understand what's going on.
Re: Show HN: A GitHub Action that quizzes you on a pull request
#26I had an NSF grant for a similar project in 2019. Ask the dev questions about their code and validate their answers using program analysis.
The initial idea was applied to classroom settings.
An Inquisitive Code Editor for Addressing Novice Programmers’ Misconceptions of Program Behavior https://austinhenley.com/pubs/Henley2021ICSE_Inquisitive.pdf
Re: Show HN: A GitHub Action that quizzes you on a pull request
#27can i automate the process of answering this pr questions too?
That was my first reaction: now I gotta build a gpt wrapper, oops, I mean agent, to answer questions to this quiz
Re: Show HN: A GitHub Action that quizzes you on a pull request
#28This action assumes that LLMs know what they're coding.
They don't, that's why we need the PR in the first place.
Re: Show HN: A GitHub Action that quizzes you on a pull request
#29We now are making bots to quiz other bots. This is a nightmare.
Re: Show HN: A GitHub Action that quizzes you on a pull request
#30> AI Agents are starting to write more code. How do we make sure we understand what they're writing? This is a good question, but also how do we make sure that humans understand the code that _other humans_ have (supposedly) written? Effective code review is hard as it implies that the reviewer already has their own mental model about how a task could/would/should have been done, or is at the very least building thei…
Code review, to me, is not about validating the output. It's about a 2nd set of eyes to check for foot guns, best practice, etc. Code review is one step above linting and one step below unit tests, for me. If someone were to submit this code for review: getUser(id: number): UserDTO { return this.mapToDTO(this.userModel.getById(id)); } and I knew that `userModel` throws an exception when it doesn't find a user (and th…
Depends how good your QA is. Where I am it is terrible so most of the time I spend in “code review” is spent checking out the code locally and testing it myself.