You want to trust that thing to review your code? It will often give you eloquent explanations, trying to convince you that code has a certain bug, while completely missing the actual bug. You'll be gaslit into chasing down dead ends, distracted from actual issues.
Example: "In JS, create a Promise that is resolved with some emitted value from an EventEmitter, or rejected if an 'error' event is emitted first. Write that concisely, using '.once' and only removing the other event respectively."
Fairly everyday thing in node. Every competent programmer would grasp the intent here pretty quick and write correct code, but somehow it's generally beyond ChatGPT.
Example ChatGPT output (comments mine):
function eventToPromise(emitter, event) {
return new Promise((resolve, reject) => {
emitter.once('error', reject); // memory leak: not removing the other listener
emitter.once(event, (data) => {
emitter.removeListener('error', reject);
resolve(data);
});
});
}
Come on now... I just start the whole prompt over to let it try again:
function promiseFromEvent(emitter, eventName) {
return new Promise((resolve, reject) => {
const resolveHandler = (data) => {
emitter.removeListener(eventName, errorHandler); // memory leak: should be removeListener('error', errorHandler)
resolve(data);
};
const errorHandler = (error) => {
emitter.removeListener(eventName, resolveHandler);
reject(error);
};
emitter.once(eventName, resolveHandler);
emitter.once('error', errorHandler);
});
}
Now I prompt "You have a bug in there, fix it."
ChatGPT: "Apologies for the oversight. You are right; there is a bug in the previous code. The issue is that if the 'error' event is emitted first, both the 'error' and 'data' event listeners should be removed, and the Promise should be rejected."
Nope. The error event listener is actually the only listener working correctly... It proceeds to introduce a cleanup function that removes both listeners (despite keeping '.once'), fixing the bug by accident, not intent. Asking it to change it so that the code adheres to the original prompts starts a downward spiral.