Live data from Hacker News

Tell HN: ChatGPT is fantastic for finding and solving issues in logs

news.ycombinator.com

71–80 of 184 posts

Re: Tell HN: ChatGPT is fantastic for finding and solving issues in logs

#71
post #67

Earlier quoted context omitted.

I love that it’s replacing sed, awk, and cut in the same way you’d bring artillery to a knife fight.

But sed, awk etc.results are predictable and reproducible. That's not guaranteed with ChatGPT.

You could try asking it to output the sed/awx/etc. commands needed to do the desired transformation reproducibly. If it's not yet good at that it will be soon.

Re: Tell HN: ChatGPT is fantastic for finding and solving issues in logs

#72
post #14

Earlier quoted context omitted.

lol, indeed. As long as the hallucination problem remains, I think we are going to see a significant hype bubble crash within a year or so. Yes, it is still useful under proper guidance, but building things on full automation that are reliable doesn't seem to be something that is actually within realms of reality at present. More innovations will be required for that.

My understanding is the base model is pretty good about knowing whether it knows stuff or not. it's human feedback training that causes it to lose that signal.

Do you have any references? I know of the emergent deception problem that seems to be created through feedback.

https://bounded-regret.ghost.io/emergent-deception-optimizat...

Re: Tell HN: ChatGPT is fantastic for finding and solving issues in logs

#73
post #45
post #37

Earlier quoted context omitted.

In addition, for recipes, it’s just a better Google. If I do “Give me a concise recipe for X” it gives me one. No fluff, no ads. Just ingredients and steps. For example. I asked for pasta carbonara, concise and then even more concise. Final result: Quick Carbonara (4 servings) Ingredients: 12 oz pasta 4 eggs 1 cup cheese 8 oz bacon 4 garlic Salt, pepper Parsley (opt) Cook pasta, save water. Mix eggs, cheese. Fry baco…

Parsley in carbonara?

Garnish.

Re: Tell HN: ChatGPT is fantastic for finding and solving issues in logs

#74
post #33

I used it to transform cryptic credit card statement items into company names, which then allowed me to query my Gmail archives for receipts and invoices from these vendors, automating a manual process of accounting backup discovery that is the bane of my very existence. I even got GPT-4 to assess whether an email likely relates to an invoice or payment so that I could limit the amount of noise extracted from my emai…

You should turn this into a product (or open source it). I'd imagine this is a pain point for many, myself included.

Re: Tell HN: ChatGPT is fantastic for finding and solving issues in logs

#75

How does that work with logs? Logs are often... Huge? How many lines of logs can you paste? Because if I first need to narrow down the log to the problematic part, I kinda already have my problem right there no? Or do you mean I do something like grab the lines with "error" in the log, hoping there aren't too many, then ask ChatGPT what it thinks about this: [ 0.135036] kernel: ACPI Error: AE_NOT_FOUND, During name l…

That log line (with the four space at the front for HN formatting) is 40 tokens [1]. You can easily fit several hundred log lines with GPT4 8k context and with the incoming 32K context, you'll be able to fit close to a thousand log lines.

That's a lot of context, especially if you can prefilter from relevant services, nodes, etc. or provide a multi-node trace

[1] https://platform.openai.com/tokenizer

Re: Tell HN: ChatGPT is fantastic for finding and solving issues in logs

#76
post #36

Earlier quoted context omitted.

> Making meal plans: what is being done here? making a list of meals to eat each day of the week? isn't this just a question of thinking what one would like to eat? why is it easier to have the meals chosen by someone else? Do you have children? Meal planning can be a quite tedious and frustrating task if you want to cook at home, eat healthy, eat tasty, vary the dishes and have meals that kids will accept.

I do have children. The thing is that I know what my children like to eat and don't like to eat, but ChatGPT does not. If I had to direct, say, a human servant who is very good at cooking, but who doesn't know my kids, to plan meals for my family, I would suggest 4-6 meals that we eat frequently, 7-10 that we eat a bit less frequently, and then maybe mention a couple of things that my kids don't like. And specific di…

We tried it to get meal ideas and help manage a new diet with restrictions on calories/fat/protein.

Measuring each across meals and snacks each day and verifying each ingredient is time-consuming.

Re: Tell HN: ChatGPT is fantastic for finding and solving issues in logs

#77
post #45
post #37

Earlier quoted context omitted.

In addition, for recipes, it’s just a better Google. If I do “Give me a concise recipe for X” it gives me one. No fluff, no ads. Just ingredients and steps. For example. I asked for pasta carbonara, concise and then even more concise. Final result: Quick Carbonara (4 servings) Ingredients: 12 oz pasta 4 eggs 1 cup cheese 8 oz bacon 4 garlic Salt, pepper Parsley (opt) Cook pasta, save water. Mix eggs, cheese. Fry baco…

Parsley in carbonara?

I wouldn't put it in carbonara, but Marcella Hazan's recipe[1] includes it and she's about as big of a "pasta sauce authority figure" as you're likely to find.

[1] https://www.latimes.com/recipe/marcella-hazans-spaghetti-car...

Re: Tell HN: ChatGPT is fantastic for finding and solving issues in logs

#78
post #70

Earlier quoted context omitted.

I keep seeing anecdotes like this, and I wonder: How do you feel about the privacy aspect of this? To do this, you had to feed your email into GPT-4, right?

I think there's a widely held misconception that anything you paste into GPT-4 will be used as raw training data by the model. Some people even seem to believe that it's learning continuously, so something you paste in could show up in an answer for another user a few minutes later. My mental model of how this works is somewhat different: - It takes months to train a model on raw data, and OpenAI train new ones (that…

Yes re treating interactions as RLHF. Could imagine them developing a flow to automatically catalog interactions as successful and unsuccessful, then cluster those by domain + interaction flow. If someone has a successful interaction in a cluster that is normally unsuccessful, treat that as a 'wild-type' prompt engineering innovation that needs to be domesticated into the model.

I think you're right that blindly training on chats would bring back the olden days of google bombing ('santorum')

And also that any company with 'improve' in their TOS isn't committing to perfect privacy

Re: Tell HN: ChatGPT is fantastic for finding and solving issues in logs

#79

How does that work with logs? Logs are often... Huge? How many lines of logs can you paste? Because if I first need to narrow down the log to the problematic part, I kinda already have my problem right there no? Or do you mean I do something like grab the lines with "error" in the log, hoping there aren't too many, then ask ChatGPT what it thinks about this: [ 0.135036] kernel: ACPI Error: AE_NOT_FOUND, During name l…

That log line ( with the four space at the front for HN formatting) is 40 tokens [1]. You can easily fit several hundred log lines with GPT4 8k context and with the incoming 32K context, you'll be able to fit close to a thousand log lines. That's a lot of context, especially if you can prefilter from relevant services, nodes, etc. or provide a multi-node trace [1] https://platform.openai.com/tokenizer

Yeah but… i can look through 100 log lines manually faster than writing a good gpt prompt. This would get useful if I can easily paste like 1M lines of logs (a few minutes of data for us), but even if that would work, it’d be prohibitively expensive I think.

In other words, I still don’t completely grok the use case that’s being shared here.

Re: Tell HN: ChatGPT is fantastic for finding and solving issues in logs

#80

Earlier quoted context omitted.

That log line ( with the four space at the front for HN formatting) is 40 tokens [1]. You can easily fit several hundred log lines with GPT4 8k context and with the incoming 32K context, you'll be able to fit close to a thousand log lines. That's a lot of context, especially if you can prefilter from relevant services, nodes, etc. or provide a multi-node trace [1] https://platform.openai.com/tokenizer

Yeah but… i can look through 100 log lines manually faster than writing a good gpt prompt. This would get useful if I can easily paste like 1M lines of logs (a few minutes of data for us), but even if that would work, it’d be prohibitively expensive I think. In other words, I still don’t completely grok the use case that’s being shared here.

The use case here is looking through logs for software that aren't familiar, especially stuff that gets touched too infrequently to internalize like, say, driver logs on a Linux workstation.

If it's faster for you to read the logs yourself, you should continue to do that. If it's bespoke personal or commercial software, chances are GPT isn't going to be trained on its meaning anyway.

Most people aren't going to be familiar with arbitrary ACPI errors. Most people would have to Google or ask GPT to even understand the acronym.

Post reply on HN