Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Tell HN: ChatGPT is fantastic for finding and solving issues in logs

news.ycombinator.com

Post reply on HN
Print view View original

81–90 of 184 posts

Re: Tell HN: ChatGPT is fantastic for finding and solving issues in logs

#81

Post by busseio »

Great use case. I have successfully used it in a similar way with SRT files (recording transcripts in SubRip Subtitle format) as well as CSV data from surveys that often has many columns with long text labels (the survey questions) and free text answers.

Ex. a prompt I had used for a developer skills survey .csv was:

> The CSV data below is the results of a skills survey sent to a group of software engineers. The first row is a header row. Please summarize this data in the areas that the People are Strong, Weak, Most Similar, and Unique:

Then, because of things I saw in the response, I asked a few follow-up questions:

> How much Azure experience is there in the group?

> Can you provide more explanation around your assessment that "Engineers generally have little to no experience with Dockers and Kubernetes."

> What other skills and experience do you see in the results that you haven't already mentioned?

To address my risk tolerance vis-a-vis the ChatGPT warnings (and previous UI leak of responses), I replaced the email addresses in the .csv file with "PersonA", "PersonB", ...

Re: Tell HN: ChatGPT is fantastic for finding and solving issues in logs

#82

Post by sangnoir »

simonw wrote: post #70
q7xvh97o2pDhNrh wrote: post #62

Earlier quoted context omitted.

I keep seeing anecdotes like this, and I wonder: How do you feel about the privacy aspect of this? To do this, you had to feed your email into GPT-4, right?

I think there's a widely held misconception that anything you paste into GPT-4 will be used as raw training data by the model. Some people even seem to believe that it's learning continuously, so something you paste in could show up in an answer for another user a few minutes later. My mental model of how this works is somewhat different: - It takes months to train a model on raw data, and OpenAI train new ones (that…

I'm absolutely horrified by people's willingness to submit private information (personal or corporate) even if it's not used for training. Data breaches happen all the time (targeted or accidental), and OpenAI is becoming a juicier target by the day.

You're right that OpenAI doesn't want the information. Consequently, OpenAI will not have security policies and processes geared for anonymization, or handling financial and health data as those are not a design goals. If I were an attacker, I'd go for the raw data rather than try to glean information off the model (in the hypothetical where user input were to be used for training)

Re: Tell HN: ChatGPT is fantastic for finding and solving issues in logs

#83

Post by iso1631 »

simonw wrote: post #70
q7xvh97o2pDhNrh wrote: post #62

Earlier quoted context omitted.

I keep seeing anecdotes like this, and I wonder: How do you feel about the privacy aspect of this? To do this, you had to feed your email into GPT-4, right?

I think there's a widely held misconception that anything you paste into GPT-4 will be used as raw training data by the model. Some people even seem to believe that it's learning continuously, so something you paste in could show up in an answer for another user a few minutes later. My mental model of how this works is somewhat different: - It takes months to train a model on raw data, and OpenAI train new ones (that…

> OpenAI DO NOT WANT your private data in their training data

But they do want it. I can see many old chat logs.

Data is a liability. Does "clear conversations" in chat.openai.com actually remove them? Or jst mark them as "deleted", but they remain in a database. I just did a data export, then a clear conversation, then another data export. The second export was empty, which seems suspiciously fast to me

Re: Tell HN: ChatGPT is fantastic for finding and solving issues in logs

#84

Post by BeefWellington »

akiselev wrote: post #75
TacticalCoder wrote: post #26

How does that work with logs? Logs are often... Huge? How many lines of logs can you paste? Because if I first need to narrow down the log to the problematic part, I kinda already have my problem right there no? Or do you mean I do something like grab the lines with "error" in the log, hoping there aren't too many, then ask ChatGPT what it thinks about this: [ 0.135036] kernel: ACPI Error: AE_NOT_FOUND, During name l…

That log line ( with the four space at the front for HN formatting) is 40 tokens [1]. You can easily fit several hundred log lines with GPT4 8k context and with the incoming 32K context, you'll be able to fit close to a thousand log lines. That's a lot of context, especially if you can prefilter from relevant services, nodes, etc. or provide a multi-node trace [1] https://platform.openai.com/tokenizer

Thousands of log lines is actually pretty tiny though. I have some verbose logging in a testing lab and JUST network traffic from a few mobile devices can easily throw out several megs per hour in logs.

Re: Tell HN: ChatGPT is fantastic for finding and solving issues in logs

#85

Post by bostonsre »

femto113 wrote: post #71
croes wrote: post #67

Earlier quoted context omitted.

But sed, awk etc.results are predictable and reproducible. That's not guaranteed with ChatGPT.

You could try asking it to output the sed/awx/etc. commands needed to do the desired transformation reproducibly. If it's not yet good at that it will be soon.

It is good at that and you can only paste so much data into the UI, so usually go this route for complex one liners.

Re: Tell HN: ChatGPT is fantastic for finding and solving issues in logs

#86

Post by HybridCurve »

simonw wrote: post #70
q7xvh97o2pDhNrh wrote: post #62

Earlier quoted context omitted.

I keep seeing anecdotes like this, and I wonder: How do you feel about the privacy aspect of this? To do this, you had to feed your email into GPT-4, right?

I think there's a widely held misconception that anything you paste into GPT-4 will be used as raw training data by the model. Some people even seem to believe that it's learning continuously, so something you paste in could show up in an answer for another user a few minutes later. My mental model of how this works is somewhat different: - It takes months to train a model on raw data, and OpenAI train new ones (that…

Could you imagine if they did?

Someone might ask it: "How do you I figure out if this person killed someone?" and it responds: "I can't be certain if they killed them but last week they asked me where they should hide the body."

But seriously, I think best argument for this is that the EU(or other euro nations) would not hesitate to go after a US company for collecting user data in violation of their data privacy laws. Even in the US, certain professionals are required to maintain confidentiality of certain records or face rather extreme penalties. OpenAI also doesn't have FAANG capital to grease Washington with yet and we know how kleptocrats love to leverage justice against newly emergent companies with valuable IP.

So if they say they don't, they had better not be or it would the likely be the end of them.

Re: Tell HN: ChatGPT is fantastic for finding and solving issues in logs

#87

Post by csomar »

I used recently to navigate a quite complex Bash script. This is using Codex.

I'd go just below the line that I don't understand and type

    # Explain the line above in detail: >
And it'd write a very decent explanation that makes sense most of the time. Basically decrypting bash code, which I suck at.

However, there was one instance where it almost freaked me out as the output was quite human like:

    # Explain the line above: >.

Re: Tell HN: ChatGPT is fantastic for finding and solving issues in logs

#88

Post by csomar »

qbasic_forever wrote: post #44
ttul wrote: post #31

Earlier quoted context omitted.

It's a calculator for all human knowledge, albeit there are some rounding errors yet to be eliminated.

No, only knowledge that was on the internet and specifically on sites like Reddit that were crawled to train the model. This is definitely not all human knowledge!

Strong disagree. Will be writing a blog post how GPT-4 understands languages that don't even exist (at least not formally).

Re: Tell HN: ChatGPT is fantastic for finding and solving issues in logs

#89

Post by croes »

femto113 wrote: post #71
croes wrote: post #67

Earlier quoted context omitted.

But sed, awk etc.results are predictable and reproducible. That's not guaranteed with ChatGPT.

You could try asking it to output the sed/awx/etc. commands needed to do the desired transformation reproducibly. If it's not yet good at that it will be soon.

The question is if you can understand the solution if it's complex.

Just like the regex for valid email addresses, it may be correct but it's hard to understand it.

I see it more as a tool for doing the tedious but simple work, if it's getting complex you get a hard time checking the correctness of the result.

Re: Tell HN: ChatGPT is fantastic for finding and solving issues in logs

#90

Post by iso1631 »

skrebbel wrote: post #79
akiselev wrote: post #75

Earlier quoted context omitted.

That log line ( with the four space at the front for HN formatting) is 40 tokens [1]. You can easily fit several hundred log lines with GPT4 8k context and with the incoming 32K context, you'll be able to fit close to a thousand log lines. That's a lot of context, especially if you can prefilter from relevant services, nodes, etc. or provide a multi-node trace [1] https://platform.openai.com/tokenizer

Yeah but… i can look through 100 log lines manually faster than writing a good gpt prompt. This would get useful if I can easily paste like 1M lines of logs (a few minutes of data for us), but even if that would work, it’d be prohibitively expensive I think. In other words, I still don’t completely grok the use case that’s being shared here.

Being able to pump a firehose of syslogs at a GPT and tell it to flag any problems would be great
Post reply on HN