Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Hacker News Activity Analysis with GPT-4 Agent

eu.getdot.ai

Post reply on HN
Print view View original

1–10 of 54 posts

Hacker News Activity Analysis with GPT-4 Agent

#1

Post by zurfer »

Hey, we are building Dot, a data bot (https://www.getdot.ai) that lets data teams enable everyone in their org to self-serve on governed data. We thought we'd demo it using the tried and true method of "show Hacker News stuff about itself".

For this analysis, we used the BigQuery dataset of HN (https://console.cloud.google.com/marketplace/product/y-combi...). We created one more table to pre-calculate yearly retention. And of course, a lot of the heavy lifting is done by OpenAI's GPT-4 models and the fantastic plotly library for visualization.

Let us know what other things you'd like to see about Hacker News data in the comments, and try our best to share the answers!

Hacker News Activity Analysis with GPT-4 Agent
eu.getdot.ai

Re: Hacker News Activity Analysis with GPT-4 Agent

#2

Post by atticora »

I've spent many years of my career building business reports at a pace that reminds me of digging a canal with a teaspoon, compared to this massive excavator. This isn't John Henry versus the steam drill, it's more like Bambi versus Godzilla. This kind of tool is going to revolutionize my industry, and fast. I hope I can surf the wave.

Great stuff.

Re: Hacker News Activity Analysis with GPT-4 Agent

#3

Post by zurfer »

atticora wrote: post #2

I've spent many years of my career building business reports at a pace that reminds me of digging a canal with a teaspoon, compared to this massive excavator. This isn't John Henry versus the steam drill, it's more like Bambi versus Godzilla. This kind of tool is going to revolutionize my industry, and fast. I hope I can surf the wave. Great stuff.

Thank you! Large models and analytics have a great match and will amplify how we can work with data.

Re: Hacker News Activity Analysis with GPT-4 Agent

#4

Post by atticora »

zurfer wrote: post #3
atticora wrote: post #2

I've spent many years of my career building business reports at a pace that reminds me of digging a canal with a teaspoon, compared to this massive excavator. This isn't John Henry versus the steam drill, it's more like Bambi versus Godzilla. This kind of tool is going to revolutionize my industry, and fast. I hope I can surf the wave. Great stuff.

Thank you! Large models and analytics have a great match and will amplify how we can work with data.

Can you get anything useful from a meta query like "List the top ten associations in this data by descending order of surprisingness."?

Re: Hacker News Activity Analysis with GPT-4 Agent

#5

Post by zurfer »

atticora wrote: post #4
zurfer wrote: post #3

Earlier quoted context omitted.

Thank you! Large models and analytics have a great match and will amplify how we can work with data.

Can you get anything useful from a meta query like "List the top ten associations in this data by descending order of surprisingness."?

https://eu.getdot.ai/share/f94644f5-c76a-4211-9c48-76800f12f... Not really :)

Dot will ask a clarifying question.

Questions like, "what kind of data do you have access to?" or "how can you help me?" would work.

Dot today works well for questions that can be answered with 1 SQL query and some Python.

Re: Hacker News Activity Analysis with GPT-4 Agent

#6

Post by andreshb »

Do you have samples of time-based cohort analysis? Most other solutions out there struggle to do the steps to generate time-based heatmaps and line graphs of cohort analysis. Averages, medians, and anything that can be done on a spreadsheet by a high schooler, GPT does well with.
Post reply on HN