Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ask HN: What API should I build a realtime data viz on top of this week?

news.ycombinator.com

Post reply on HN
Print view View original

1–10 of 12 posts

Ask HN: What API should I build a realtime data viz on top of this week?

#1

Post by zevaverbach »

I’m about to have a rare childless week of vacation in a boring town with decent internet access, and I’d like your advice for what to work on!

I’m a senior-ish full stack dev (Python + React) and I’m interested in getting some experience with real-time data viz using Highcharts or AG Charts.

I don’t want to get bogged down in scraping but other than that I’m open to any suggestions for the project as far as:

    - data source
    - what protocol to use
    - whether to do JAM or SSR 
    - aesthetics/styling
Please don’t tell me to “take it easy”, I already knocked that out in the first part of vacay. :-)

Re: Ask HN: What API should I build a realtime data viz on top of this week?

#4

Post by tycoon177 »

I'm not sure if you're in the US, but there are a lot of data sources at https://data.gov/ that are pretty interesting! A similar service may be available for other countries.

If you want to really get out of your comfort zone, I would suggest a new language/paradigm. I've recently been loving learning Elixir with Phoenix liveview. It fills a lot of the full-stack needs and lets you focus on your backend data and your frontend display.

Re: Ask HN: What API should I build a realtime data viz on top of this week?

#5

Post by zevaverbach »

tycoon177 wrote: post #4

I'm not sure if you're in the US, but there are a lot of data sources at https://data.gov/ that are pretty interesting! A similar service may be available for other countries. If you want to really get out of your comfort zone, I would suggest a new language/paradigm. I've recently been loving learning Elixir with Phoenix liveview. It fills a lot of the full-stack needs and lets you focus on your backend data and you…

Thanks so much for the suggestions! (Yes, I’m in the US)

Do you know offhand if any of those data sources update frequently enough for “real-time” visualizations?

Re: Ask HN: What API should I build a realtime data viz on top of this week?

#6

Post by tycoon177 »

zevaverbach wrote: post #5
tycoon177 wrote: post #4

I'm not sure if you're in the US, but there are a lot of data sources at https://data.gov/ that are pretty interesting! A similar service may be available for other countries. If you want to really get out of your comfort zone, I would suggest a new language/paradigm. I've recently been loving learning Elixir with Phoenix liveview. It fills a lot of the full-stack needs and lets you focus on your backend data and you…

Thanks so much for the suggestions! (Yes, I’m in the US) Do you know offhand if any of those data sources update frequently enough for “real-time” visualizations?

I kind of doubt it, but you could fake a realtime api by streaming results from your backend of choice as if you were getting it from an api. Some of the data is large enough that you could download it async on your backend and stream results as you parse it! I'm actually downloading the liquor sales for my state since 2012 and so far it's over 2gb of data, so definitely stream-worthy.

I did also find this just now. https://github.com/bytewax/awesome-public-real-time-datasets

Re: Ask HN: What API should I build a realtime data viz on top of this week?

#7

Post by jbernsteiniv »

The USDA has an awesome API called Food Data Central. The data is public domain in terms of licensing and the rate limit is about 1000 calls per hour per IP address (which is extremely reasonable especially for early testing).

https://fdc.nal.usda.gov/api-guide.html

Re: Ask HN: What API should I build a realtime data viz on top of this week?

#8

Post by tycoon177 »

zevaverbach wrote: post #5
tycoon177 wrote: post #4

I'm not sure if you're in the US, but there are a lot of data sources at https://data.gov/ that are pretty interesting! A similar service may be available for other countries. If you want to really get out of your comfort zone, I would suggest a new language/paradigm. I've recently been loving learning Elixir with Phoenix liveview. It fills a lot of the full-stack needs and lets you focus on your backend data and you…

Thanks so much for the suggestions! (Yes, I’m in the US) Do you know offhand if any of those data sources update frequently enough for “real-time” visualizations?

Oh, and if you do stick with react, I'd suggest this library! It's super powerful. I'm using it for some visualizations at work and it's pretty awesome what can be done. https://recharts.org/en-US/

Re: Ask HN: What API should I build a realtime data viz on top of this week?

#9

Post by zevaverbach »

tycoon177 wrote: post #8
zevaverbach wrote: post #5

Earlier quoted context omitted.

Thanks so much for the suggestions! (Yes, I’m in the US) Do you know offhand if any of those data sources update frequently enough for “real-time” visualizations?

Oh, and if you do stick with react, I'd suggest this library! It's super powerful. I'm using it for some visualizations at work and it's pretty awesome what can be done. https://recharts.org/en-US/

Thanks again, I’ll have a look at this.

Re: Ask HN: What API should I build a realtime data viz on top of this week?

#10

Post by zevaverbach »

jbernsteiniv wrote: post #7

The USDA has an awesome API called Food Data Central. The data is public domain in terms of licensing and the rate limit is about 1000 calls per hour per IP address (which is extremely reasonable especially for early testing). https://fdc.nal.usda.gov/api-guide.html

Thanks so much for the suggestion, I will have a look although I’m hoping for something which updates at least a few dozen times per minute.
Post reply on HN