31–40 of 47 posts
In the json data sample the site shows, what does 'tone' field means?
These scores are averaged for each spot on the map, showing the associated color.
But then, Amazon's awash with tat too these days, and I wouldn't be surprised at anyone selling their product there.
I've never heard of RapidAPI before, but it seems kind of scammy. As in, nice idea, bit just filled with low-quality spam APIs wrapping presumably free or cheaper services, not that RapidAPI itself is a scam. But then, Amazon's awash with tat too these days, and I wouldn't be surprised at anyone selling their product there.
I see it as a good way to test the waters. If people are interested in it, I'll eventually move it off of Rapid API for direct access and use Stripe for subscriptions.
As someone who has built event pipelines from GDELT data, I'd advise folks to not start (a GDELT project) with high expectations. Not only are there plenty of duplicates, but tagging quality is all over the place, especially for non-Western media.
Agreed, it's very hit or miss in some situations. I'm hoping to keep evolving this to clean it up and eek out more salient information. I think GDELT is the best source, nothing else comes close to coverage in my opinion, it's just that all the articles it gathers revolve around world events, as opposed to all of the millions of topics of discussion and niche interests people post about.
Hey there, first off cool stuff I can see this being plenty useful for various projects. We're exploring GDELT data for our own needs and I was wondering if you wouldn't mind sharing what were some rough spots or gotchas using the project?
- Missing keywords. Huge topics like "brexit" won't be found, so you need to extract those yourself.
- Dirty titles. You need to extract the article titles from the website metadata, and more often than not they'll pre/append their site name like "[title] - DailyStormer" and links like "[title]: Business, Stocks, News" etc. That requires NLP to fix.
- Non-standardized locations. Location tags are all over the place, so "united_states" might have many tags like "us", "usa", "america", "the_states", etc. I'm still working on combining these tags in the API.
Hey there, first off cool stuff I can see this being plenty useful for various projects. We're exploring GDELT data for our own needs and I was wondering if you wouldn't mind sharing what were some rough spots or gotchas using the project?
Thanks! There are a lot of issues with GDELT data, but the things that come to mind recently are: - Missing keywords. Huge topics like "brexit" won't be found, so you need to extract those yourself. - Dirty titles. You need to extract the article titles from the website metadata, and more often than not they'll pre/append their site name like "[title] - DailyStormer" and links like "[title]: Business, Stocks, News" e…
In the json data sample the site shows, what does 'tone' field means?
Tone is the author's wording of the article that conveys their feelings towards the subject. The lower the tone/sentiment score, the more negative the article. These scores are averaged for each spot on the map, showing the associated color.
I've never heard of GDELT before, but I've found news websites to have incredibly tight ToS's which prevent you from keeping a database of their articles offline, regardless of whether or not you distribute the articles.
Earlier quoted context omitted.
Tone is the author's wording of the article that conveys their feelings towards the subject. The lower the tone/sentiment score, the more negative the article. These scores are averaged for each spot on the map, showing the associated color.
this is cool @prohobo! may I ask how you classify tone? I have a completely unrelated project (enterprise app with thousands of documents) on which I'd like to apply to. TIA.
Curious, are you getting _all_ your data from GDELT, or are you also scraping from news sites. I've never heard of GDELT before, but I've found news websites to have incredibly tight ToS's which prevent you from keeping a database of their articles offline, regardless of whether or not you distribute the articles.