Live data from Hacker News

Pirate Weather: A free, open, and documented forecast API

pirateweather.net

51–60 of 201 posts

Re: Pirate Weather: A free, open, and documented forecast API

#51
post #26

Earlier quoted context omitted.

If you are using the current version of iOS, the new weather app rocks in terms of functionality (IMHO better than the previous version of Dark Sky). If you are on an old version of iOS and mourn the loss of DarkSky app, I personally like MyRadar - it has some very nice features and I used in tandem with DarkSky in the past. If you are after writing an iOS app, you already have an Apple developer account allowing for…

The new iOS Weather app certainly has lots of functionality, lots of data, but I mourn the clean, streamlined Dark Sky interface. Plus the temperature map, unlike the radar map, no longer allows you to change to a future date/time to see how the temperature changes are going to progress across a larger region, which I always found interesting. Very unfortunate.

Also the live map just looks weird and interpolated now.

Re: Pirate Weather: A free, open, and documented forecast API

#52

How does this differ from NOAA weather.gov api? https://www.weather.gov/documentation/services-web-api Or is this a friendlier overlay of their interface?

Good question! It's the same underlying models, but three key differences: 1. Pirate Weather returns data following the Dark Sky syntax, as opposed to the custom NWS format. 2. Pirate Weather has global coverage, the NWS API is only for the US. 3. The NWS one uses human forecasters to come up with their forecasts, compared to Pirate Weather's use of raw model results.

Re: Pirate Weather: A free, open, and documented forecast API

#53

> Why "PirateWeather"? I've always thought that the HRRR model was pronounced the same way as the classic pirate "ARRR". Also, there is one company out there that thinks APIs can be copyrighted, which might apply here. This answers the first question I had upon seeing the name, which was, is it free, open, documented and legal ? Based on the above the answer seems to be "probably". Very commendable effort, and I hope…

I agree, PirateWeather seems like a misbrand here. When I read it, i thought it was stealing weather data or something. On the web, the term "pirate" generally doesn't mean good things. This name almost implies that it is illegal or something. I'm imagining designing a software product around this and presenting it to a C-Level, explaining that we use "PirateWeather" and I think I'm going to get grilled with lots of…

I'm the dev for this, so can shed some light on this! Weather Underground was a pretty out there choice- I guess the kind of people who like putting together weather APIs aren't great at naming things. I considered "Bright Ground", and still have the domain name for it, so maybe it'd be worth spinning up another API endpoint/ branding that uses that and has a more commercial focus, keeping Pirate Weather as the free and open source branding.

Re: Pirate Weather: A free, open, and documented forecast API

#56
post #15

The tech diagram on AWS is insane. This is what I call vendor lock-in porn I'm wondering how much your monthly bill is?

This is like the 5th iteration of that darn diagram, and I still can't quite get it right. The other comments are correct, the underlying setup isn't that complicated, and it wouldn't be too tricky to build it for another cloud, since all that's really happening is a Python script reading a NetCDF file. The real perk to building everything using a server-less approach was that everything scaled down to 1 user (me), and then has scaled up beautifully as more people have signed up.

The monthly bill isn't great, but has been manageable through donations so far. I'm handling about 10 million API calls/ month, so lots of Lambda invocations, but they're all thankfully very short with tiny outbound data transfer totals, and luckily inbound is free. If the costs ever get out of hand is to throttle the request rate down, but there's still a few optimizations I can do on the AWS side that should help (ARM here I come!).

Re: Pirate Weather: A free, open, and documented forecast API

#57

> Why "PirateWeather"? I've always thought that the HRRR model was pronounced the same way as the classic pirate "ARRR". Also, there is one company out there that thinks APIs can be copyrighted, which might apply here. This answers the first question I had upon seeing the name, which was, is it free, open, documented and legal ? Based on the above the answer seems to be "probably". Very commendable effort, and I hope…

I'm the dev behind this, so always great to hear things like this! I really struggled to try to come up with a name. My first thought was "Bright Ground" (opposite of Dark Sky), but that seemed a little too on the nose. Luckily, the legal aspect of this (I'm in the clear!) was pretty well settled after the final Oracle v. Google case, but at the time was a big enough deal that it seemed relevant, and the HRRR model w…

Lol I named my version "Bright Earth" as an opposition to dark sky (brightearth.app)

Re: Pirate Weather: A free, open, and documented forecast API

#58
post #34

Earlier quoted context omitted.

Hello > This diagram is actually pretty simple The diagram looks like an ad > All it uses are Lambdas (serverless functions), S3 buckets (object storage), and SNS (broadcast/push queues) Do you actually need all of this or do you use it because Amazon tells you to? I know for instance you cannot use Amazon SES without also using S3 and Lambda > So if the builder of this wanted to move to GCP or Azure, they are not re…

Also a cloud engineer. I use a similar setup professionally and for various personal projects. For someone who isn't familiar with standing up cloud resources the diagram can look overwhelming but once you play around with AWS for a bit, most of the resources you see are fairly boilerplate. VPC is essentially just setting up a virtual LAN for your resources. S3 is being used as an API mediator to NOAA. CloudFront is…

I'm the dev behind this, and really appreciate all the insight from actual cloud professionals! Your guess here is spot on, I designed it so that I could more or less fit in the free tier with exactly one user, with costs scaling pretty linearly afterwards. There are a few more optimizations I could do, but it's honestly pretty impressive how much traffic (I'm at about 10 million calls/ month) can be (somewhat) cheaply handled using AWS

Re: Pirate Weather: A free, open, and documented forecast API

#59

Earlier quoted context omitted.

I agree, PirateWeather seems like a misbrand here. When I read it, i thought it was stealing weather data or something. On the web, the term "pirate" generally doesn't mean good things. This name almost implies that it is illegal or something. I'm imagining designing a software product around this and presenting it to a C-Level, explaining that we use "PirateWeather" and I think I'm going to get grilled with lots of…

I'm the dev for this, so can shed some light on this! Weather Underground was a pretty out there choice- I guess the kind of people who like putting together weather APIs aren't great at naming things. I considered "Bright Ground", and still have the domain name for it, so maybe it'd be worth spinning up another API endpoint/ branding that uses that and has a more commercial focus, keeping Pirate Weather as the free…

That would actually be a great idea.

Re: Pirate Weather: A free, open, and documented forecast API

#60
post #34

Earlier quoted context omitted.

I am an Infrastructure Architect (aka "Cloud Architect") so I design cloud systems like this on the daily. The "vendor lock-in" argument always makes me laugh. Its the #1 thing I hear all day long. This diagram is actually pretty simple. It looks worse than it is. All it uses are Lambdas (serverless functions), S3 buckets (object storage), and SNS (broadcast/push queues). There appears to be one traditional server in…

Hello > This diagram is actually pretty simple The diagram looks like an ad > All it uses are Lambdas (serverless functions), S3 buckets (object storage), and SNS (broadcast/push queues) Do you actually need all of this or do you use it because Amazon tells you to? I know for instance you cannot use Amazon SES without also using S3 and Lambda > So if the builder of this wanted to move to GCP or Azure, they are not re…

Wow, I struck a nerve. I'm happy to address these points however.

> The diagram looks like an ad

Lol, Its an architecture diagram. You could swap the AWS-specific icons for generic ones I suppose, and it wouldn't change anything. It is fulfilling its purpose of explaining how all the services connect together to deliver the product. Just because it is an AWS Lambda icon doesn't mean you couldn't make it an Azure Function icon instead and perform the same goal. You're just too focused on hating AWS here to see the forest through the trees.

> Do you actually need all of this or do you use it because Amazon tells you to? I know for instance you cannot use Amazon SES without also using S3 and Lambda

This is just a standard event-driven architecture. There's really nothing exciting to see here. Data comes in, it gets stored somewhere (S3), that triggers an event (SNS), a compute service (lambda in this case, but could be anything, even a standard VM, bare metal or anything else) picks up the task and processes it or performs a job on the data and stores it, it triggers another event, something else picks it up, and so forth. This isn't an AWS design, its just an event driven architecture design and this is how they work.

SES can be used standalone. It doesn't require Lambda or S3 like you postulate. There are only a few times AWS requires something else and its usually Cloudwatch or S3 and these will sometimes be the destinations required for specific types of logging or auditing and so forth.

AWS is forcing you to do nothing here. The creator chose this stuff. But I assume they chose it to keep it free. Most of this will survive under the free tier until the project becomes massive. If it inches over the free tier, it will still be cheap. That's probably the incentive for a lot of this.

> You're saying that I cannot move to other cloud provider without my existing code becoming useless?

Correct your code is your code. Think about a lambda. In this scenario data comes from NOAA and is put in a bucket. You write a serverless function that takes that data out of the bucket, reformats the NOAA data into your proprietary format and puts it in another bucket. The code that does that is written in Go, Python, C++, Java, or whatever you want. If written correctly, it accepts data, processes it, and outputs it. So if you move to another cloud provider your code would still work. It might run in an Azure Function instead of an AWS Lambda, but that doesn't matter. Your code does the same thing you don't need to throw it out.

> What about the data pipelines and business logic?

Hard to say without more information. But its possible BI dashboards need to be changed to point to the new service and stuff like that. Sure. Again, you're not switching clouds in an afternoon. But the point its the next cloud system is eerily similar. Its not like you have to rebuild, it would be more of a refactor.

> There is: avoiding vendor-specific APIs altogether

Possibly. But if built correctly your code doesn't need to be aware of the environment it is in. But there might be cases where you interact with the services directly, like downloading from S3. This would change with the next provider possibly (although most actually have S3-compatible APIs). But most of your application will not directly interact with the cloud, it will interact with the services. So for example you use RDS to host a managed Postgres db, but your application is just interacting with postgres, not AWS here. But you're right there might be some scenarios that use vendor-specific APIs.

> I don't have to. There are things like Railway, Fly.io, PlanetScale, Supabase, Upstash, Minio, which can work without locking me in

I fail to see how these are any different than tying yourself to a product like S3 or Lambda. In many ways, these solutions are TRUE vendor lock-in, with all the vendor specific APIs that you live in fear of. Fly.io is a PaaS, which is going to be way harder to move away from than switching from AWS to Azure. PlanetScale, Minio, and Upstash are literally no different than equivalent products in AWS/GCP/Azure. I guess you could host the instance that runs these products ondifferent clouds and it would be the same, but you're still tying yourself to something. The risk of tying yourself to a startup is higher than tying yourself to Amazon/Microsoft/Google. You're trading one evil for another, in most ways you are actually losing freedom with these not gaining it.

> You are contradicting yourself

My point is that there isn't as much vendor lock-in that people fear. Yes it exists, but don't live your life in fear of it. Yes you would need to refactor stuff here and there. But the same architecture diagram we saw for AWS is basically the same one that would exist in Azure or GCP. The underlying tools don't change. The marketing names and logos change, which clearly bothers you, but the underlying system doesn't change.

Post reply on HN