Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Pirate Weather: A free, open, and documented forecast API

pirateweather.net

Post reply on HN
Print view View original

31–40 of 201 posts

Re: Pirate Weather: A free, open, and documented forecast API

#31

Post by jacurtis »

NoboruWataya wrote: post #3

> Why "PirateWeather"? I've always thought that the HRRR model was pronounced the same way as the classic pirate "ARRR". Also, there is one company out there that thinks APIs can be copyrighted, which might apply here. This answers the first question I had upon seeing the name, which was, is it free, open, documented and legal ? Based on the above the answer seems to be "probably". Very commendable effort, and I hope…

I agree, PirateWeather seems like a misbrand here. When I read it, i thought it was stealing weather data or something. On the web, the term "pirate" generally doesn't mean good things. This name almost implies that it is illegal or something.

I'm imagining designing a software product around this and presenting it to a C-Level, explaining that we use "PirateWeather" and I think I'm going to get grilled with lots of questions and concerns based on the name alone.

This is a good service and should be "branded" with a better name. Maybe a play on the whole DarkSky name like LightSky or "Sunset" which works exceptionally well since DarkSky was sunset by Apple. Maybe StarrySky, LateSky, NewSky.

I am usually someone who says that names don't matter as much as people think they do, but PirateWeather just seems like a huge hit in the wrong direction. But the product is solid so maybe it can survive despite the name.

Re: Pirate Weather: A free, open, and documented forecast API

#32

Post by xd1936 »

jron wrote: post #5

I was really hoping https://merrysky.net/ had a graphical forecast similar to weather underground's 10 day. I've yet to find anything else that quickly shows everything you'd want to know in a single image. Even the mouseover timeline is perfection. It is so good and seemingly exclusive to WU that I sometimes wonder if they hold a patent for it.

I agree with you, but I've also found the https://windy.com timeline along the bottom to be similarly informative, with ECMWF data to boot.

Re: Pirate Weather: A free, open, and documented forecast API

#33

Post by gen220 »

ushakov wrote: post #15

The tech diagram on AWS is insane. This is what I call vendor lock-in porn I'm wondering how much your monthly bill is?

Which part is insane?

Put another way, how would you implement this without self-managing pieces like nginx/apache, rabbitmq/kafka, or mongodb on a compute instance?

The pieces of managed infra they're depending on are all pretty interchangeable from one cloud provider to another. It's not the cheapest way to solve the problem if you discount the cost of management, upgrades, etc. But if you factor those costs in, it's quite competitive.

Re: Pirate Weather: A free, open, and documented forecast API

#34

Post by ushakov »

jacurtis wrote: post #27
ushakov wrote: post #15

The tech diagram on AWS is insane. This is what I call vendor lock-in porn I'm wondering how much your monthly bill is?

I am an Infrastructure Architect (aka "Cloud Architect") so I design cloud systems like this on the daily. The "vendor lock-in" argument always makes me laugh. Its the #1 thing I hear all day long. This diagram is actually pretty simple. It looks worse than it is. All it uses are Lambdas (serverless functions), S3 buckets (object storage), and SNS (broadcast/push queues). There appears to be one traditional server in…

Hello

> This diagram is actually pretty simple

The diagram looks like an ad

> All it uses are Lambdas (serverless functions), S3 buckets (object storage), and SNS (broadcast/push queues)

Do you actually need all of this or do you use it because Amazon tells you to? I know for instance you cannot use Amazon SES without also using S3 and Lambda

> So if the builder of this wanted to move to GCP or Azure, they are not really locked to AWS. This can all be built in another cloud

You're saying that I cannot move to other cloud provider without my existing code becoming useless?

> Assuming they are building it with Infrastructure as Code (such as Terraform) then they would need to convert the provider and change resource blocks

What about the data pipelines and business logic?

> There is almost no way to entirely remove vendor lock-in

There is: avoiding vendor-specific APIs altogether

> Closest you could come is by designing everything yourself on bare metal servers and renting those from a cloud provider

I don't have to. There are things like Railway, Fly.io, PlanetScale, Supabase, Upstash, Minio, which can work without locking me in

> What do you gain by switching?

Freedom

> There isn't significant vendor lock-in, this could be rebuilt fairly easily (assuming they used IaC) on any major cloud provider

You are contradicting yourself

Re: Pirate Weather: A free, open, and documented forecast API

#35

Post by ushakov »

gen220 wrote: post #33
ushakov wrote: post #15

The tech diagram on AWS is insane. This is what I call vendor lock-in porn I'm wondering how much your monthly bill is?

Which part is insane? Put another way, how would you implement this without self-managing pieces like nginx/apache, rabbitmq/kafka, or mongodb on a compute instance? The pieces of managed infra they're depending on are all pretty interchangeable from one cloud provider to another. It's not the cheapest way to solve the problem if you discount the cost of management, upgrades, etc. But if you factor those costs in, it…

I would go to cloud providers, which can offer me managed rabbitmq, kafka or mongodb

Certainly, I wouldn't get any vendor-specific services like Lambda or SNS (in this example)

Re: Pirate Weather: A free, open, and documented forecast API

#36

Post by manigandham »

ushakov wrote: post #34
jacurtis wrote: post #27

Earlier quoted context omitted.

I am an Infrastructure Architect (aka "Cloud Architect") so I design cloud systems like this on the daily. The "vendor lock-in" argument always makes me laugh. Its the #1 thing I hear all day long. This diagram is actually pretty simple. It looks worse than it is. All it uses are Lambdas (serverless functions), S3 buckets (object storage), and SNS (broadcast/push queues). There appears to be one traditional server in…

Hello > This diagram is actually pretty simple The diagram looks like an ad > All it uses are Lambdas (serverless functions), S3 buckets (object storage), and SNS (broadcast/push queues) Do you actually need all of this or do you use it because Amazon tells you to? I know for instance you cannot use Amazon SES without also using S3 and Lambda > So if the builder of this wanted to move to GCP or Azure, they are not re…

> "I know for instance you cannot use Amazon SES without also using S3 and Lambda"

You can absolutely use SES without S3 and Lambda. I've used it many times in various projects.

Re: Pirate Weather: A free, open, and documented forecast API

#37

Post by ushakov »

manigandham wrote: post #36
ushakov wrote: post #34

Earlier quoted context omitted.

Hello > This diagram is actually pretty simple The diagram looks like an ad > All it uses are Lambdas (serverless functions), S3 buckets (object storage), and SNS (broadcast/push queues) Do you actually need all of this or do you use it because Amazon tells you to? I know for instance you cannot use Amazon SES without also using S3 and Lambda > So if the builder of this wanted to move to GCP or Azure, they are not re…

> "I know for instance you cannot use Amazon SES without also using S3 and Lambda" You can absolutely use SES without S3 and Lambda. I've used it many times in various projects.

Not on the receiving side

https://docs.aws.amazon.com/ses/latest/dg/receiving-email-co...

Re: Pirate Weather: A free, open, and documented forecast API

#38

Post by manigandham »

ushakov wrote: post #15

The tech diagram on AWS is insane. This is what I call vendor lock-in porn I'm wondering how much your monthly bill is?

It's rather simple, just messy to look at. It's a timer that triggers a cluster that processes raw data files and uploads the JSON to a server with a reliable file system. Then a Lambda function processes API requests to read that JSON and serve a response.

"Lock-in" is just a trade-off like any other engineering decision, same as what programming language or API schema you choose. AWS is massive and reliable, and these services are cheap and widely available so I don't see much of an issue here.

Re: Pirate Weather: A free, open, and documented forecast API

#39

Post by kempbellt »

ushakov wrote: post #34
jacurtis wrote: post #27

Earlier quoted context omitted.

I am an Infrastructure Architect (aka "Cloud Architect") so I design cloud systems like this on the daily. The "vendor lock-in" argument always makes me laugh. Its the #1 thing I hear all day long. This diagram is actually pretty simple. It looks worse than it is. All it uses are Lambdas (serverless functions), S3 buckets (object storage), and SNS (broadcast/push queues). There appears to be one traditional server in…

Hello > This diagram is actually pretty simple The diagram looks like an ad > All it uses are Lambdas (serverless functions), S3 buckets (object storage), and SNS (broadcast/push queues) Do you actually need all of this or do you use it because Amazon tells you to? I know for instance you cannot use Amazon SES without also using S3 and Lambda > So if the builder of this wanted to move to GCP or Azure, they are not re…

Also a cloud engineer. I use a similar setup professionally and for various personal projects.

For someone who isn't familiar with standing up cloud resources the diagram can look overwhelming but once you play around with AWS for a bit, most of the resources you see are fairly boilerplate.

VPC is essentially just setting up a virtual LAN for your resources. S3 is being used as an API mediator to NOAA. CloudFront is a CDN for your API Gateway. Lambdas run your logic. API Gateway triages api requests to lambdas, and a couple other services act as mediators.

There is some vender lock-in in that everything here is built on AWS, but all the major cloud providers have similar/equivalent services. If you decided to move to GCP or Azure you could probably move the entire stack in a few days (maybe more depending on how much your lambdas use AWS specific internal APIs).

If vendor lock-in is a really big concern for you, you can run everything on an EC2 instance running Ubuntu instead. That way you can just spin up your service on another Ubuntu instance in another datacenter, or locally, or whatever.

Soooo, yes. There is some vendor lock-in here, but not much.

To answer your cost question. I run a very similar setup for personal projects and I rarely exceed AWS's free tier for most services. On a high-usage month it's around $85. It isn't super optimized. I could make it cheaper and nearly free if I put in the work.

That said, cost for a service like this scales very proportionally to usage. For example, AWS API Gateway can process 1 million requests for free before they start asking you for money. If the service becomes super popular we'd likely see the "Buy me a coffee" button promoted a little more and eventually you may see a paid tier as an option, but as it is, it's probably pretty affordable to run.

Re: Pirate Weather: A free, open, and documented forecast API

#40

Post by aequitas »

ushakov wrote: post #35
gen220 wrote: post #33

Earlier quoted context omitted.

Which part is insane? Put another way, how would you implement this without self-managing pieces like nginx/apache, rabbitmq/kafka, or mongodb on a compute instance? The pieces of managed infra they're depending on are all pretty interchangeable from one cloud provider to another. It's not the cheapest way to solve the problem if you discount the cost of management, upgrades, etc. But if you factor those costs in, it…

I would go to cloud providers, which can offer me managed rabbitmq, kafka or mongodb Certainly, I wouldn't get any vendor-specific services like Lambda or SNS (in this example)

The Lambda functions will just be running your business logic code. Only specific part Lambda provides is the glue. Which is easy to replace with any other cloud or self hosted alternative.
Post reply on HN