Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Show HN: S3mini – Tiny and fast S3-compatible client, no-deps, edge-ready

github.com

Post reply on HN
Print view View original

41–50 of 105 posts

Re: Show HN: S3mini – Tiny and fast S3-compatible client, no-deps, edge-ready

#41

Post by dboreham »

supriyo-biswas wrote: post #14
tom1337 wrote: post #11

Earlier quoted context omitted.

checksumming does make sense because it ensures that the file you've transferred is complete and what was expected. if the checksum of the file you've downloaded differs from the server gave you, you should not process the file further and throw an error (worst case would probably be a man in the middle attack, not so worse cases being packet loss i guess)

> checksumming does make sense because it ensures that the file you've transferred is complete and what was expected. TCP has a checksum for packet loss, and TLS protects against MITM. I've always found this aspect of S3's design questionable. Sending both a content-md5 AND a x-amz-content-sha256 header and taking up gobs of compute in the process, sheesh... It's also part of the reason why running minio in its singl…

Well known (apparently not?) that applications can't rely on TCP checksums.

Re: Show HN: S3mini – Tiny and fast S3-compatible client, no-deps, edge-ready

#42

Post by greener_grass »

pier25 wrote: post #39
yard2010 wrote: post #21

Tangibly related: Bun has a built-in S3-compatible client. Bun is a gift, if you're using npm consider making the switch.

Proividing built APIs to not rely on NPM is one of the most interesting aspects of Bun IMO.

Can someone explain the advantage of this?

If I want S3 access, I can just use NPM

If I don't want S3 access, I don't want it integrated into my runtime

Re: Show HN: S3mini – Tiny and fast S3-compatible client, no-deps, edge-ready

#43

Post by zackify »

yard2010 wrote: post #21

Tangibly related: Bun has a built-in S3-compatible client. Bun is a gift, if you're using npm consider making the switch.

I came here to say the same thing.

Rather ship oven/bun through docker and have a 90mb container vs using node.

Re: Show HN: S3mini – Tiny and fast S3-compatible client, no-deps, edge-ready

#44

Post by zackify »

eknkc wrote: post #38
biorach wrote: post #33

Earlier quoted context omitted.

What were the most significant missing bits?

The worst thing is issues without any visibility. The other day I was toying with the MCP server ( https://github.com/modelcontextprotocol/typescript-sdk ). I default to bun these days and the http based server simply did not register in claude or any other client. No error logs, nothing. After fiddling with my code I simply tried node and it just worked.

It definitely works in bun just fine. I have a production built mcp server with auth running under bun.

Now if you convert the request / response types to native bun server, it can be finicky.

But it works fine using express under bun with the official protocol implementation for typescript.

Actually writing a book about this too and will be using bun for it https://leanpub.com/creatingmcpserverswithoauth

Re: Show HN: S3mini – Tiny and fast S3-compatible client, no-deps, edge-ready

#45

Post by alwyn »

supriyo-biswas wrote: post #14
tom1337 wrote: post #11

Earlier quoted context omitted.

checksumming does make sense because it ensures that the file you've transferred is complete and what was expected. if the checksum of the file you've downloaded differs from the server gave you, you should not process the file further and throw an error (worst case would probably be a man in the middle attack, not so worse cases being packet loss i guess)

> checksumming does make sense because it ensures that the file you've transferred is complete and what was expected. TCP has a checksum for packet loss, and TLS protects against MITM. I've always found this aspect of S3's design questionable. Sending both a content-md5 AND a x-amz-content-sha256 header and taking up gobs of compute in the process, sheesh... It's also part of the reason why running minio in its singl…

In my view one reason is to ensure integrity down the line. You want the checksum of a file to still be the same when you download it maybe years later. If it isn't, you get warned about it. Without the checksum, how will you know for sure? Keep your own database of checksums? :)

Re: Show HN: S3mini – Tiny and fast S3-compatible client, no-deps, edge-ready

#46

Post by ecshafer »

akouri wrote: post #29

This is awesome! Been waiting for something like this to replace the bloated SDK Amazon provides. Important question— is there a pathway to getting signed URLs?

You can just use s3 vis rest calls if you dont like their sdk.

Re: Show HN: S3mini – Tiny and fast S3-compatible client, no-deps, edge-ready

#49

Post by supriyo-biswas »

alwyn wrote: post #45
supriyo-biswas wrote: post #14

Earlier quoted context omitted.

> checksumming does make sense because it ensures that the file you've transferred is complete and what was expected. TCP has a checksum for packet loss, and TLS protects against MITM. I've always found this aspect of S3's design questionable. Sending both a content-md5 AND a x-amz-content-sha256 header and taking up gobs of compute in the process, sheesh... It's also part of the reason why running minio in its singl…

In my view one reason is to ensure integrity down the line. You want the checksum of a file to still be the same when you download it maybe years later. If it isn't, you get warned about it. Without the checksum, how will you know for sure? Keep your own database of checksums? :)

If we're talking about bitrot protection, I'm pretty sure S3 would use some form of checksum (such as crc32 or xxhash) on each internal block to facilitate the Reed-Solomon process.

If it's verifying whether if it's the same file, you can use the Etag header which is computed server side by S3. Although I don't like this design as it ossifies the checksum algorithm.

Re: Show HN: S3mini – Tiny and fast S3-compatible client, no-deps, edge-ready

#50

Post by nikeee »

akouri wrote: post #29

This is awesome! Been waiting for something like this to replace the bloated SDK Amazon provides. Important question— is there a pathway to getting signed URLs?

I've built an S3 client with similar goals like TFA, but supports pre-signing:

https://github.com/nikeee/lean-s3

Pre-signing is about 30 times faster than the AWS SDK and is not async.

You can read about why it looks like it does here: https://github.com/nikeee/lean-s3/blob/main/DESIGN_DECISIONS...

Post reply on HN