Live data from Hacker News

How to find the AWS account ID of any S3 bucket

tracebit.com

211–220 of 227 posts

Re: How to find the AWS account ID of any S3 bucket

#211

There seems to be a large discussion of whether account IDs are "secret" or "private" or "confidential" or whatever. From my point of view, that entirely misses the point. The problem here is that what's revealed here is the relationship between buckets and account IDs, which allows discovery of shared ownership of buckets (unless you use a micro-account approach). I probably don't care if you can discover that 23432…

Accounts on AWS are pretty cheap (free?) - why would you host everything on the same account?

free as in: AWS does not charge you

not free as in: you have to manage it (for example give a CI role access via OIDC, create a role for you to assume to do stuff via the console, etc)

Re: How to find the AWS account ID of any S3 bucket

#212

AWS account ID == Your IP address. It may be sensitive, but someone needs to know it to get s*$t done. Illustrative example: I had to deal with a third party that we needed to integrate with because of anti-money laundering procedures a year or two ago. I wanted my team to setup a privatelink with the organization because that's generally more secure than an open sftp port. The company refused citing security reasons…

> get s*$t done

What were you hoping to achieve with this utterly pointless self-censorship?

Re: How to find the AWS account ID of any S3 bucket

#213

Earlier quoted context omitted.

Once I have an AWS account ID, my next trick is to grant cross-account bucket policies to discover role names in the account.

How does this work?

If you put a role ARN in the principal section of a bucket policy, AWS will check if the role exists and fail the policy update request if not. Even if it's not in the same account. Don't know if there's another way but you can manually enumerate roles from there

Re: How to find the AWS account ID of any S3 bucket

#214

Earlier quoted context omitted.

AWS PrivateLink has another property that generally makes it undesirable for these types of integrations: communication is bidirectional, and IP subnets should not overlap. We (as a vendor ourselves) typically integrate as a VPC Endpoint Service, where communication is unidirectional and our service is exposed as a load balancer’s endpoint within the customer’s VPC.

I thought PrivateLink was branding for vpc interface endpoints? There's no ip subject restriction for that because it's basically a proxy. Are you thinking of vpc peering? VPC endpoints seem preferable in this situation.

Oh I apologize, you’re completely correct and I am confusing PrivateLink with VPC peering. Thank you for correcting me.

Re: How to find the AWS account ID of any S3 bucket

#215
post #189
post #124

Earlier quoted context omitted.

Those wouldn't follow any particular partial pattern though would they?

They don't need to — in fact, you might be depending on them not following any particular pattern. Think: treating the Account IDs as pre-hashed keys, and then specifying prefix patterns as ways of sharding the hash keys onto a set of buckets, to evenly distribute access (and therefore traffic) by customer.

Do you ever really need to do that on AWS though? Can you really bog down S3 by having loads of requests to a single bucket?

Re: How to find the AWS account ID of any S3 bucket

#216

Earlier quoted context omitted.

Jesus, I have been in tech for 25 years and we're still throwing shit across this divide. Well we are, if nothing, consistent. The next nerd who pinches and inch of the coder/ops divide is going to make a billion.

¯\_(ツ)_/¯ Not GP, I tried to answer as friendly as possible. Their approach is even more weird given that Security goes really hand in hand with systems infrastructure In any small enough organization, security will be managed by devops/sre. If the org becomes large enough, it will start dedicating people to Security.

I didn't think you were wrong I just thought the exchange and its tone was very stereotypical of Developers and Operations (and I was amused). Each being on their side of the fence and only throwing things over the top.

It used to be really bad... "works on my machine", bad deploy instructions etc. We paved a lot of these roads with (really fucking bad) strategies like containers. At some point maybe we will learn to write software that is operationalizable. Projects like tigerbeetle give me hope that this is our next evolution.

Re: How to find the AWS account ID of any S3 bucket

#217

Earlier quoted context omitted.

Just need enough information to send to either Amazon or the FBI.

Shouldn't the bucket name / URL be enough?

The answer to these types of questions is, depends upon who gets the case. Generally, the further along you've gotten within legal limits, the better the chance that the case will be given serious consideration and time.

Re: How to find the AWS account ID of any S3 bucket

#218
post #215
post #189

Earlier quoted context omitted.

They don't need to — in fact, you might be depending on them not following any particular pattern. Think: treating the Account IDs as pre-hashed keys, and then specifying prefix patterns as ways of sharding the hash keys onto a set of buckets, to evenly distribute access (and therefore traffic) by customer.

Do you ever really need to do that on AWS though? Can you really bog down S3 by having loads of requests to a single bucket?

GET requests? Probably not. PUT/DELETE requests? I think so, yes. All updates to the bucket ultimately bottleneck at an update to the meta-version of the bucket-record-object in the object-storage-system's bucket-metadata store (itself probably something like DynamoDB / BigTable / etc.)

Given the way these IaaSs' distributed KV stores all manage writes (i.e. by having a cluster of transactor nodes that per-key write-linearization responsibility for parts of the keyspace is sharded across — such that writes are fanned out to a particular designated transactor-node given the key's hash-slot), a very large S3 user, generating an extremely high level of metadata-update concurrency against a bucket, could very likely write-contend that bucket's metadata / have a "hot" bucket-metadata key; experience low perf due to that; and solve that by sharding the bucket (swapping one too-hot metadata key for N somewhat-hot metadata keys.)

I want to give an intuition-building example here, of an IaaS feature that wouldn't exist / wouldn't be exposed to the user if not for object-storage buckets being metadata-write-contended at scale. I'm not very familiar with the AWS ecosystem, though, so I'm not sure what the good example is for AWS. What I do know is GCP, so here's a GCP example: Google Cloud Dataflow allows you to set a temporary workspace GCS bucket on a per-job basis (gcsTempLocation). And, IIRC, Google's Cloud Architects advise to not have a bunch of active Dataflow jobs sharing the same gcsTempLocation — regardless of whether they use distinct key prefixes to namespace the temp files. Given that each job would be doing a lot of little serial updates to the temp bucket — and given that Dataflow jobs can each be highly internally concurrent — you're already potentially putting out O(N^2) ~concurrent updates to that bucket. You really don't want to make it O(N^3).

Re: How to find the AWS account ID of any S3 bucket

#219
post #211

Earlier quoted context omitted.

Accounts on AWS are pretty cheap (free?) - why would you host everything on the same account?

free as in: AWS does not charge you not free as in: you have to manage it (for example give a CI role access via OIDC, create a role for you to assume to do stuff via the console, etc)

My favorite factoid about AWS accounts is that there's a global, hard rate limit for their deletion. It's actually a pain point for us.

Re: How to find the AWS account ID of any S3 bucket

#220

Earlier quoted context omitted.

> the AWS policy execution has a number of “operators” and “operands” That is correct. The IAM condition language is flexible and does not prevent you from doing strange things.

It’s a consequence of weak typing choices - not an inevitable result of allowing flexibility. Doing glob matching on account IDs is like doing concatenation with guids, applying a bitshift to a UTF8 string, or running a regex on an integer. It is a nonsensical operation, and - as shown here - results in surprising security properties of the resulting system. Surprising security properties are an undesirable result in…

They have ARN's which include the account which a glob match is useful. Something like "arn:aws:*:*:1234567890:*" is useful but "arn:aws:*:*:1234567*:*" isn't
Post reply on HN