Live data from Hacker News

Postgres sequences can skip 32 unexpectedly

incident.io

21–30 of 81 posts

Re: Postgres sequences can skip 32 unexpectedly

#21
post #15

The initial design was quite flawed, in addition to not using sequences they should not use one DB object per organization, but rather a single object with an "organization" field.

That's a little unfair, I don't think there's enough information in the article to come to a conclusion either way.

Re: Postgres sequences can skip 32 unexpectedly

#22
post #12

Earlier quoted context omitted.

Most places do not care about this. e.g. our local Subway restaurants print purchase number of the day to every receipt that customer gets. So you can see how many sales certain Subway restaurant has done in a single day by going to that restaurant before it closes and buying something.

If you can find a way to take a representative sample of their stores, you can front-run their quarterly earnings report and make a guaranteed profit off them.

That is absolutely true, but an individual store is unlikely to care.

Re: Postgres sequences can skip 32 unexpectedly

#23
post #8

Earlier quoted context omitted.

Do you mean enumeration, whereby an attacker starts at some ID and tries several in sequence? It has never been a problem for me and my applications. Just because you know a record exists, doesn't mean you can see it. For example, if you are authorized to view https://www.example.com/records/100 , and you decide to try https://www.example.com/records/101 , then the code will check to see if you're authorized to see r…

> I suppose there are situations where it's a problem if someone finds out that record 101 even exists, but not in any of my apps. Well, it's things like say, an invoice number. I can buy something from you. And then 7 days later I buy something else from you. If the invoice numbers are in sequence, I just gained quite a bit of information about how fast you are selling things. That's the kind of information leak tha…

Yes; it’s often called the German tank problem:

https://en.wikipedia.org/wiki/German_tank_problem

Re: Postgres sequences can skip 32 unexpectedly

#24
post #9

Earlier quoted context omitted.

Obscurity isn’t a substitute for security. You still need to have proper authentication and authorization in place.

It can still leak information, like if a companies customers have sequential id's and your id is 590 then it is reasonable to assume that the company has had around 500-600 customers. It is usually not something to worry about, but in some cases you want to avoid leaking that info.

Or Jira tickets. When you submit two support cases in a few days and the number increased by 20 while the company is still a startup, you are pretty sure to have no decent answer.

Re: Postgres sequences can skip 32 unexpectedly

#27

Thank you for uncovering this edge case. To summarize, Postgres reserves a batch of 32 serial numbers from its sequence objects. Then, in the case of a crash, or the promotion of a "follower", that batch is lost. I consider ID numbers somewhat opaque, like GUIDs but maybe not that opaque. Fretting about gaps in ID numbers can cause hair loss. This is just one of many ways gaps can happen. It is just an artifact of "s…

At this point, you can just add a key to the column, set it to max(column) + 1 in an update, and forget about the sequence.

Either way, it's better done in a column that isn't the primary key.

Re: Postgres sequences can skip 32 unexpectedly

#28
post #23
post #8

Earlier quoted context omitted.

> I suppose there are situations where it's a problem if someone finds out that record 101 even exists, but not in any of my apps. Well, it's things like say, an invoice number. I can buy something from you. And then 7 days later I buy something else from you. If the invoice numbers are in sequence, I just gained quite a bit of information about how fast you are selling things. That's the kind of information leak tha…

Yes; it’s often called the German tank problem : https://en.wikipedia.org/wiki/German_tank_problem

Thanks for sharing this. I'm not saying this will definitely come up in my day job, but it's at least possible, and now I'll know what to call it. :)

Re: Postgres sequences can skip 32 unexpectedly

#29
post #16

Earlier quoted context omitted.

You can't stop anyone from doing what he describes if you allow customers to see the invoice number. IMO, your sequential sequence number should be for tax audit purposes only.

That may be your opinion, but an invoice in at least the Netherlands needs an monotonically increasing number. And it needs to be on the invoice and your customer needs to be able to see it. [edit] It's even EU-wide, see article 226(2) of directive 2006/112/EC: > a sequential number, based on one or more series, which uniquely identifies the invoice;

Nothing stops you from using the "...or more series" part to generate numbers that are specific to the day, the hour or, if you feel like it, the minute.

Today's first invoice could be 2021-07-16-001, the second one 2021-07-16-002, etc.

If you really don't want people to be able to guess your invoice volume from numbers alone, there are various ways to do that while still being compliant to EU laws.

Post reply on HN