181–190 of 240 posts
Earlier quoted context omitted.
But if my application only uses 100 schemas, I only care about a collision if it's with one of those 100.
You have a collision if any two schemas share the id, not if a specific schema collides with any of the others. So it is exactly like the birthday paradox.
This reminded me of a tight-packed binary format we used in the trading systems domain almost 20 years ago. Instead of including metadata/field names in each message, it had a central message dictionary that every client/server would first download a copy from. Messages had only type IDs, followed by binary packed data in the correct field order. Because of microsecond latency requirements, we even avoided the serial…
Earlier quoted context omitted.
Protobuf supports sum types in the higher-level generated descriptors and languages -- on the wire they're just encoded as, well... oneof a number of possible options.
Which results in very painful inconsistencies when you’re dealing with the same schema on different platforms.
This reminded me of a tight-packed binary format we used in the trading systems domain almost 20 years ago. Instead of including metadata/field names in each message, it had a central message dictionary that every client/server would first download a copy from. Messages had only type IDs, followed by binary packed data in the correct field order. Because of microsecond latency requirements, we even avoided the serial…
I assume you were using C++? I'm not sure what you describe is possible these days due to UB. At the very least just casting bytes received over the wire to a type is UB, so you technically need a memcpy() and hope that the compiler optimises it out.
This is what JSON should have been extended to. But Douglas Crockford just don't want to innovate anything, just like Gruber didn't want to make a proper specification of the Markdown format. Sometimes people are keeping innovation back. Fortunately this did not happend with html. The main thing missing from the text format is a magic and version number. At least the binary format has it.
The dominance of JSON shows that Crockford made some good decisions, even though we may not agree with them on any given day.
Earlier quoted context omitted.
That just reminded me of the most mysterious scaling issue I ever faced. We had a message to disseminate market data for multiple markets (e.g. IBM: 100/100.12 @ NYSE, 101/102 @ NASDAQ etc.). The system performed admirably under load testing (think 50,000 messages per second). One day we onboarded a single new regional exchange and the whole market data load test collapsed. We searched high and low for days without s…
Yeah, that's part of the trick for large listing responses to be spread across frames. Usually with some indicator like a "more flag" so the client can say "get me the next sequence by requesting the next index in the listup with the prior btree index. People do this all the time with large databases and it's a very similar use case.
Previous discussions: https://news.ycombinator.com/item?id=11546098 https://news.ycombinator.com/item?id=23921610
Thanks! Macroexpanded: Amazon Ion - https://news.ycombinator.com/item?id=23921610 - July 2020 (110 comments) Amazon open-sources Ion – a binary and text interchangable, typed JSON-superset - https://news.ycombinator.com/item?id=11546098 - April 2016 (163 comments)
Earlier quoted context omitted.
I assume you were using C++? I'm not sure what you describe is possible these days due to UB. At the very least just casting bytes received over the wire to a type is UB, so you technically need a memcpy() and hope that the compiler optimises it out.
Yes, it was C++. I was unfamiliar with the acronym "UB" so did a Google search. Does it mean "Undefined Behavior"? If I remember correctly, primitive types other than strings are memcpy'd. GetStr basically returned a char* to the right place in the buffer.
Earlier quoted context omitted.
You have a collision if any two schemas share the id, not if a specific schema collides with any of the others. So it is exactly like the birthday paradox.
Yeah, but that collision probably doesn’t matter because there’s a bunch of other variables that need to come together for it to be an issue at all.
Did anything ever become of the lispy language that was being built using Ion as its homoiconic syntax? I'm afraid I can't recall what it was called. Fusion maybe?