Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Cap’n Proto

capnproto.org

Post reply on HN
Print view View original

91–100 of 194 posts

Re: Cap’n Proto

#91

Post by 1amzave »

kentonv wrote: post #46
teddyh wrote: post #40

Earlier quoted context omitted.

I thought the best summary of the big-endian/little-endian question was written in 1980 in Internet Experiment Note (IEN) 137: https://www.ietf.org/rfc/ien/ien137.txt Also more reader-friendly here: https://www.computer.org/csdl/mags/co/1981/10/01667115.pdf

I think figure 1 illustrates a common misunderstanding. People who object to little-endian are often imagining it in their heads as order (3): they imagine that the bits of each byte are ordered most-significant to least-significant (big-endian), but then for some reason the bytes are ordered the opposite, least-significant to most-significant (little-endian). That indeed would make no sense. But because most archite…

> When using little-endian, you imagine that the bits are in little-endian order, to be consistent with the bytes, and then everything is nice and consistent.

But isn't that kind of at odds with how shifting works? (i.e. that a left shift moves towards the "bigger" bits and a right shift moves toward the "smaller" ones.) Perhaps for a Hebrew or Arabic speaker this all works out nicely, but for those of us accustomed to progressing from left to right it seems a bit backwards...

Re: Cap’n Proto

#92

Post by kentonv »

1amzave wrote: post #91
kentonv wrote: post #46

Earlier quoted context omitted.

I think figure 1 illustrates a common misunderstanding. People who object to little-endian are often imagining it in their heads as order (3): they imagine that the bits of each byte are ordered most-significant to least-significant (big-endian), but then for some reason the bytes are ordered the opposite, least-significant to most-significant (little-endian). That indeed would make no sense. But because most archite…

> When using little-endian, you imagine that the bits are in little-endian order, to be consistent with the bytes, and then everything is nice and consistent. But isn't that kind of at odds with how shifting works? (i.e. that a left shift moves towards the "bigger" bits and a right shift moves toward the "smaller" ones.) Perhaps for a Hebrew or Arabic speaker this all works out nicely, but for those of us accustomed…

Well.... Our numbers are Arabic, after all...

Re: Cap’n Proto

#93

Post by Hydraulix989 »

StephanTLavavej wrote: post #28
kentonv wrote: post #11

Hi all, Cap'n Proto author here. Thanks for the post. Just wanted to note that although Cap'n Proto hasn't had a blog post or official release in a while, development is active as part of the Sandstorm project ( https://sandstorm.io ). Cap'n Proto -- including the RPC system -- is used extensively in Sandstorm. Sandboxed Sandstorm apps in fact do all their communications with the outside world through a single Cap'n…

As of MSVC 2015 Update 3, the support for C++11 is nearly complete. The missing pieces are that Expression SFINAE is partially supported (what's there is good enough for the STL and Boost, with certain workarounds that you must be aware of), and C99 preprocessor support is full of bugs. Aside from that, all features are present and generally usable. (Some are buggier than others; e.g. we're overhauling multithreading…

There's still a bunch of minor differences/annoyances with the MS toolchain that require me to go in and make a bunch of small changes everywhere to my project, which compiles fine on clang/g++ (and I wouldn't say I'm doing anything _crazy_ like the aforementioned "Expression SFINAE").

Right now, my project's rather small 100 kLOC codebase compiles (with very minimal #ifdef hackery) on Android, iOS, Mac, and Linux; but Windows is still very much a WIP, even some of my third-party libraries don't compile on VC. I'm actually considering trying the MINGW toolchain at this point, and I'd be curious to hear anyone's thoughts?

I'm sure part of it is that the compiler itself probably developed with the tight feedback loop from developing and maintaining large codebases in house at Microsoft (like Windows and Office). It's pretty hard when part of the internal pressure to support large codebases like that conflicts with the need to conform to outside third-party standards. I've heard great things about the compiler people at Microsoft, and I'm sure they have a technically strong team, but they are most likely caught in the middle of this organizational deadlock.

I'm sure it doesn't help that I'm not mentioning any specifics, I'm going to be revisiting the Windows desktop port of my mobile app soon (which I gave up and haven't touched for a couple months), and everything will be fresh in my mind again. I do vaguely remember something about having to explicitly add more #include statements to pull in header files that were already getting pulled in by my other compilers.

Re: Cap’n Proto

#94

Post by david-given »

Hydraulix989 wrote: post #93
StephanTLavavej wrote: post #28

Earlier quoted context omitted.

As of MSVC 2015 Update 3, the support for C++11 is nearly complete. The missing pieces are that Expression SFINAE is partially supported (what's there is good enough for the STL and Boost, with certain workarounds that you must be aware of), and C99 preprocessor support is full of bugs. Aside from that, all features are present and generally usable. (Some are buggier than others; e.g. we're overhauling multithreading…

There's still a bunch of minor differences/annoyances with the MS toolchain that require me to go in and make a bunch of small changes everywhere to my project, which compiles fine on clang/g++ (and I wouldn't say I'm doing anything _crazy_ like the aforementioned "Expression SFINAE"). Right now, my project's rather small 100 kLOC codebase compiles (with very minimal #ifdef hackery) on Android, iOS, Mac, and Linux; b…

I used to develop the Windows port of WordGrinder (my console word processor) using Mingw for Linux, testing with Wine and packaging using nsys for Linux. This actually worked pretty well and allowed a complete Windows development flow from Linux using the same rolling automation for both platforms.

The big problem was Wine, which can behave quite differently to real Windows in some places. It's certainly not bug-for-bug compatible. So I kept finding myself having to boot into Windows anyway for debugging, so eventually I just installed an msys development system on Windows with gvim and used that.

There was also something else wrong with mingw for Linux, but for the life of me I can't remember what. There was something about mingw being different to mingw32? Missing headers? It's been too long.

Re: Cap’n Proto

#95

Post by _0w8t »

kentonv wrote: post #11

Hi all, Cap'n Proto author here. Thanks for the post. Just wanted to note that although Cap'n Proto hasn't had a blog post or official release in a while, development is active as part of the Sandstorm project ( https://sandstorm.io ). Cap'n Proto -- including the RPC system -- is used extensively in Sandstorm. Sandboxed Sandstorm apps in fact do all their communications with the outside world through a single Cap'n…

I am curious, what is your take on Thrift? Is it really an improvement on protocol buffers?

Re: Cap’n Proto

#96

Post by izacus »

kentonv wrote: post #20
makmanalp wrote: post #17

Earlier quoted context omitted.

I thought that the main reason we didn't do this was because it's was hard - platform inconsistencies like 32/64 bit, endianness, not to mention differences between how languages store things, etc. The thing that irks me about these methods is that if you're using a capnproto Int rather than a regular Int, doesn't that mean that you're basically forgoing a lot of functionality that was built around and works with the…

> platform inconsistencies like 32/64 bit, In terms of data layout, 32-bit vs. 64-bit architecture only really affects pointer size. But Cap'n Proto does not encode native pointers (that obviously wouldn't work), so this turns out not to matter. > endianness, It turns out almost everything is little-endian now. Also, big-endian architectures almost always have efficient instructions for loading little-endian data. So…

> In terms of data layout, 32-bit vs. 64-bit architecture only really affects pointer size. But Cap'n Proto does not encode native pointers (that obviously wouldn't work), so this turns out not to matter.

Unless you're programming in C or C++ (or using a library from them), where size of int, long, etc. may change depending on architecture and compiler.

Re: Cap’n Proto

#97

Post by izacus »

QuercusMax wrote: post #72
jackmott wrote: post #38

Earlier quoted context omitted.

>data structures already have to sit in memory looking a certain way, why can't we just squirt that on the wire instead of some fancy bespoke type-length-value struct? In C/C++ ya can! When making games in college that is exactly what we did. Take the struct, dump it into the socket. I was rather shocked when trying to recreate the same system in C#. "I can't? I CAN'T ?"

That's an absolutely terrible idea if you want any kind of forward / backward compatibility. Just about any system is better than dumping structs over the network, once you're dealing with a system that can evolve.

Yeah, I once had to write an Android / Desktop C++ implementation of such an API that was squirting iOS datastructures over socket. The API was built by a programmer who didn't know what endianess (or size of the structs) was.

That was a special kind of hell :/

Re: Cap’n Proto

#98

Post by megak1d »

I've always liked the look of this, saw it a while back but we are still using protobuf in our .NET environment simply due to the "free" schema generation using AOP/attributes [ProtoContract]/[ProtoMember] in Marc Gravell's excellent protobuf-net (https://github.com/mgravell/protobuf-net) project - I assume this would also be possible for cap'n proto.

Re: Cap’n Proto

#99

Post by cyphar »

wtbob wrote: post #31

> The Cap’n Proto encoding is appropriate both as a data interchange format and an in-memory representation, so once your structure is built, you can simply write the bytes straight out to disk! Eh, I'd rather pay the cost of serialisation once and deserialisation once, and then access my data for as close to free as possible, rather than relying on a compiler to actually inline calls properly. > Integers use little-…

> > Integers use little-endian byte order because most CPUs are little-endian, and even big-endian CPUs usually have instructions for reading little-endian data.

> sob There are a lot of things Intel has to account for, and frankly little-endian byte order isn't the worst of them, but it's pretty rotten. Writing 'EFCDAB8967452301' for 0x0123456789ABCDEF is perverse in the extreme. Why? Why?

Little endian means that a CPU designer can make buses shorter, which makes the CPU more efficient and smaller. There are also several benefits from the programming side. So it is actually better than big endian in /some/ cases, at the cost of being less intuitive to humans.

So while Intel did choose little endian, they had very good reason to (and it's probably why everything except SystemZ and POWER use it).

Post reply on HN