191–200 of 441 posts
I'm in the exploratory stages of starting a noncommercial project focused on hosting, processing and serving a vast amount of climate data for scientists and researchers. I've "seeded" this project with my own bare metal hardware, electricity and internet access. This includes 136 vCPUs, 512GB of RAM and 128TB of hard drive storage capacity (with room for another 456TB). As my own money permits, I plan to slowly add…
Yes! (at least for me). I'm building a gamified iOS keyboard (self employed), and I have to build an auto–corrective, predictive engine for it. That was a detour for my main target, but is part of the MVP of a keyboard (as we are in 2018, and it's expected by every user). The constraints of time and memory, really pushed me in revisiting a lot of data structures (mostly tries and DAWG), there is not much information…
I had a look at the website, but don't quite understand how this will work, as there are no animations of the keyboard in action. It does sound undeniably interesting though.
You might as well have asked who among us won the lottery. I've been struggling with this for years now; after 4 boring jobs I finally gave up even the expectation that what I'll be asked to do will be even remotely interesting (or socially useful). The chance of landing an interesting job is absolutely minuscule - the market rewards whatever it rewards at the moment, and that's where jobs flock. These days, one of t…
Earlier quoted context omitted.
While I don't agree with Marx, your sense of ennui seems to reasonably represent what I understand to have been his concepts of reification and alienation. One might ask: what does the alternative look like? From what I can tell Marx is a bit wishy-washy on that point. Hayek more helpfully explained that the market provides a mechanism for determining value, not virtue . That it works, but doesn't uplift. And that tr…
"Ennui". I like the word. Thanks! > One might ask: what does the alternative look like? From what I can tell Marx is a bit wishy-washy on that point. I have no clue either. But it feels to me as if we're turning humanity into a single superorganism, with each one of us being just a cell, except all those cells have brains capable of wishing they'd play on the same level as the superorganism. Related: I've never had a…
I'm in the exploratory stages of starting a noncommercial project focused on hosting, processing and serving a vast amount of climate data for scientists and researchers. I've "seeded" this project with my own bare metal hardware, electricity and internet access. This includes 136 vCPUs, 512GB of RAM and 128TB of hard drive storage capacity (with room for another 456TB). As my own money permits, I plan to slowly add…
Wow :) High-level questions: - At the risk of birthing a page-long subthread on ZFS-vs-everything-else... what storage solution are you using, and why? - What sort of hardware are you using? (This is a non-catalyzing question, and is just out of curiosity) - How did this get started, and how are you managing this? - Will you ever be interested in accepting donations or funding? (Including on a voluntary basis; and in…
> - At the risk of birthing a page-long subthread on ZFS-vs-everything-else... what storage solution are you using, and why?
Right now, XFS and HDFS. This is primarily due to my prior experience with it; I have considered ZFS, and might move to that later on. I may also dispense with HDFS for shared storage and to reduce the redundant JVM memory footprint. I'm attracted to ZFS for compression and error correction features, I just don't have any actual experience using it.
> - What sort of hardware are you using? (This is a non-catalyzing question, and is just out of curiosity)
Four bare metal servers, three of which have dual Xeon E5-2630 V4s (10 core, 20 thread 2.2 Ghz), one of which has an i7-6900K (8 core, 16 thread 3.2 Ghz). The latter server has 10TB SATA SSD capacity and four GTX 1080 GPUs. The other three have 2TB of NVMe SSD capacity. Each of the four has 128GB DDR4 2400 Mhz RAM. Currently all storage is local to the hardware, but there is a SuperMicro SC847 for future expansion. Every server has dual 10Gb/s SFP+ connections in bonded 802.3ad LACP, and are connected via a 16 port 10G network switch in the same rack. I think that's everything at a high level, off the top of my head.
> - How did this get started, and how are you managing this?
I joined John Baez and a few other scientists/researchers working on the Azimuth Climate Project, which sought to preserve critical snapshots of climate data in the event of mass defunding. Later on I decided to take this a step further since those snapshots were becoming very out of date and the multifarious data repositories were never very well documented, organized or normalized. Not sure what you mean by "managing this" though.
> - Will you ever be interested in accepting donations or funding? (Including on a voluntary basis; and including with clear stipulations/structure about management)
I initiated the process of starting a formal nonprofit, but not for the purpose of soliciting donations; rather, just so that it would be clear it's a noncommercial activity. I might be interested in that kind of thing - in either way you mentioned - once I have exhausted my own reasonable resources for the task and have to significantly expand.
> - What sorts of decisions/motivated this initiative?
It began with reading worrydream's blog post, "What can a technologist do about climate change?": http://worrydream.com/ClimateChange/. Not much more to it than that.
> - Do you have any interest in creating this as a "community hub", with a central code repo that data scientists can push updates to that then get run on the cluster? If the visualization data (or prerendered bits of it) are openly cached/accessible, having the code that generated the data equivalently open/available could be interesting.
Yes, that's an interesting idea.
> - What kind of availability/openness are you looking at with data? (TL;DR translation: rate limiting)
Well everything is intended to be extremely transparent, so I'm going to open source all software (infrastructure, development, research, etc). I'm also going to keep all data open. In practice there will probably be a limit of 1000 or so queries per day per IP address, but I'll burn that bridge when I get to it. It really depends on how "real time" queries end up being, and how much abuse the system actually receives.
You might as well have asked who among us won the lottery. I've been struggling with this for years now; after 4 boring jobs I finally gave up even the expectation that what I'll be asked to do will be even remotely interesting (or socially useful). The chance of landing an interesting job is absolutely minuscule - the market rewards whatever it rewards at the moment, and that's where jobs flock. These days, one of t…
Earlier quoted context omitted.
Is this view that common? I'm all for work-life balance and working on passion projects on the side. But do that many people really think of their developer job as a salary and a daily grind?
A job doesn't have to be a calling, and a job that isn't a calling doesn't have to be a soul-crushing grind. Most developer jobs are for CRUD apps, and you'll rarely be doing greenfield development. There's not a whole lot of room for passion there. If you just treat it as a job, though, rather than something you have to force yourself to be passionate about, it doesn't need to be that interesting. Once you make a li…
https://octavian.ai if you're curious.
Earlier quoted context omitted.
There should be a name to the phenomenon that humans seem to become crushingly depressed once they start to grasp the big picture dynamics of the system they're encapsulated within. I think it's an effect that we're not quite prepared for as we reach closer and closer to science dotting all its i's. It's as if childhood primes us to think that life is all about figuring out what the heck is going on around us, and as…
I agree. Maybe there is a word for that? Personally, understanding the "big picture dynamics" of systems is kind of an intellectual porn for me. I loved doing that for as long as I remembered. It always came with the sense of control - understanding the systems means you could tweak it to your liking, if you so desired. Finally realizing that the system in which we live is basically uncontrollable by an individual wi…