291–300 of 416 posts
https://dataintensive.net/ I can't recommend this book enough. I have a CS background, and still had quite a few "I can't believe this thing has been hiding in plain sight!" moments while reading it.
Are there good books for data intensive desktop apps? Like games or CAD design tools?
Makefiles. I always dismissed them as a C compiler thing. Something that could never be useful for Python programming. But nowadays every project I create has a Makefile to bind together all task involved on that project. From bootstrapping the dev environment, running checks/test, starting a devserver, building releases and container images. Makefiles are just such a nice place to put scripts for these common tasks…
Earlier quoted context omitted.
TDD with Python : https://www.obeythetestinggoat.com/
Is that useful if you’re never writing django apps?
To me, the most important chapters are
- https://www.obeythetestinggoat.com/book/chapter_mocking.html - https://www.obeythetestinggoat.com/book/chapter_purist_unit_...
Having said that, the concepts are universal.
Read the curriculum of an undergraduate computer science course and read up on the things you haven't heard of. Some courses will even have lecture notes available. E.g. these four pages are the university of Cambridge masters in computer science: https://www.cl.cam.ac.uk/teaching/1819/part1a-75.html https://www.cl.cam.ac.uk/teaching/1819/part1b-75.html https://www.cl.cam.ac.uk/teaching/1819/part2-75.html https://www…
1. Profiler. There's a standard tool that tells you what part of your code is slow. Over half the time it'll find something dumb and easy to fix instead of whatever you expected. 2. SQL / relational database schemas. Persistence opens up a lot of capabilities. And databases themselves are very well-optimized; if you do any nontrivial data manipulation it's likely that whatever the query planner comes up with will be…
I loved this book: https://users.aalto.fi/~ssarkka/pub/cup_book_online_20131111...
and also Thomas Schoen group does great work on Sequential Monte Carlo (SMC), MCMC for sequential data :) http://user.it.uu.se/~thosc112/index.html
They are also building a probabilistic programming language for sequential data! https://github.com/lawmurray/Birch
Earlier quoted context omitted.
In 2019 Makefiles are a useful tool for automating project-level things. Too often webapps will require you to install X to install Y to run producing artifact Z. Since Make is old and baked and everywhere, specifying "make Z" is a useful project-level development process. It's not tied to a language (e.g. Tox) nor a huge runtime (Docker). Make is small enough in scope to be easy, and large enough to be capable witho…
Does make files ever actually work that way? In my experience they always require you to install a bunch of packages for libraries which usually only tells you the package names for ubuntu so you have to hunt down what the package is called on your distro or if that version of the package is even in the repos.
Can you break down your problem into a bunch of rules of the form “To produce this file, I need to run these shell commands, which read those other files over there as input”? If so, Make can take care of figuring out which steps actually need to be run.
Earlier quoted context omitted.
This 1000x. I put of just getting by with shell script for some years and when I finally decided to get deeper into it, it's magical. A good series of piped commands with tools available basically everywhere can solve problems you had no idea could be so simple to solve.
have any good resources for this?
https://www.ibm.com/developerworks/library/l-sed1/index.html https://developer.ibm.com/tutorials/l-awk1/
Learning how to use dtrace / bpftrace [0] is very valuable if you ever need to get into serious systems profiling. There are some really cool data structures out there you might not know about. One of my favorite basic ones that I get a lot of use out of is the trie [1] (a.k.a. prefix tree). Very useful for IP calculations. Also look into probabilistic data structures [2], very amazing things can be done with them. […
Approximate Windows equivalent of DTrace is sysinternal process monitor, freeware. Very useful sometimes.
[0] https://techcommunity.microsoft.com/t5/Windows-Kernel-Intern...
Read the curriculum of an undergraduate computer science course and read up on the things you haven't heard of. Some courses will even have lecture notes available. E.g. these four pages are the university of Cambridge masters in computer science: https://www.cl.cam.ac.uk/teaching/1819/part1a-75.html https://www.cl.cam.ac.uk/teaching/1819/part1b-75.html https://www.cl.cam.ac.uk/teaching/1819/part2-75.html https://www…
Do Cambridge courses not have labs/projects? I looked at the course materials on a few of the courses and couldn't find any. Or are they given out to students separately?