31–40 of 108 posts
The linked slides "Python in HPC" [0] are quite awesome, from which I learned about mpi4py's [1] existance. Definitely going to give it a try in the future; using the MPI API with an "easy-to-follow" syntax should be fun. [0] https://hpc.nih.gov/training/handouts/171121_python_in_hpc.p... [1] http://mpi4py.scipy.org/docs/
Don't overlook all of the % commands, such as %edit. If you are familiar with emacs keybindings, it has a very good built in editor as well. You can also load snippets from saved files, and save your history to a file. Or individual lines to files using range type expressions. In short it is very easy to get code in and out of IPython. Another great trick. Anywhere you want to debug or play in your scripts run `impor…
I'm not sure I would compare it favorably to emacs in pretty much anything other than mindshare. And, don't get me wrong, that is huge and not intended as a vanity comment. The keyboard shortcuts it has are superficial. My number one shortcuts in emacs are compile, jump to next error, grep/occur, index, and magic. Just up and down? Obviously u use them a lot, but the arrow keys do work fine. Beginning of line and beg…
The one other super handy shortcut is CTRL-R for shell like reverse search. That is pretty sweet.
Earlier quoted context omitted.
It is worth noting that there is an argument that it is a worthwhile task for students to learn how to setup complex computing environments, as it better prepares them for the real world. However, in reality, there just isn't time within a single semester to do this for a class of 100+ students. So implementations such as this one trade-off that learning for a greater focus on computational theory and its implementat…
Agreed. At the beginning of class, walk students through the setup. Then for every project after, let them use the pre-rolled systems.
At Harvard we've built out an infrastructure to allow us to deploy JupyterHub to courses with authentication managed by Canvas. It has allowed us to easily deploy complex set-ups to students so they can do really cool stuff without having to spend hours walking them through setup. Instructors are writing their lectures as IPython notebooks, and distributing them to students, who then work through them in their Jupyte…
I'm torn. On the one hand that's really cool to get everything configured and up and running so students can get to the interesting parts. On the other hand, learning how to configure your own environment is kind of an essential part of working with any tool that forces you to understand at least some of the structure involved.
At Harvard we've built out an infrastructure to allow us to deploy JupyterHub to courses with authentication managed by Canvas. It has allowed us to easily deploy complex set-ups to students so they can do really cool stuff without having to spend hours walking them through setup. Instructors are writing their lectures as IPython notebooks, and distributing them to students, who then work through them in their Jupyte…
I'm torn. On the one hand that's really cool to get everything configured and up and running so students can get to the interesting parts. On the other hand, learning how to configure your own environment is kind of an essential part of working with any tool that forces you to understand at least some of the structure involved.
We built an online service on top of Jupyter which takes away the effort of handling JupyterHub. It would be great if we could hear some feedback in the context of this conversation. We feel that DataCabinet is better in ways because it provides: a. Autoscaling according to number of users b. Sharing full containers easily between people. You can install pip/conda binaries and share with students/users. c. Shared storage so nbgrader works seamlessly. Here is a full comparison: https://datacabinet.info/pricing.html
Please excuse our landing page, it just got created today and we are fixing it.
The content in the article is a bit dated. Jupyter + Numba + Dask is the direction scientific computing (in Python) is taking. Ipyparallel is not really scalable in my experience.
If interested, I spent some time on a comment thread a few days ago describing how my experience leads me to believe the Notebook environment (not all of Jupyter / IPython, just the Notebook part) is actually only appropriate for a tiny subset of pedagogical or throw-away situations, and should be avoided most of the time and avoided in most of the cases it’s marketed for (especially anything having to do with ‘repro…
Agreed. Notebook environments are great for exploration, discovery and pedagogy. They aren't so good for productionizing code. We found this out the hard way when we tried to productionize ML code in Jupyter. We had to export to .py and add boilerplate. This works fine unless there is back and forth iteration between modeling and prod, which there invariably is; our data scientists had to make changes to the notebook…
For example, I might make two shell tabs in tmux, and make one a small rectangle towards the bottom of the screen (holds my running IPython session), and a large rectangle above it (holds my Emacs where I’m editing source code).
And I might have a third shell tab somewhere that detects any time source files are changed and re-runs unit tests.
The linked slides "Python in HPC" [0] are quite awesome, from which I learned about mpi4py's [1] existance. Definitely going to give it a try in the future; using the MPI API with an "easy-to-follow" syntax should be fun. [0] https://hpc.nih.gov/training/handouts/171121_python_in_hpc.p... [1] http://mpi4py.scipy.org/docs/
mpi4py is a solid implementation, but the docs aren’t the greatest, especially if you’re not familiar with MPI.