Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Advanced computing with IPython

lwn.net

Post reply on HN
Print view View original

91–100 of 108 posts

Re: Advanced computing with IPython

#91

Post by mlthoughts2018 »

cbcoutinho wrote: post #90
nightcracker wrote: post #60

Earlier quoted context omitted.

I solve this in my personal workflow by extracting the important bit to a module, editing in that module, and testing/exploring changes in a notebook by reloading the module.

This is how I work as well, where all the code I'm working with in a jupyter notebook is directly visible on my screen. Any other code is generally 'finished' and put into a text editor. Additionally, I use the following settings in my ipython_config.py file to automatically reload modules: c.InteractiveShellApp.extensions = [ 'autoreload' ] c.InteractiveShellApp.exec_lines [ '%autoreload 2' ]

Note that the autoreload features can be very tricky to use safely with Python.

For example, at least in some previous versions, Caffe and TensorFlow make incompatible assumptions about the ability to claim all available GPU memory. So there can be situations where you first import Caffe, then later import TensorFlow with restrictions on its GPU policy. If you naively re-import the Caffe code, it can evict TensorFlow from whatever GPUs it had reclaimed, and coming up with a group of settings that reliably prevent this, across possibly different machine where the notebook will be run, is very tricky.

This once led to a huge time sink because someone on my team created a mistaken GitHub issue claiming our TensorFlow model had a bug (since the notebook was producing an error). We spent all this time trying to reproduce it and figure out why it wasn't working, and eventually realized it was because of this hidden auto-reload setting on his specific IPython setup that caused Caffe to evict TensorFlow just for his specific usage pattern, resulting in strange errors because the TensorFlow model was no longer loaded in GPU memory.

There can be other problems too, like auto-reloading modules that have large start-up times (say if they load a very large model into memory). Sometimes you want to re-run a cell without auto-reload, even if you still want selective auto-reload functionality in other parts.

Re: Advanced computing with IPython

#92

Post by newen »

xapata wrote: post #68
commandlinefan wrote: post #38

Earlier quoted context omitted.

I thought so, too - enough that I checked the date, but the date on the article is yesterday. Maybe it was written a while ago and just now uploaded. I thought it was strange to refer to IPython as a standalone interpreter outside of Jupyter... I mean, you can, but I don't think anybody does anymore.

> I don't think anybody does anymore Counter-example: I use IPython in the terminal, outside of Jupyter.

Me too! I actually hook up an Ipython terminal to a Jupyter notebook and in order to do REPL type work in the terminal. Much better to work in a REPL terminal and also save my work in the notebook.

Re: Advanced computing with IPython

#93

Post by mlthoughts2018 »

joshuamorton wrote: post #88
mlthoughts2018 wrote: post #80

Earlier quoted context omitted.

> "You can very easily do the exact same things with raw python files, its just that in the ecosystem you work in, raw text files are treated more maturely." This is non-sequitur to the whole discussion. You can write bad code in any tool. That has no bearing on this. Instead we should ask, "what does it require to write good code in a given tool." In plain source files, we know the answer, with lots of theory of des…

>In notebooks, the answer is that you have to jump through a lot of hoops to write things in a non-notebook-way -- that is, specifically in a way where you factor things out into the text files anyway -- if you want those good patterns. I guess I still consider this a "notebook-y" way. Its just good practice, instead of bad practice. But still notebooky. You seem to be using notbooky to mean sloppy and disorganized,…

> "You seem to be using notbooky to mean sloppy and disorganized"

Not exactly. I'm using "notebooky" to mean whatever the prominent, advertised, praised and recommended usage patterns and workflows are for a large body of notebooks and from prominent presentations using the notebook as the lingua franca, especially in circles where the notebook is claimed to be central to "reproducible science" or where the notebook is described like a software equivalent of a "lab notebook."

The way that the notebook community, from academics and prominent leaders, to people who give talks this way, on down to data science practitioners, recommends using notebooks seems to inherently result in what you call "sloppy and disorganized" code. That code is the intended type of workflow, which is why I am trying to distill out principles for why it's arguably not a good idea. (Meaning why the intended way to use notebooks is self-defeating.)

Re: Advanced computing with IPython

#94

Post by meko »

kaitai wrote: post #44
w0m wrote: post #29

Earlier quoted context omitted.

Agreed. At the beginning of class, walk students through the setup. Then for every project after, let them use the pre-rolled systems.

I'd actually do the opposite. Let them use pre-rolled first, then when they actually know and care about how the system is set up, have them set it up the way they like it. (I am actually leading a machine learning for high-schoolers camp in 2 weeks and we are using Jupyter notebooks so that all students, with heterogeneous backgrounds, will start in the same place and get to the fun stuff fast. Many will never have…

I wish I had ML camps growing up! Mine were photography and adobe flash :<

Re: Advanced computing with IPython

#95

Post by bunderbunder »

pm90 wrote: post #67
chrxr wrote: post #11

Earlier quoted context omitted.

It is worth noting that there is an argument that it is a worthwhile task for students to learn how to setup complex computing environments, as it better prepares them for the real world. However, in reality, there just isn't time within a single semester to do this for a class of 100+ students. So implementations such as this one trade-off that learning for a greater focus on computational theory and its implementat…

Its a valid point to consider that as part of the learning experience. I don't think that students with limited time should be forced to go through that though. If you get them hooked on to computing, their natural curiosity would lead them to explore it further. Bogging them down with unnecessary setup stuff would probably only make them think that this shit takes way too much effort etc. I've had many courses that…

Same here.

I want to say that, if it's pedagogically valuable, then it needs to be made into a small lab course (or part of the lab unit for an intro class), and taught once, in an organized manner.

And then stop letting professors hide behind this lame excuse so that they can get on with teaching the stuff that their course is actually about.

Re: Advanced computing with IPython

#96

Post by akhilcacharya »

chrxr wrote: post #11
chrxr wrote: post #10

At Harvard we've built out an infrastructure to allow us to deploy JupyterHub to courses with authentication managed by Canvas. It has allowed us to easily deploy complex set-ups to students so they can do really cool stuff without having to spend hours walking them through setup. Instructors are writing their lectures as IPython notebooks, and distributing them to students, who then work through them in their Jupyte…

It is worth noting that there is an argument that it is a worthwhile task for students to learn how to setup complex computing environments, as it better prepares them for the real world. However, in reality, there just isn't time within a single semester to do this for a class of 100+ students. So implementations such as this one trade-off that learning for a greater focus on computational theory and its implementat…

It's Harvard. They'll figure it out if push comes to shove.

Re: Advanced computing with IPython

#97

Post by akhilcacharya »

chrxr wrote: post #10

At Harvard we've built out an infrastructure to allow us to deploy JupyterHub to courses with authentication managed by Canvas. It has allowed us to easily deploy complex set-ups to students so they can do really cool stuff without having to spend hours walking them through setup. Instructors are writing their lectures as IPython notebooks, and distributing them to students, who then work through them in their Jupyte…

>Our most ambitious so far has been setting up each student in the course with a p2.xlarge machine with cuda and TensorFlow so they could do deep learning work for their final projects.

Wow! How expensive was this? Do you do any sort of shutdown/startup work or use pre-empt instances?

Re: Advanced computing with IPython

#98

Post by akhilcacharya »

nightski wrote: post #63
sametmax wrote: post #51

Earlier quoted context omitted.

If you have 100 students, you will have 100 different mistakes to debug in the setup. A setup is not a program, there is rarely an easy way to pinpoint a problem, and so it takes a lot of time to setup just one, let alone a 100. When the number of hours are limited, it's best to skip it entirely, and just provide a solid paper tutorial.

I'm so glad my college had class sizes of 10-15 and not 100, what a waste of money.

> what a waste of money.

Uh, you know its Harvard we're talking about right?

Re: Advanced computing with IPython

#99

Post by chrxr »

akhilcacharya wrote: post #97
chrxr wrote: post #10

At Harvard we've built out an infrastructure to allow us to deploy JupyterHub to courses with authentication managed by Canvas. It has allowed us to easily deploy complex set-ups to students so they can do really cool stuff without having to spend hours walking them through setup. Instructors are writing their lectures as IPython notebooks, and distributing them to students, who then work through them in their Jupyte…

>Our most ambitious so far has been setting up each student in the course with a p2.xlarge machine with cuda and TensorFlow so they could do deep learning work for their final projects. Wow! How expensive was this? Do you do any sort of shutdown/startup work or use pre-empt instances?

The answer for this particular course was, "very expensive". Each students machine shuts down after 20 minutes of inactivity (defined as no Jupyter process currently running), so that saves money, but in this specific instance many of the students were running extremely long jobs, so that didn't really help us.

The average cost over all of the other courses was something like $2-3 per month per student. The deep learning course ended up being closer to $20 per student. Thanks to Amazon Educate almost the entire cost was covered with credit.

Re: Advanced computing with IPython

#100

Post by akhilcacharya »

chrxr wrote: post #99
akhilcacharya wrote: post #97

Earlier quoted context omitted.

>Our most ambitious so far has been setting up each student in the course with a p2.xlarge machine with cuda and TensorFlow so they could do deep learning work for their final projects. Wow! How expensive was this? Do you do any sort of shutdown/startup work or use pre-empt instances?

The answer for this particular course was, "very expensive". Each students machine shuts down after 20 minutes of inactivity (defined as no Jupyter process currently running), so that saves money, but in this specific instance many of the students were running extremely long jobs, so that didn't really help us. The average cost over all of the other courses was something like $2-3 per month per student. The deep lear…

Interesting. Do you think your use case could be helped by doing only the training remotely on a single GPU/GPU cluster, and doing the rest of the development work on a cheaper machine? (basically just equivalent of estimator train() runs on a faster machine that quits afterward).

I wrote a tool that does that with Keras but I'm not sure if it's actually useful for real-world use cases.

Post reply on HN