Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Advanced computing with IPython

lwn.net

Post reply on HN
Print view View original

81–90 of 108 posts

Re: Advanced computing with IPython

#81

Post by chrxr »

nightski wrote: post #63
sametmax wrote: post #51

Earlier quoted context omitted.

If you have 100 students, you will have 100 different mistakes to debug in the setup. A setup is not a program, there is rarely an easy way to pinpoint a problem, and so it takes a lot of time to setup just one, let alone a 100. When the number of hours are limited, it's best to skip it entirely, and just provide a solid paper tutorial.

I'm so glad my college had class sizes of 10-15 and not 100, what a waste of money.

Average class size of those using JupyterHub was more like 30, excluding two very large, very popular classes. Worth remembering all the classes have labs and sections which are much smaller.

Re: Advanced computing with IPython

#82

Post by HerrMonnezza »

uasnew wrote: post #15
chrxr wrote: post #10

At Harvard we've built out an infrastructure to allow us to deploy JupyterHub to courses with authentication managed by Canvas. It has allowed us to easily deploy complex set-ups to students so they can do really cool stuff without having to spend hours walking them through setup. Instructors are writing their lectures as IPython notebooks, and distributing them to students, who then work through them in their Jupyte…

Is this something that is open source so that other schools could use it?

If I'm allowed a shameless plug (I'm the developer), there is ElastiCluster [1] which can deploy (among others) JupyterHub on various IaaS cloud infrastructures. We have used it at UZH to provide teaching environments for various courses. By default it only comes with the local "passwd" authenticator, but anything that works with Jupyter or Linux/PAM can be used relatively easily (= write an Ansible playbook).

Feel free to reach out to me if you would like more info.

[1]: https://gc3-uzh-ch.github.com/elasticluster

Re: Advanced computing with IPython

#83

Post by Bromskloss »

spot wrote: post #76
Bromskloss wrote: post #73

Earlier quoted context omitted.

Indeed. If I could only figure out how to have it automatically run `from math import *` (and then present me with the interactive shell, I could use it as a calculator too.

http://ipython.readthedocs.io/en/stable/config/options/termi...

My problem is that I don't know how to make it accept spaces in the command to be run. This works:

    ipython3 --InteractiveShellApp.exec_lines='["print(2)"]'
This does not work:

    ipython3 --InteractiveShellApp.exec_lines='["from math import *"]'
The latter command results in the following error:

    [TerminalIPythonApp] CRITICAL | The 'exec_lines' trait of a TerminalIPythonApp instance must be a list, but a value of class 'str' (i.e. '["from') was specified.

Re: Advanced computing with IPython

#84

Post by takluyver »

Bromskloss wrote: post #83
spot wrote: post #76

Earlier quoted context omitted.

http://ipython.readthedocs.io/en/stable/config/options/termi...

My problem is that I don't know how to make it accept spaces in the command to be run. This works: ipython3 --InteractiveShellApp.exec_lines='["print(2)"]' This does not work: ipython3 --InteractiveShellApp.exec_lines='["from math import *"]' The latter command results in the following error: [TerminalIPythonApp] CRITICAL | The 'exec_lines' trait of a TerminalIPythonApp instance must be a list, but a value of class '…

Rather than fiddling around with quoting bash, you can put the import line in a startup file: http://ipython.readthedocs.io/en/stable/interactive/tutorial...

Re: Advanced computing with IPython

#85

Post by Something1234 »

AdmiralAsshat wrote: post #32

Deep research uses aside, I often prefer to use IPython because it's simply a better shell than the default Python shell. You get basic niceties like tab completion and being able to up-arrow to revise an earlier multi-line command (like a function) without it being an exercise in frustration.

The multiline history is so important for an interactive shell with a language that considers indentation important.

Re: Advanced computing with IPython

#86

Post by spot »

Bromskloss wrote: post #83
spot wrote: post #76

Earlier quoted context omitted.

http://ipython.readthedocs.io/en/stable/config/options/termi...

My problem is that I don't know how to make it accept spaces in the command to be run. This works: ipython3 --InteractiveShellApp.exec_lines='["print(2)"]' This does not work: ipython3 --InteractiveShellApp.exec_lines='["from math import *"]' The latter command results in the following error: [TerminalIPythonApp] CRITICAL | The 'exec_lines' trait of a TerminalIPythonApp instance must be a list, but a value of class '…

idk, it works for me:

    $ ipython3 --InteractiveShellApp.exec_lines='["from math import *"]'
    Python 3.6.5 |Anaconda, Inc.| (default, Apr 26 2018, 08:42:37)
    Type 'copyright', 'credits' or 'license' for more information
    IPython 6.4.0 -- An enhanced Interactive Python. Type '?' for help.

    In [1]: sin(5)
    Out[1]: -0.9589242746631385

Re: Advanced computing with IPython

#87

Post by nl »

da_chicken wrote: post #13
chrxr wrote: post #10

At Harvard we've built out an infrastructure to allow us to deploy JupyterHub to courses with authentication managed by Canvas. It has allowed us to easily deploy complex set-ups to students so they can do really cool stuff without having to spend hours walking them through setup. Instructors are writing their lectures as IPython notebooks, and distributing them to students, who then work through them in their Jupyte…

I'm torn. On the one hand that's really cool to get everything configured and up and running so students can get to the interesting parts. On the other hand, learning how to configure your own environment is kind of an essential part of working with any tool that forces you to understand at least some of the structure involved.

(Professional ML developer here)

It's almost a waste of time teaching people to setup a deep learning stack.

NVidia will break everything you do with every release and any instructions you write will be outdated in weeks.

For example, the TensorFlow/CUDA/CuDNN installation changes continually because you can't install the default releases of any of them and get a working system.

Re: Advanced computing with IPython

#88

Post by joshuamorton »

mlthoughts2018 wrote: post #80
joshuamorton wrote: post #77

Earlier quoted context omitted.

>To me this falls flat because that’s not why people want to use a notebook. Generally they want to use it because it’s superficially easier to jumble all concerns into a single context and not think about coupling or separation, and just disregard testing and other best practices. But this is not unique to notebooks. You can very easily do the exact same things with raw python files, its just that in the ecosystem y…

> "You can very easily do the exact same things with raw python files, its just that in the ecosystem you work in, raw text files are treated more maturely." This is non-sequitur to the whole discussion. You can write bad code in any tool. That has no bearing on this. Instead we should ask, "what does it require to write good code in a given tool." In plain source files, we know the answer, with lots of theory of des…

>In notebooks, the answer is that you have to jump through a lot of hoops to write things in a non-notebook-way -- that is, specifically in a way where you factor things out into the text files anyway -- if you want those good patterns.

I guess I still consider this a "notebook-y" way. Its just good practice, instead of bad practice. But still notebooky.

You seem to be using notbooky to mean sloppy and disorganized, whereas I mean it as interactive and literate. Those need not be correlated, and I think that "driver scripts" still benefit a lot from the interativity and literateness

Re: Advanced computing with IPython

#89

Post by Bromskloss »

spot wrote: post #86
Bromskloss wrote: post #83

Earlier quoted context omitted.

My problem is that I don't know how to make it accept spaces in the command to be run. This works: ipython3 --InteractiveShellApp.exec_lines='["print(2)"]' This does not work: ipython3 --InteractiveShellApp.exec_lines='["from math import *"]' The latter command results in the following error: [TerminalIPythonApp] CRITICAL | The 'exec_lines' trait of a TerminalIPythonApp instance must be a list, but a value of class '…

idk, it works for me: $ ipython3 --InteractiveShellApp.exec_lines='["from math import *"]' Python 3.6.5 |Anaconda, Inc.| (default, Apr 26 2018, 08:42:37) Type 'copyright', 'credits' or 'license' for more information IPython 6.4.0 -- An enhanced Interactive Python. Type '?' for help. In [1]: sin(5) Out[1]: -0.9589242746631385

I see. I have iPython 5.1.0. Maybe there was a bug that has been fixed in yours.

Re: Advanced computing with IPython

#90

Post by cbcoutinho »

nightcracker wrote: post #60
wenc wrote: post #22

Earlier quoted context omitted.

Agreed. Notebook environments are great for exploration, discovery and pedagogy. They aren't so good for productionizing code. We found this out the hard way when we tried to productionize ML code in Jupyter. We had to export to .py and add boilerplate. This works fine unless there is back and forth iteration between modeling and prod, which there invariably is; our data scientists had to make changes to the notebook…

I solve this in my personal workflow by extracting the important bit to a module, editing in that module, and testing/exploring changes in a notebook by reloading the module.

This is how I work as well, where all the code I'm working with in a jupyter notebook is directly visible on my screen. Any other code is generally 'finished' and put into a text editor.

Additionally, I use the following settings in my ipython_config.py file to automatically reload modules:

c.InteractiveShellApp.extensions = [ 'autoreload' ]

c.InteractiveShellApp.exec_lines [ '%autoreload 2' ]

Post reply on HN