Voila – From notebooks to standalone web applications and dashboards • Hacker News Board

Re: Voila – From notebooks to standalone web applications and dashboards

#21

Post by BiteCode_dev » Tue, Aug 31, 2021, 8:47 AM UTC

pavlovskyi wrote: post #14

BiteCode_dev wrote: post #9

Earlier quoted context omitted.

That's because they are not programmers. One should not expect them to be experts in 2 fields. They do their work with the tool provided, and if we want a better output, we need to provide better tooling or accept what comes out. I want my physicists to spend their mental effort on physics, not on software architecture.

I agree that they are not programmers but in my opinion it cannot become an explanation to write a code which become non repeatable, especially if they have big impact on how the flow will look on production environment.

Scientists are not meant to create code that ends up on production. Once they have a working concept, they should team up with a programmer to make it live.

Again, if it's not possible, then you accept the imperfection of the result, or provide better tooling.

There is no blame to put on them whatsoever.

Re: Voila – From notebooks to standalone web applications and dashboards

#22

Post by pavlovskyi » Tue, Aug 31, 2021, 8:48 AM UTC

vladev wrote: post #17

Note that there's also streamlit [1]. It uses regular python files, rather than notebooks, so they can be easily version controlled. And it has more UI tools. [1]: https://streamlit.io/

In my daily routine we are using streamlit and it is pretty decent, mainly because you do not have to care much about backend. And, what was mention by you it has impressive amount of UI tools and relatively active community.

Re: Voila – From notebooks to standalone web applications and dashboards

#23

Post by throwaway210222 » Tue, Aug 31, 2021, 8:54 AM UTC

BiteCode_dev wrote: post #21

pavlovskyi wrote: post #14

Earlier quoted context omitted.

I agree that they are not programmers but in my opinion it cannot become an explanation to write a code which become non repeatable, especially if they have big impact on how the flow will look on production environment.

Scientists are not meant to create code that ends up on production. Once they have a working concept, they should team up with a programmer to make it live. Again, if it's not possible, then you accept the imperfection of the result, or provide better tooling. There is no blame to put on them whatsoever.

I disagree, many, many scientists hire a professional statistician to do the stats for their papers.

Similarly, they should hire experienced, qualified software engineers to write/check the software in their papers.

They don't because 'everyone can code - its just logic'.

Re: Voila – From notebooks to standalone web applications and dashboards

#24

Post by jstx1 » Tue, Aug 31, 2021, 9:05 AM UTC

BiteCode_dev wrote: post #21

pavlovskyi wrote: post #14

Earlier quoted context omitted.

I agree that they are not programmers but in my opinion it cannot become an explanation to write a code which become non repeatable, especially if they have big impact on how the flow will look on production environment.

Scientists are not meant to create code that ends up on production. Once they have a working concept, they should team up with a programmer to make it live. Again, if it's not possible, then you accept the imperfection of the result, or provide better tooling. There is no blame to put on them whatsoever.

> Scientists are not meant to create code that ends up on production.

You might be mixing up the terms. I don't think that the point was about "scientists" in general, it was about "data scientists". The first is a common term used to describe someone who does science in some professional capacity. The second one is a very broad job title within software which very often includes writing code that ends up in production - at some data science roles that might even be your main responsiblity.

Re: Voila – From notebooks to standalone web applications and dashboards

#25

Post by rogue7 » Tue, Aug 31, 2021, 9:08 AM UTC

I recently used this to do a POC at my day job. I was able to demo a machine learning tool quite smoothly to executives.

Later it was implemented in production with a regular stack (Flask + Vue).

Voila is really empowering for e.g. data scientists that are comfortable in a jupyter environment but aren't js wizards. Running locally, I just love the reactivity it provides: you don't worry about sync between front-end and back-end, everything is propagated through websockets I believe (Jupyter is Tornado-based).

However, for production you might want to use another tool, since it (currently) executes every session in isolation, so every time an user connects it re-runs everything from scratch. Moreover, the round-trips to the server can be slow if you are e.g. in a different continent so this degrades the UX.

Here is an example of a small ML app I built with Voilà (this will probably crash due to HN hug of death™), and JAX on the backend: http://grad-descent.herokuapp.com/

Re: Voila – From notebooks to standalone web applications and dashboards

#26

Post by BiteCode_dev » Tue, Aug 31, 2021, 9:17 AM UTC

jstx1 wrote: post #19

BiteCode_dev wrote: post #9

Earlier quoted context omitted.

That's because they are not programmers. One should not expect them to be experts in 2 fields. They do their work with the tool provided, and if we want a better output, we need to provide better tooling or accept what comes out. I want my physicists to spend their mental effort on physics, not on software architecture.

> That's because they are not programmers. Beginners not being aware of some best practices doesn't automatically make them not-programmers. I work as a data scientist and I see it as part of software development. It's just a different domain - some people do front-end, some do mobile or embedded, I do data science.

Data scientists are not programmers. They are data scientists.

Just like I'm not a data scientists, I'm a programmer.

Now, I can use pandas in a pinch and makes pretty graphs, but my statistical analysis will never be on part with yours.

Just like a pianist hobbyist will have a hard time to rival somebody who does that 40 hours a week, although he may be able to play a few fantastic pieces.

Hell, even a web dev programmers, if ask to code a GUI desktop app, is not going to do a good job.

IT is becoming a very large field.

And scientists are not even from this field.

Re: Voila – From notebooks to standalone web applications and dashboards

#27

Post by BiteCode_dev » Tue, Aug 31, 2021, 9:18 AM UTC

throwaway210222 wrote: post #23

BiteCode_dev wrote: post #21

Earlier quoted context omitted.

Scientists are not meant to create code that ends up on production. Once they have a working concept, they should team up with a programmer to make it live. Again, if it's not possible, then you accept the imperfection of the result, or provide better tooling. There is no blame to put on them whatsoever.

I disagree, many, many scientists hire a professional statistician to do the stats for their papers. Similarly, they should hire experienced, qualified software engineers to write/check the software in their papers. They don't because 'everyone can code - its just logic'.

Err... That's kinda my point ?

Re: Voila – From notebooks to standalone web applications and dashboards

#28

Post by bluescarni » Tue, Aug 31, 2021, 9:19 AM UTC

BiteCode_dev wrote: post #9

pavlovskyi wrote: post #4

I really like Jupyter notebooks to build a simple concept and then move to .py files. But what I observe, especially at the entry level or junior level jobs in data science is that people spend huge amount of its work on jupyter, which did not focus on how to plan flow properly. What I meant is that there is very short path from usefullness to overkill.

That's because they are not programmers. One should not expect them to be experts in 2 fields. They do their work with the tool provided, and if we want a better output, we need to provide better tooling or accept what comes out. I want my physicists to spend their mental effort on physics, not on software architecture.

As a scientist myself, I strongly disagree. If you spend a non-negligible amount of your time telling a computer what to do, you are, indeed, a programmer. And as such you should be expected to become a decently proficient programmer.

Physicists are not mathematicians, and yet they are required to acquire a relatively high degree of proficiency in maths because maths is a fundamental tool in their job, and nobody would argue otherwise.

The attitude of considering programming a mundane craft to be picked up as-you-go is the main reason why the scientific software landscape is such a shitshow.

/rant

Re: Voila – From notebooks to standalone web applications and dashboards

#29

Post by BiteCode_dev » Tue, Aug 31, 2021, 9:25 AM UTC

jstx1 wrote: post #24

BiteCode_dev wrote: post #21

Earlier quoted context omitted.

Scientists are not meant to create code that ends up on production. Once they have a working concept, they should team up with a programmer to make it live. Again, if it's not possible, then you accept the imperfection of the result, or provide better tooling. There is no blame to put on them whatsoever.

> Scientists are not meant to create code that ends up on production. You might be mixing up the terms. I don't think that the point was about "scientists" in general, it was about "data scientists". The first is a common term used to describe someone who does science in some professional capacity. The second one is a very broad job title within software which very often includes writing code that ends up in producti…

Even among devs with very close speciality, the difference in productivity is immense. Take a iOS team, make them dev a MacOS desktop app, and see their output plummet in their of productivity or quality. They will end up doing a good job, but it will take between 6 months and a year to catch up with a specialized team.

A data scientist is not even a somebody trained as a programmer. Their strong suit is data analysis, and it turns out one of the tool to manipulate data today are programming languages so their do it.

But I as a Python trainer, I train data analyst regularly, and they don't have a clue about language ecosystems, how the OS work, data formats or reliable software architecture.

They mainly want to output their graph, pdf report or other media to serve their conclusion. They may want to create some reusable algo, or machine learning model, but that's the limit most of them hit.

If one take their code and put it in prod (which I know happens, don't get me wrong), that's not the data scientist fault. They are doing their job, in which programming is just one of the many means to an end, and is not their specialty.

Re: Voila – From notebooks to standalone web applications and dashboards

#30

Post by jstx1 » Tue, Aug 31, 2021, 9:27 AM UTC

BiteCode_dev wrote: post #26

jstx1 wrote: post #19

Earlier quoted context omitted.

> That's because they are not programmers. Beginners not being aware of some best practices doesn't automatically make them not-programmers. I work as a data scientist and I see it as part of software development. It's just a different domain - some people do front-end, some do mobile or embedded, I do data science.

Data scientists are not programmers. They are data scientists. Just like I'm not a data scientists, I'm a programmer. Now, I can use pandas in a pinch and makes pretty graphs, but my statistical analysis will never be on part with yours. Just like a pianist hobbyist will have a hard time to rival somebody who does that 40 hours a week, although he may be able to play a few fantastic pieces. Hell, even a web dev progr…

As a data scientist I spend more time writing software and building things than I spend doing statistical analysis (and I like it that way). That's why I consider it part of software development. Don't be tripped up by the word "scientist" in the job title (I wrote another comment about this in this thread). Also, it's a very broad field and it varies a lot from company to company - if you've had some interactions with data scientists, don't assume that what they do applies across the industry. I'm basing what I'm writing on having worked at/with companies of different sizes and in different industries.