Live data from Hacker News

Ditch Excel and Use Julia Data Frames

erik-engheim.medium.com

11–20 of 30 posts

Re: Ditch Excel and Use Julia Data Frames

#11
Julia and excel are fundamentally different.

In Excel, the logic/analysis part is on the same level as the data. The data and operations done on it are a single file. A datum is in a cell, and a formula is in a cell. You can move formulae around like you can move data around. You are viewing and operating on the data/logic in the same context. When you open an excel document, what you see right away is the culmination of all analyses done on the data.

With julia, the data and logic are separate entities. You have to first import data[1]. The result dataframe is created by explicitly running cells/scripts on the import. When I view the dataframe, I cannot just point-click-edit it. Any edit I make is a separate line of code to run. Then the whole script is run again over the whole dataframe to get the new result. This is opposed to excel which creates dependencies on a cell-level and live-updates the spreadsheet with each edit, but without processing the whole spreadsheet.

Since logic is its own entity (script) in julia, it is easier to write long and more complex processes: things you cannot easily do in a single-line formula parallelized over cells in excel. Also, since the processing and editing are explicitly spelled out in a script, it is easier to keep track of changes and reason about logic. So they both have their advantages.

[1]: You can even do that in excel. Import csvs, databases etc.

Re: Ditch Excel and Use Julia Data Frames

#12

People have been saying this for over 20 years now. Excel was, is and will remain a foundational aspect of compute tasks for programmers and non-programmers alike. It is not by chance that Excel is popular.

A lot of that is also simply momentum. Many people are utterly unaware of the alternatives outside the MS Office bubble. I have met so many people who thought the only program that exist for writing text is MS Word. Seriously. It is not even funny. But that is how people often get trained early on. You get taught to use MS Word at school and other context and never get exposed to alternative tools or ways of doing th…

I've been writing software for many years. I still use Excel for quick personal finance stuff, calculator and many other random lists, etc.

It takes 5 seconds to launch and you're already productive.

It has its place. Not in a data science pipeline obviously.

Re: Ditch Excel and Use Julia Data Frames

#13

People have been saying this for over 20 years now. Excel was, is and will remain a foundational aspect of compute tasks for programmers and non-programmers alike. It is not by chance that Excel is popular.

A lot of that is also simply momentum. Many people are utterly unaware of the alternatives outside the MS Office bubble. I have met so many people who thought the only program that exist for writing text is MS Word. Seriously. It is not even funny. But that is how people often get trained early on. You get taught to use MS Word at school and other context and never get exposed to alternative tools or ways of doing th…

In all honesty, no one (at least, no one in the real actual world) wants alternatives to things that already work. Give me a standard that's good enough and we'll see how to deal with edge-cases. That's what internet is for, dammit.

Yes, I know a million different better pieces of software exists for my peculiar task. No, i don't care. I have other things to do in my life than babysitting a damn computer.

Re: Ditch Excel and Use Julia Data Frames

#15

Julia and excel are fundamentally different. In Excel, the logic/analysis part is on the same level as the data. The data and operations done on it are a single file. A datum is in a cell, and a formula is in a cell. You can move formulae around like you can move data around. You are viewing and operating on the data/logic in the same context. When you open an excel document, what you see right away is the culminatio…

Are there any existing FOSS tools that let you do a similar thing to what Excel does in the data layer but with Julia (or another programming language)?

i.e. I’m in my cell and want to type my Julia right in that cell and watch it live update when I hit enter, similar to excel

Re: Ditch Excel and Use Julia Data Frames

#16

What's with "ditch Excel"? Why not "ditch Pandas", which this is much closer to. Better yet, why not just create a tutorial on using Julia without mentioning other tools, and let the reader decide? Julia lets you SUM an entire column! Unbelievable! And you can rename a column? I'm blown away! Excel has tons of point-and-click capabilities that don't require IMPORTing anything or knowing what an IOBuffer is. When you…

"...why not just create a tutorial on using Julia without mentioning other tools, and let the reader decide?"

This. Every single blog post about Julia disqualifies other tools and programming languages, most of the times very stridently.

The Julia fanboys are well known for disparaging non-stop against Python and R.

Re: Ditch Excel and Use Julia Data Frames

#17

Julia and excel are fundamentally different. In Excel, the logic/analysis part is on the same level as the data. The data and operations done on it are a single file. A datum is in a cell, and a formula is in a cell. You can move formulae around like you can move data around. You are viewing and operating on the data/logic in the same context. When you open an excel document, what you see right away is the culminatio…

Yes, I think the best approach is to combine tools. But what I think is a big problem is that people stretch the practical application of tools like Excel too far. They insist on using Excel far beyond its practical application.

Personally I never use spread sheets, other than some simple plotting. I often prefer using a simple CSV viewer instead. I find it more practical to use tools which work natively with CSV, as that is the format one will be using in Julia, Python, R or whatever programming tools you use.

Re: Ditch Excel and Use Julia Data Frames

#18

What's with "ditch Excel"? Why not "ditch Pandas", which this is much closer to. Better yet, why not just create a tutorial on using Julia without mentioning other tools, and let the reader decide? Julia lets you SUM an entire column! Unbelievable! And you can rename a column? I'm blown away! Excel has tons of point-and-click capabilities that don't require IMPORTing anything or knowing what an IOBuffer is. When you…

It is meant as an example for beginners to understand how you work with data in tables with a programming language. You got to walk before you can run.

The advantage of a programming language such as Julia is that you can simply copy paste code examples to manipulate data by just reading a blog entry as in the case.

You cannot do that with Excel. When you begin getting into more advance Excel functionality, it is not obvious how it works, and learning it will require a very visual approach. You cannot search through a blog and copy some code. You need to look at pictures and videos showing how the GUI is operated.

Alternatively read a textual description of what you need to click on. Rather cumbersome.

Re: Ditch Excel and Use Julia Data Frames

#19
post #10

you can do this with pivot tables. if your database has more than 1m entries you can do some preprocessing with powerquery. you don't even have to learn a single line of new code, if you've never coded before. you only need to know how to point and click with a mouse. and the best thing is, you can take whatever you've done and send it down to whoever happens to have installed Excel on their machine, which, in a corp…

Julia is just as easy to install as Excel. There is literally no difference. If you cannot do a one click install, then how are you supposed to figure out how to make a Pivot table?

Of course learning to code is a higher barrier. I don't think anybody challenges that. But as you begin using Excel much beyond the trivial I will argue it is actually easier to learn how to code.

Code has the advantage, that it is very easy to document and explain with text. Explaining how to do things in a Excel through text is not easy. You will need images, videos etc which are not searchable. You cannot copy-paste and image to repeat an operation from Excel. With code in contrast you can copy and paste in code you find in blogs, help documentation or anywhere else.

Basically is much easier to reuse and modify. Once you learn how to code you can advance your capabilities much quicker.

Re: Ditch Excel and Use Julia Data Frames

#20
post #9

What is the benefit of using Julia over R?

A more user-friendly, versatile and higher performance language. E.g. Julia can do everything R, Python, Fortran, Matlab and C can do in principle all in one language.

You can use R and Python packages from Julia. You can call Fortran and C code from it. Unlike R and Python you don't need to have a module rewritten to C or Fortran code to get high enough performance.

In fact Julia will often outperform C and Fortran code. There are researchers which have switched from Fortran to Julia to get a higher productivity programming language expecting to get a performance drop only to realize they got a 3x performance increase.

The main downside of Julia is that it is a very new language. R and Python and old and hence well known, but Julia is a rapidly growing and taking over serious areas. Next generation climate models are built using Julia e.g.

Post reply on HN