Viewing profile — deshpand

Growing up in a somewhat remote part of India, I would tune to BBC, Radio Australia to listen to test cricket commentary, on short wave. I have fond memories and owe a lot of my pe…

Thanks for the clarification on this particular point. I was not aware of it.

I don't know all the history, but Magnus comes off as above the game. “I am playing at least one more day here in New York and, if I do well, another day after that,” Really? And I…

Of late, I have been googling a lot of SAS and may have contributed to its rise in ranking! Not coding in SAS but moving SAS to Python. Speaking of enterprise Java, there's a ton o…

Not sure why you were downvoted. Fitness, equipment (racket and strings) are the reason if not the height. For height, above a certain level, say 6 ft 2 inch, it starts becoming a …

I used to wear cotton long pants (thermals) all year, under the suit pant. Not letting the suit pant touch the skin made it easier to wear it for a very long time (multiple months)…

I didn't mean to exaggerate. I think part of the improvement was the luxury of refactoring which should generally reduce the bloat. And, as someone else said, part of the issue is …

A ~100k LOC project in a statically typed system with type hierarchies, interfaces, contracts and boiler plate will boil down to ~10k LOC in a dynamically typed language like Pytho…

Another advice is to have a thick skin and ignore the jerks, like for instance someone downvoting your note that was written with a good intention, sharing something learnt the har…

For longevity, I will provide one advice to younger people.. get away from the mouse as much as you can and use the keyboard. My working style is editing code in vim, dark screen, …

Can you please elaborate why? I worked in Manhattan for many years, lived a few as well, and moved out in 2009.

I think it's helpful to keep in mind that Python is general purpose and used in many domains in addition to data analysis (no matter which side of the walrus operator you are on)

A little over a year ago, there was an incident with a lot of sites being down and Cloudflare/Lumen being involved (per Hacker News chatter, I only ready about the incident here). …

In the end, it's not about an OPEN SOURCE tool being perfect but whether it is helping you solving a problem. If it did not help you and YOU don't consider it production ready, the…

Do you have any citation on why "Dask doesn't consider their distributed version" to be ready? If it is your own view, then that's ok. I think dask is in heavy usage in real produc…

We use dask heavily, along with rest of the pydata ecosystem. I guess we are in the 'sweet spot' where data doesn't fit memory, to begin with, but once we perform any filtering and…

Not easy, unless you have an ability to identify opportunities that others don't have (such as applying your domain knowledge on a new trend, as I mentioned). You may do better tha…

Technically, beating the market should be like being in the top 50 percentile of your class. The reason most big funds fail to do this is because they need to overcome the fees the…

I believe compression is still a work in progress in duckdb. My current use is with parquet files which compress well but don't lend themselves to updates like a db.

I work heavily with pandas and dask (when you want to use multiple cores), using parquet files for storage. We see a lot of benefits in selectively bringing in duckdb into the mix.…

Many enterprises are coming up with patterns where they replicate the data from the database (say Redshift) into parquet files (data lake?) and directing more traffic including ana…

If you can completely stay away from Python/pandas, get all your work done with typed languages like Scala/Java, that's good. A lot of scientists and non-CS folks are using Python/…

Spark may be a mature solution for truly big data, in a SQL like fashion, 1TB and more. But I constantly see it being misused, even with datasets as small as 5GB. Maybe the valuati…

R language is an open source implementation of the original S language from IBM. Its name was a play on S and also based on the fact that both the authors of the R language have fi…

A little python training in the python stack (pandas/numpy/matplotlib or other visualization libraries) can go a long way to simplifying tech stack and get rid of these mind numbin…