Live data from Hacker News

R: Introduction to Data Science (2019)

rafalab.dfci.harvard.edu

131–139 of 139 posts

Re: R: Introduction to Data Science (2019)

#131

Earlier quoted context omitted.

Am I correct in understanding that you have the spot price of Hopper card compute as your underlying and then come up with a pricing equation for some derivative instruments for that?

In a friction free scenario it would be a standard future, yes. The reality is closer to an option on an FX forward, with a very nasty empirical MC as Q* for the payoff equivalence. I’m not fancy enough, I know when to sub-contract!

Have you tried looking at SABR?

If what you have is FX-like I wouldn't be able to help beyond that anyway, FX modelling is its own thing and I haven't done anything there since the obligatory uni courses(in equity space myself). AFAIK the general way to do things in rates/FX is SABR for vanilla and then PDE/MonteCarlo for exotics, but I was never on an FX desk so don't want to point you in the wrong direction.

Re: R: Introduction to Data Science (2019)

#132
post #18

Earlier quoted context omitted.

Pk/PD work for pharmaceutical data analysis I didn’t like using R at first but I’ve come to appreciate the speed that comes with months of experience. It’s a language which feels like it has a lot of magical incantations you need to remember - the default namespace is much more crowded. Functions like sapply vs mapply are tricky to reason about from the documentation alone. The values NA vs Null vs integer(0) are all…

> Functions like sapply vs mapply are tricky to reason about from the documentation alone. Could you please expand on that? It's unclear what you're referring to. > The values NA vs Null vs integer(0) are all used as standins for real thrown errors and knowing which one to check for after calling a function can be tough. `checkmate::assert_numeric()` (or similar) with base R you want isTRUE(): `stopifnot(isTRUE(is.fi…

I’ll check out those assert functions and I only meant that having apply,lapply,sapply,mapply took some getting used to most programming languages I had used prior include only 1 map / apply function and it’s still not clicking for me when there are performance penalties choosing one over another although I’ve read in the documentation that there are pros and cons

Re: R: Introduction to Data Science (2019)

#133

Earlier quoted context omitted.

In a friction free scenario it would be a standard future, yes. The reality is closer to an option on an FX forward, with a very nasty empirical MC as Q* for the payoff equivalence. I’m not fancy enough, I know when to sub-contract!

Have you tried looking at SABR? If what you have is FX-like I wouldn't be able to help beyond that anyway, FX modelling is its own thing and I haven't done anything there since the obligatory uni courses(in equity space myself). AFAIK the general way to do things in rates/FX is SABR for vanilla and then PDE/MonteCarlo for exotics, but I was never on an FX desk so don't want to point you in the wrong direction.

As I’m sure you can tell, pricing exotic derivatives isn’t my day job.

But your reminder to think of SABR/implied-vol is useful: I think there’s a convexity argument that can be made around how fat the tails would need to be.

I’m not sure anyone is going to be thrilled at “anywhere between one hundred dollars and one hundred million dollars”, but my job is to figure out the bounds.

Re: R: Introduction to Data Science (2019)

#134
post #123

Earlier quoted context omitted.

Not parent and I also consider R to be a surprisingly solid general purpose PL but the one reason that I can think of is that formulas are part of the language. Formulas are extremely well-suited for data analysis but odd outside of that.

No R is not a "surprisingly solid general purpose PL". Not in comparison to better option such as Ruby, Python, Perl etc.

But yes in comparison to Stata which is its main competitor in the space where I am using it. This does make it possible to write much more modular, legible, and well-tested R code than Stata do files.

Re: R: Introduction to Data Science (2019)

#135
post #75

Earlier quoted context omitted.

I've done some work for scientists where they used C++ extensions to R for heavy number crunching. For their workflow, R is really nice. Don't know how common this is though.

Rcpp is pretty common in major performance sensitive packages. The CppCast did an interview with Dirk Eddelbuettel about it in 2022: https://cppcast.com/rcpp/

yeah, that's what it was, I was taking apart and documenting an RCPP module they had, but for which they no longer had access to the coder. It was pretty cool work, would be happy to do more with RCPP

Re: R: Introduction to Data Science (2019)

#136

i'm an old R user, now migrated fully to python. For those of you who us R still what is your use case? We found R has a really hard time integrating into data pipelines and was best used as a standalone tool by individuals, which doesn't really work in our particular professional setup where everyone works collaboratively together. What we found was that R had alot of packages but most haven't been touched in years…

I had thought this book was redone by someone in Python. Does anyone remember seeing that?

Re: R: Introduction to Data Science (2019)

#137
post #112

Earlier quoted context omitted.

shiny is nice for one-off data dashboards and single-purpose mini-apps. I see the python equivalents are like dash/plotly. Shiny is not a full fledged web framework, and isn't a viable replacement for e.g. Django. Aside -- we tried using dash in our production app and then had to remove it after a month, because these types of frameworks that spit out front-end code are almost never flexible enough to do what you act…

I'd highly encourage you to look into shiny more. No, it's not django, but it's a much richer framework than dash, and you can always bring your own HTML if what it generates for you isn't sufficient.

I'm not arguing that dash is better than shiny -- I think shiny is probably better!

But the fact that there's no Django for R means shiny's a dead-end for a production web app.

Re: R: Introduction to Data Science (2019)

#138
post #112

Earlier quoted context omitted.

I'd highly encourage you to look into shiny more. No, it's not django, but it's a much richer framework than dash, and you can always bring your own HTML if what it generates for you isn't sufficient.

I'm not arguing that dash is better than shiny -- I think shiny is probably better! But the fact that there's no Django for R means shiny's a dead-end for a production web app.

> there's no django for R

ambiorix might be what you're looking for.

check it out: https://ambiorix.dev/

it provides: - routing - api generation - templating - web sockets

Re: R: Introduction to Data Science (2019)

#139
post #73
post #30

Earlier quoted context omitted.

> I can’t remember the last time a library incompatibility led to a show stopper. Oh, it’s very common unless you basically only use much better (and ‘rig’ and PPM also help in small but important ways). But it’s still dire. At work we are facing these issues every other week on some code base.

I'd love to hear more about this because from my perspective renv does seem to solve 95% of the challenges the folks face in practice. I wonder what makes your situation different? What are we missing in renv?

Oh, I totally agree that ‘renv’ probably solves 95% of problems. But those pesky 5%…

I think that most problems are ultimately caused by the fact that R packages cannot really declare versioned dependencies (most packages only declare `>=` dependency, even though they could also give upper bounds [1]; and that is woefully insufficient), and installing a package’s dependencies will (almost?) always install the latest versions, which may be incompatible with other packages. But at any rate ‘renv’ currently seems to ignore upper bounds: e.g. if I specify `Imports: dplyr (>= 0.8), dplyr (The single one thing that causes most issues for us at work is a binary package compilation issue: the `configure` file for ‘httpuv’ clashes with our environment configuration, which is based on Gentoo Prefix and environment modules. Even though the `configure` file doesn’t hard-code any paths, it consistently finds the wrong paths for some system dependencies (including autotools). According to the system administrators of our compute cluster this is a bug in ‘httpuv’ (I don’t understand the details, and the configuration files look superficially correct to me, but I haven’t tried debugging them in detail, due to their complexity). But even if it were fixed, the issue would obviously persist for ‘renv’ projects requiring old versions.

(We are in the process of introducing a shared ‘renv’ package cache; once that’s done, the particular issue with ‘httpuv’ will be alleviated, since we can manually add precompiled versions of ‘httpuv’, built using our workaround, to that cache.)

Another issue is that ‘renv’ attempts to infer dependencies rather than having the user declare them explicitly (a la pyproject.toml dependencies), and this is inherently error-prone. I know this behaviour can be changed via `settings$snapshot.type("explicit")` but I think some of the issues we’re having are exacerbated by this default, since `renv::status()` doesn’t show which ones are direct and which are transitive dependencies.

Lastly, we’ve had to deactivate ‘renv’ sandboxing since our default library is rather beefy and resides on NFS, and initialising the sandbox makes loading ‘renv’ projects prohibitively slow — every R start takes well over a minute. Of course this is really a configuration issue: as far as I am concerned, the default R library should only include base and recommended packages. But it in my experience it is incredibly common for shared compute environments to push lots of packages into the default library. :-(

---

[1] R-exts: “A package or ‘R’ can appear more than once in the ‘Depends’ field, for example to give upper and lower bounds on acceptable versions.”

Post reply on HN