Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Skip – A programming language to skip the things you have already computed

skiplang.com

Post reply on HN
Print view View original

91–100 of 103 posts

Re: Skip – A programming language to skip the things you have already computed

#91

Post by thom »

Incremental computation is a fascinating area of research, I've followed Matthew Hammer's work on Adapton[1] for a while, as well as Daco Harkes work on IceDust[2]. The issues are obviously quite deep (i.e. how do can you change a single value and recalculate quicksort without the effects of the change fanning out massively?) whether you're targeting existing languages and runtimes or building from scratch.

I would very much like to see this as an area of active development both in programming languages and distributed computing platforms. Current streaming platforms are great and it's getting easier and easier to create declarative data flow pipelines, doing real-time processing, splitting things into windows etc. But on top of that I imagine incorporating incremental computation, allowing you to keep a durable history of the event stream (or at least manageable parts of it) in a way that automagically allows incremental recalculation of upstream data when an underlying fact is retracted or updated.

[1] http://adapton.org/

[2] https://dcharkes.github.io/

Re: Skip – A programming language to skip the things you have already computed

#92

Post by thom »

thom wrote: post #91

Incremental computation is a fascinating area of research, I've followed Matthew Hammer's work on Adapton[1] for a while, as well as Daco Harkes work on IceDust[2]. The issues are obviously quite deep (i.e. how do can you change a single value and recalculate quicksort without the effects of the change fanning out massively?) whether you're targeting existing languages and runtimes or building from scratch. I would v…

Of course if that already exists then I'd love to hear about it! There seem to be all sorts of hybrid stream/batch systems but none with quite this focus. Effectively I want stuff that streams into a time series database in realtime, with the opportunity to edit or delete events later. On top of that there would be a processing pipeline with familiar streaming paradigms (transforming data, windowing data arbitrarily, aggregating on top of that) with the additional magic that the necessary events/windows/calculations re-fire if some underlying data changes. All in a magically efficient and transparent way.

Re: Skip – A programming language to skip the things you have already computed

#93

Post by hobofan »

qwerty456127 wrote: post #89
hobofan wrote: post #85

Earlier quoted context omitted.

There was Axum[0], a research language by Microsoft. [0]: https://en.wikipedia.org/wiki/Axum_(programming_language)

Unfortunately, it has been discontinued long ago almost immediately after being introduced. And for the .Net ecosystem this means it can hardly be used nowadays as it is probably incompatible with modern libraries.

I used it for my senior project in high school (or the German equivalent of that), to write a lattice gas cellular automaton. One day after deciding on my topic, it was announced by Microsoft that they would discontinue it.

Even back then it was very buggy, and in a normal size program, you would certainly run into some bugs. Sadly Microsoft has taken most resources regarding Axum down some while after discontinuing it, so if someone would want to try it out they would run into a lot of dead links.

Based on my fading memories (but take this with a grain of salt, since I was much less experienced back then), I would say that Axum and it's actor-focused model provided not enough upsides to warrant it being its own programming languages. I had better experiences using libraries like Akka in JVM languages, or Actix in Rust.

Re: Skip – A programming language to skip the things you have already computed

#94

Post by the_duke »

dgreensp wrote: post #32
jverlaguet wrote: post #24

Earlier quoted context omitted.

Skip is not meant to be a system programming language. So the comparison with Rust for example is difficult. > "I see a lot of C++ influence" I don't see what gave you that impression. Perhaps the syntax? But I don't think it looks more like C++ than Java or C#. > Why would I choose to invest in your language over one of these other ones that has more momentum? There are several reasons, but of course I am biased: 1-…

I can see why someone who stumbles on Skip would classify it as "systems-ish": General-purpose statically-typed language that compiles to native code with an emphasis on being fast with predictable performance. Go is thought of as a systems language, and it is a garbage-collected language meant for writing servers! How would you describe when to use Skip? It looks really interesting.

I don't know anyone who considers Go a systems language. That wording was also removed from the Go website a long time ago.

The GC and the heavy weight runtime make it unsuitable in this space.

Skip seems very similar to Swift and Java, with inspiration from Rust wrt mutability.

Re: Skip – A programming language to skip the things you have already computed

#95

Post by taneq »

qwerty456127 wrote: post #51
taneq wrote: post #49

Earlier quoted context omitted.

> and run as expected as soon as they get compiled successfully If a non-trivial program runs as expected on the first try I get really suspicious.

So did I, the first Scala experience was shockingly pleasant (I indeed "couldn't believe my eyes" and literally thought "that's so weird, it probably works just so wrong that it results in an illusion of working right", it felt like magic and this way frightening): it either fails at compile time (throwing very informative error messages some of which can be unintuitive and seem weird as long as you don't know what d…

The hardest part of software development is figuring out exactly what you mean it to do, though. Making a program that runs without crashing and does something is relatively easy.

Re: Skip – A programming language to skip the things you have already computed

#97

Post by int_19h »

mrgriffin wrote: post #90
int_19h wrote: post #81

Earlier quoted context omitted.

I think integrating this sort of thing into the type system is long overdue. Ideally, in a OO language, what I want is to be able author a single class, but then "overload on immutable", so to speak. So there are some class members that are shared for both implementations, and then there are some that are only there when the object is mutable, or is immutable (in the most extreme case, there are no shared data member…

I might be misremembering, but isn't overloading on immutability what C++ does with the const keyword on member functions? e.g. struct Foo { A bar(); // If the instance is non-const. A bar() const; // If the instance is const. }; Of course this is C++ so it's up to you to make sure that all the types contained in your object are also immutable when they're const-ed (e.g. avoid pointers). And the mutable keyword is a…

Const pointers are not a problem - a pointer is a value, what it points to is a whole different thing altogether, and you can mark either or both as "const" as needed. That part is done right.

As far as changing representation, that's exactly what I meant by "class members" (and why it doesn't say "class methods"). An immutable map should have a different internal representation from a mutable map, for example, to permit for efficient copy-with-update. But logically, they're both maps. C++ punts on this problem - a map is a map, and the only thing that "const" does is make its data immutable, which in practice means that immutable containers aren't used much because there's no way to do copy-with-update (e.g. given a const std::list, you can't efficiently create a new const std::list with an item prepended, like you can in Lisp or ML - you have to create a whole const std::list, copying all items from the original one).

Languages that tried to tackle this basically just came up with a pattern where you just write two completely different classes, that have a common pattern in the name by convention (like Map and ImmutableMap), and implement a common interface/trait. But there's no link between those two classes other than convention, and the common interface has to be authored manually, even though it can be derived entirely from the two classes in practice.

So, what I'm proposing is basically a better way to write Map and ImmutableMap in a way that would 1) make it clear that those are semantically related, even though they're distinct in implementation, and 2) automatically derive the common trait. So when I write a function, I can say "takes a Map", and that's a trait that covers both "mutable Map" and "immutable Map" - automatically!

Re: Skip – A programming language to skip the things you have already computed

#98

Post by zackmorris »

Serow225 wrote: post #84
zackmorris wrote: post #67

Earlier quoted context omitted.

Thank you, I feel like Skip is very close to the language I've always wanted. Its MVCC looks very similar to Clojure's, I'm having trouble finding a good link about it: https://sw1nn.com/blog/2012/04/11/clojure-stm-what-why-how/ The gist of it is that instead of tracking multiple separate locations in memory for values, the software transactional memory (STM) references data by value. So if you assign the value {a: 4…

FWIW, MATLAB does a lot of this kind of optimization in its internal implementation of the core datatypes. There are a lot of smart compiler/JIT people that have worked hard on it over the years to optimize the internal guts despite the naive design of the language due to its age and some poor design decisions made ages ago that cannot be changed due to extreme backwards compatibility requirements. Kind of like javas…

Thank you, I didn't know that MATLAB did that. I'm a huge fan of it and hope that someday other matrix programming frameworks like TensorFlow, OpenCL and CUDA move to a similar general-purpose syntax that frees the developer from having to deal with the boilerplate associated with old OpenGL-style buffer representations.

Re: Skip – A programming language to skip the things you have already computed

#99

Post by frankmcsherry »

thom wrote: post #92
thom wrote: post #91

Incremental computation is a fascinating area of research, I've followed Matthew Hammer's work on Adapton[1] for a while, as well as Daco Harkes work on IceDust[2]. The issues are obviously quite deep (i.e. how do can you change a single value and recalculate quicksort without the effects of the change fanning out massively?) whether you're targeting existing languages and runtimes or building from scratch. I would v…

Of course if that already exists then I'd love to hear about it! There seem to be all sorts of hybrid stream/batch systems but none with quite this focus. Effectively I want stuff that streams into a time series database in realtime, with the opportunity to edit or delete events later. On top of that there would be a processing pipeline with familiar streaming paradigms (transforming data, windowing data arbitrarily,…

Shameless plug for something that does all of these things (perhaps not exactly as you want, but ..)

https://github.com/frankmcsherry/differential-dataflow

Re: Skip – A programming language to skip the things you have already computed

#100

Post by thom »

frankmcsherry wrote: post #99
thom wrote: post #92

Earlier quoted context omitted.

Of course if that already exists then I'd love to hear about it! There seem to be all sorts of hybrid stream/batch systems but none with quite this focus. Effectively I want stuff that streams into a time series database in realtime, with the opportunity to edit or delete events later. On top of that there would be a processing pipeline with familiar streaming paradigms (transforming data, windowing data arbitrarily,…

Shameless plug for something that does all of these things (perhaps not exactly as you want, but ..) https://github.com/frankmcsherry/differential-dataflow

Wonderful! I feel bad that I didn't follow the work after hearing about Naiad originally. Where might I go to read more about the fault tolerance story - I note that's an ongoing area of research in the README. Also the link to the Kafka adapter is dead, is that work ongoing or does the story end at the capture/replay API? (Happy to bombard you with email if that's any more manageable for you!)

Reading the 'when not to use Timely Dataflow' section on sorting, are there any ideas from Nominal Adapton that are relevant in this distributed setting, when we're talking about small updates or inserts?

Post reply on HN