Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Table Oriented Programming (2002)

web.archive.org

Post reply on HN
Print view View original

31–40 of 91 posts

Re: Table Oriented Programming (2002)

#31

Post by naasking »

marcosdumay wrote: post #21
naasking wrote: post #17

Earlier quoted context omitted.

I'm no longer convinced of the need for row or record polymorphism. It encourages passing around types that have no clear domain or purpose, so I think it inhibits understanding in general. Do you have any examples where it's indispensable?

Nothing is indispensable as long as you have a Turing complete language. That is a really bad mindset to use. Anyway, are you complaining that the types are abstract? (That is as bad a complaint as it sounds.) Or do you have something different in mind?

You're taking indispensable too literally. If you have to commonly write 1,000 lines of code without a feature, but the feature permits you to to reduce this to 1 line of code, I'd consider that to be pretty indispensable.

Where the indispensable line is is debatable, hence my request for an example.

Re: Table Oriented Programming (2002)

#32

Post by xwolfi »

chrisaycock wrote: post #26
rscho wrote: post #20

Earlier quoted context omitted.

On the website you linked: "Embedding Dataframes into an existing language would not be possible." I don't think it would be an issue for languages with good metaprogramming facilities.

Ah, I see what you're referring to. The hardest thing is the load() function, particularly in the REPL. It looks dynamic, but is actually static. Pulling off this slight-of-hand requires both type providers and automatic compile-time function evaluation on arbitrary expressions. F# is the only other language I know of that has type providers. They invented it. As for CTFE, languages like Zig and D require the user to…

I don't think I get it. I do a lot of pandas in a bank so I recognize your dataframes for what they are, but what advantage do you have over python+pandas ?

I hate Python (I'm a Java dev helping Quants), but it's that or KDB, and I think I could murder the creator of KDB :D And I have to admit Pandas is instinctive, Python is easy enough to extend, what are you doing that's so important you made a language for it ?

Re: Table Oriented Programming (2002)

#33

Post by kerblang »

This idea of "the database should be invisible!" abstraction was widely pursued back in the 90's when people were still obsessed with "the network should be invisible!" and Remote Procedure Calls (RPC). A lot of ORM's still reflect this obsession, and some programmers still get angry that they should have to deal with "this low-level SQL nonsense!"

Attempts to make I/O invisible failed and failed and failed again, and continue failing and failing again because it turns out that I/O is incredibly fundamental and not something you can just wave off as "low-level details". A networked database is a massive abstraction in its own right, and if invisible I/O is a doomed abstraction, forget invisible databases. Well, first go fail a few more times, then forget it, because we're not quite there yet on this one, are we...

The bigger the abstraction, the more it leaks. Sometimes you have enough headroom to go further, and sometimes you have to recognize that you've gone way too far.

Re: Table Oriented Programming (2002)

#34

Post by beaumayns »

chrisaycock wrote: post #11

I built my own table-oriented language out of frustrations I had with with time-series analysis: https://www.empirical-soft.com Empirical has statically typed Dataframes. It can infer the type of a file's contents at compile time using a ton of metaprogramming techniques. >>> let trades = load("trades.csv") >>> trades symbol timestamp price size AAPL 2019-05-01 09:30:00.578802 210.5200 780 AAPL 2019-05-01 09:30:00.58…

This is pretty cool. I've had thoughts (or dreams, more accurately :) of a language like this every time I get a runtime 'type error in q. I gotta say, I prefer q's syntax, though :)

Re: Table Oriented Programming (2002)

#35

Post by mdcfrancis »

chrisaycock wrote: post #26
rscho wrote: post #20

Earlier quoted context omitted.

On the website you linked: "Embedding Dataframes into an existing language would not be possible." I don't think it would be an issue for languages with good metaprogramming facilities.

Ah, I see what you're referring to. The hardest thing is the load() function, particularly in the REPL. It looks dynamic, but is actually static. Pulling off this slight-of-hand requires both type providers and automatic compile-time function evaluation on arbitrary expressions. F# is the only other language I know of that has type providers. They invented it. As for CTFE, languages like Zig and D require the user to…

I've written similar in Julia, you can see the record type used in https://www.juliapackages.com/p/namedtuples. The full library, not in the open source, uses this type for time series analysis. It's all type safe and allowed expressions such as x = vwap( ts, 5) - l1( vwap( ts, 5)) through to a time moving PCA. Julia makes writing this sort of thing short and quick. The total impl was only a thousand lines or so of code.

Re: Table Oriented Programming (2002)

#36

Post by xwolfi »

gpderetta wrote: post #24
jasode wrote: post #5

I remember reading this essay when it first came out. To try and reword it using modern terms: The author wishes that programming languages had database persistence capabilities as 1st-class built-in syntax instead of cumbersome bolted-on API functions. Examples where database syntax (i.e. SQL syntax) is 1st-class without noisy syntax of function calls, without command strings in quotes, etc : - business languages li…

The K language of kdb also count, but being proprietary and having a fairly impenetrable and alien syntax haven't helped it branching out of the niche where it is very successfully.

Isn't Q the language, and Kx the company ?

I have to use it at work, helping quants try to actually maintain production code rather than just vomit horrible one-time scripts they patch together in an endless stream of layers on top of layers.

This thing should be banned. Everything is one-letter, it's impossible to Google, it takes the opposite decision of every imaginable convention, I have yet to see someone who can read his own stuff 2 weeks later. Notwithstanding free tools to query force-expire every 3 months (QPad grrr) and as you said it's so closed they have to take webdevs like me to just help them maintain it all eventually.

It's risky for the bank (and we're not a small one :s), it's expensive for the programmer, and it's misery for the quants (but they all feel like geniuses spending weeks on simple stuff on kdb, so their misery is only in people looking at them wasting away their brain like that).

Re: Table Oriented Programming (2002)

#37

Post by Avshalom »

This idea (or at the least nostalgia for xBase) pops up every now and then and while it certainly isn't describing Prolog I think the idea would be a lot more interesting if the authors had enough familiarity to compare and contrast.

Re: Table Oriented Programming (2002)

#38

Post by lincpa »

zokier wrote: post #8

I'm in the opinion that tables would make a lot of sense as first-class citizens for shell environments. Lots of data typically handled in shells is inherently tabular in nature (for example the outputs of ls and ps etc) and some of the common tools also are intended for tables (awk in the forefront, but also cut and sort as examples). But in practice lot of it is currently very ad-hoc, and handles any sort of edge c…

No post body was provided.

Re: Table Oriented Programming (2002)

#39

Post by mamcx »

For people like me, that worked in FoxPro, this is the dream.

Despite the claim this kind of tools is for "basic CRUD" they could do much more, much better, precisely because can deal MUCH better with the most challenged kind of programming:

CRUD apps.

Making apps in finance, erps, bussines, etc, are far more complex and challenging than build chat apps, where the scope is MORE clear and the features, reduced.

"Simple" crud apps NEVER stay simple.

NEVER.

If you allow it, in no time you are building a mix of your owm RDBMs, programming language, API orchestation, authorization framework, inference engines, hardware interfaces and more...

then, it must run in "Windows, Linux, Mac, Android, iOS, Web, Rasperry, that computer that is only know here in this industry", "please?"... and it will chases, also, all fads, all the time.

The request/features pipeline never end. The info about what to do is sketchy at best.

The turnaround to bring results is measure in HOURS/DAYs.

So, no.

No language without this, is in fact, good for the niche.

Re: Table Oriented Programming (2002)

#40

Post by gpderetta »

xwolfi wrote: post #36
gpderetta wrote: post #24

Earlier quoted context omitted.

The K language of kdb also count, but being proprietary and having a fairly impenetrable and alien syntax haven't helped it branching out of the niche where it is very successfully.

Isn't Q the language, and Kx the company ? I have to use it at work, helping quants try to actually maintain production code rather than just vomit horrible one-time scripts they patch together in an endless stream of layers on top of layers. This thing should be banned. Everything is one-letter, it's impossible to Google, it takes the opposite decision of every imaginable convention, I have yet to see someone who ca…

You are right. As far as I understand Q is K plus the additional sql-like syntax.

And yes, it is nigh-unreadable. I have yet to learn it, but I think there is value in succinctness at least for throwaway scripts.

Post reply on HN