Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Table Oriented Programming (2002)

web.archive.org

Post reply on HN
Print view View original

41–50 of 91 posts

Re: Table Oriented Programming (2002)

#41

Post by xwolfi »

gpderetta wrote: post #40
xwolfi wrote: post #36

Earlier quoted context omitted.

Isn't Q the language, and Kx the company ? I have to use it at work, helping quants try to actually maintain production code rather than just vomit horrible one-time scripts they patch together in an endless stream of layers on top of layers. This thing should be banned. Everything is one-letter, it's impossible to Google, it takes the opposite decision of every imaginable convention, I have yet to see someone who ca…

You are right. As far as I understand Q is K plus the additional sql-like syntax. And yes, it is nigh-unreadable. I have yet to learn it, but I think there is value in succinctness at least for throwaway scripts.

Yeah but there s low value in throwaway, and it's not succint, it's 1-letter. ONE lol, for every imaginable labguage keyword.

For instance to parse a binary encoded dictionary from a table column: -9!'columnName(I remember it because it s the first time I spent a day on something so useless yet so indispensible yet so fucked up). I challenge you to google it.

He likes succint hihihi.

Re: Table Oriented Programming (2002)

#42

Post by DenisM »

jasode wrote: post #5

I remember reading this essay when it first came out. To try and reword it using modern terms: The author wishes that programming languages had database persistence capabilities as 1st-class built-in syntax instead of cumbersome bolted-on API functions. Examples where database syntax (i.e. SQL syntax) is 1st-class without noisy syntax of function calls, without command strings in quotes, etc : - business languages li…

How about LINQ, in particular LINQ2SQL?

Re: Table Oriented Programming (2002)

#43

Post by chrisaycock »

xwolfi wrote: post #32
chrisaycock wrote: post #26

Earlier quoted context omitted.

Ah, I see what you're referring to. The hardest thing is the load() function, particularly in the REPL. It looks dynamic, but is actually static. Pulling off this slight-of-hand requires both type providers and automatic compile-time function evaluation on arbitrary expressions. F# is the only other language I know of that has type providers. They invented it. As for CTFE, languages like Zig and D require the user to…

I don't think I get it. I do a lot of pandas in a bank so I recognize your dataframes for what they are, but what advantage do you have over python+pandas ? I hate Python (I'm a Java dev helping Quants), but it's that or KDB, and I think I could murder the creator of KDB :D And I have to admit Pandas is instinctive, Python is easy enough to extend, what are you doing that's so important you made a language for it ?

Empirical is statically typed. Python and q/kdb+ are dynamically typed.

I spent years using those products in finance. I would set-up a simulation that would crash after four hours because of a misspelled column name. Empirical prevents that by refusing to run a script that has a type error or unresolved identifier. No more crashed overnight sims!

Re: Table Oriented Programming (2002)

#44

Post by CRConrad »

kragen wrote: post #2

I'm glad this got posted! I wanted to reread this a couple of years ago and couldn't find it. Any idea what happened to TopMind?

My history with ****e *****s goes way back... we must have had more than a decade of pro-vs-contra-OOP flame wars on various Web fora, starting over twenty years ago; but in the last ten or so I haven't heard (directly) from him.

In the mean time, I have softened my stance and can admit that traditional inheritance-based OOP may not be ultimate panacea, but I doubt he has softened his anti-OOP stance at all. :-)

He is / was present on at least Slashdot and, I noticed, the original (now archived, i.e. read-only) C2 Wiki, and probably a few others I'm forgetting right now, under the names "Tablizer", "TopMind" (or sometimes, IIRC, just "Top".)

Re: Table Oriented Programming (2002)

#45

Post by iamwil »

chrisaycock wrote: post #43
xwolfi wrote: post #32

Earlier quoted context omitted.

I don't think I get it. I do a lot of pandas in a bank so I recognize your dataframes for what they are, but what advantage do you have over python+pandas ? I hate Python (I'm a Java dev helping Quants), but it's that or KDB, and I think I could murder the creator of KDB :D And I have to admit Pandas is instinctive, Python is easy enough to extend, what are you doing that's so important you made a language for it ?

Empirical is statically typed. Python and q/kdb+ are dynamically typed. I spent years using those products in finance. I would set-up a simulation that would crash after four hours because of a misspelled column name. Empirical prevents that by refusing to run a script that has a type error or unresolved identifier. No more crashed overnight sims!

You should say this under the question of how it's different than Julia.

It's not enough to say it's statically typed, since not everyone is convinced of the benefits based on the context they're coming from.

I just saw a talk by Rich Hickey about Clojure, and he eschews static typing, since he thinks of it as a coupling in a language. And based on the types of programs he writes and runs, he hasn't seen a benefit.

So I think when you're specific about what statically typed buys you in the context of the job Empirical does for you, I think it's more convincing.

Re: Table Oriented Programming (2002)

#46

Post by marcle »

jasode wrote: post #5

I remember reading this essay when it first came out. To try and reword it using modern terms: The author wishes that programming languages had database persistence capabilities as 1st-class built-in syntax instead of cumbersome bolted-on API functions. Examples where database syntax (i.e. SQL syntax) is 1st-class without noisy syntax of function calls, without command strings in quotes, etc : - business languages li…

Another relevant software category is the statistical analysis languages, including SAS, Stata and SPSS.

Old-school SAS included only two data types (floats and character strings), but allowed for SQL and sequential data-steps to live together. Persistence was baked in. The floats could be used to represent dates, datetimes and other formats. I particularly appreciated being able to use macros to define a data-step view to split the follow-up for an individual from a table. Such a view could then be collapsed using SQL. More recently, R tools such as dplyr have brought together data-frames and relational operations. However, I miss the sequential coding in SAS, using macros as higher-level tools to define the logic, including corner cases.

For strictly typed records, I have always wanted to spend more time with SML# [0] this allows for record updating, with close ties to SQL -- an under-appreciated version of SML.

[0] https://github.com/smlsharp/smlsharp

Re: Table Oriented Programming (2002)

#47

Post by chrisaycock »

mdcfrancis wrote: post #35
chrisaycock wrote: post #26

Earlier quoted context omitted.

Ah, I see what you're referring to. The hardest thing is the load() function, particularly in the REPL. It looks dynamic, but is actually static. Pulling off this slight-of-hand requires both type providers and automatic compile-time function evaluation on arbitrary expressions. F# is the only other language I know of that has type providers. They invented it. As for CTFE, languages like Zig and D require the user to…

I've written similar in Julia, you can see the record type used in https://www.juliapackages.com/p/namedtuples . The full library, not in the open source, uses this type for time series analysis. It's all type safe and allowed expressions such as x = vwap( ts, 5) - l1( vwap( ts, 5)) through to a time moving PCA. Julia makes writing this sort of thing short and quick. The total impl was only a thousand lines or so of…

I checked your website; do you have an example of how to load data from a file into NamedTuples? Specifically, can NamedTuples infer type from an external source?

Also, do you have an example of what a displayed table looks like? Julia has a DataFrames package that can display a table. I am curious to know how your time-series library displays a table.

Re: Table Oriented Programming (2002)

#48

Post by Animats »

Oh, that kind of table. I was expecting decision tables.[1]

"Smart contracts" for Etherium should have been decision tables. But no, they had to make it Turing-complete. A good thing about decision tables is that there's a finite and small number of cases, so they can be exhaustively tested. Also, they're readable. That's what you want for contracts. Not Solidity programs, which are expensively insecure.

[1] https://en.wikipedia.org/wiki/Decision_table

Re: Table Oriented Programming (2002)

#49

Post by BeFlatXIII »

jasode wrote: post #5

I remember reading this essay when it first came out. To try and reword it using modern terms: The author wishes that programming languages had database persistence capabilities as 1st-class built-in syntax instead of cumbersome bolted-on API functions. Examples where database syntax (i.e. SQL syntax) is 1st-class without noisy syntax of function calls, without command strings in quotes, etc : - business languages li…

Although it's not a relational database, MUMPS is another example of a language where there is nothing special whatsoever about manipulating the database compared to manipulating the same data but stored in a variable in RAM.

Re: Table Oriented Programming (2002)

#50

Post by mdcfrancis »

chrisaycock wrote: post #47
mdcfrancis wrote: post #35

Earlier quoted context omitted.

I've written similar in Julia, you can see the record type used in https://www.juliapackages.com/p/namedtuples . The full library, not in the open source, uses this type for time series analysis. It's all type safe and allowed expressions such as x = vwap( ts, 5) - l1( vwap( ts, 5)) through to a time moving PCA. Julia makes writing this sort of thing short and quick. The total impl was only a thousand lines or so of…

I checked your website; do you have an example of how to load data from a file into NamedTuples? Specifically, can NamedTuples infer type from an external source? Also, do you have an example of what a displayed table looks like? Julia has a DataFrames package that can display a table. I am curious to know how your time-series library displays a table.

unfortunately, I don't have access to that code anymore, I wrote a number of loaders for different data set types including CSV. The time series were all modeled as forward iterating stream of tuples, so there is no specific table abstraction. There is an implicit assumption that the stream is ordered by the join key, in a time series this being the timestamp, though nothing in the implementation enforced that.

Joins are always n-way merge joins, so you can write something like y = 2x^2 - 3z + c and fold that into a single streaming operation y = f( x, z, c ) where y, x, z and c are time streams.

When rendered to screen they looked very similar to your examples. With plugins in the IDE you could directly plot and array of time series as a chart.

Since the time I wrote NamedTuples the Julia core team folded the functionality into the core of Julia https://docs.julialang.org/en/v1/manual/types/#Named-Tuple-T.... This is the core of https://juliadb.org/ all credit to the Julia core team

Post reply on HN