Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Sem: New primitive for code understanding – not LSPs, but entities on top of Git

ataraxy-labs.github.io

Post reply on HN
Print view View original

31–40 of 67 posts

Re: Sem: New primitive for code understanding – not LSPs, but entities on top of Git

#31

Post by Brian_K_White »

bendjejdjdh wrote: post #21
rohanucla wrote: post #18

Earlier quoted context omitted.

I am sorry, should have put up a warning there, but You can do sem unsetup, if you go to the github, you will understand more about the way to reverse it.

tone deaf comment. "read the docs to undo it" is user hostile.

"Ah yeah, you're right. I apologize for that. Here is what to do. I'll update the page."

What an asshole! Plus the uninstall steps were completely inconsiderate single 2 word command. Outrageous.

I can't even think of a better possible response.

Re: Sem: New primitive for code understanding – not LSPs, but entities on top of Git

#32

Post by jiggunjer »

Another potential use case: This may help jujutsu auto split a large revision into small orthogonal revs.

Sometimes agent makes a monolithic commit and it's a lot of work to manually split code you didn't write. After such an auto split I can manually squash related revs into feature/ticket level.

Re: Sem: New primitive for code understanding – not LSPs, but entities on top of Git

#33

Post by rohanucla »

hankbond wrote: post #27
rohanucla wrote: post #11

Earlier quoted context omitted.

This is a really interesting direction, you're essentially talking about data flow or taint analysis, where you track how a value propagates through copies and transformations rather than just following call edges. Honestly pure static analysis gets you partway there but it hits real limits once you run into dynamic dispatch, runtime branching, or serialization boundaries where data gets written somewhere and read ba…

https://en.wikipedia.org/wiki/Taint_checking I'm sorry for distracting from your engaging and thoughtful reply but I can't help but giggle at the name of this concept.

haha definitely!

Re: Sem: New primitive for code understanding – not LSPs, but entities on top of Git

#35

Post by rohanucla »

cpard wrote: post #34

This is really neat. I’m working on something similar but for data artifacts not just code. It’s very encouraging to see that this kind of tooling helps both humans and models, that was what made me starting to work on that.

Thanks! The data artifacts angle is really interesting. in some ways the problem is even harder there because data pipelines have less explicit structure than code, I guess.

Re: Sem: New primitive for code understanding – not LSPs, but entities on top of Git

#36

Post by rohanucla »

jiggunjer wrote: post #32

Another potential use case: This may help jujutsu auto split a large revision into small orthogonal revs. Sometimes agent makes a monolithic commit and it's a lot of work to manually split code you didn't write. After such an auto split I can manually squash related revs into feature/ticket level.

That's a really compelling use case actually

Re: Sem: New primitive for code understanding – not LSPs, but entities on top of Git

#37

Post by gwerbin »

rohanucla wrote: post #35
cpard wrote: post #34

This is really neat. I’m working on something similar but for data artifacts not just code. It’s very encouraging to see that this kind of tooling helps both humans and models, that was what made me starting to work on that.

Thanks! The data artifacts angle is really interesting. in some ways the problem is even harder there because data pipelines have less explicit structure than code, I guess.

The artifacts themselves have more structure, but diffing is hard because of size: what exactly do you show in the different? Row-level? Summary statistics? How do you keep it from getting slow on bigger datasets?

Then there are plots saved as images which have basically no structure at all exposed.

Re: Sem: New primitive for code understanding – not LSPs, but entities on top of Git

#38

Post by gwerbin »

cpard wrote: post #34

This is really neat. I’m working on something similar but for data artifacts not just code. It’s very encouraging to see that this kind of tooling helps both humans and models, that was what made me starting to work on that.

There is still no good "data diff" tool that I can run on, say, a big pile of CSV or Parquet. Something with DVC integration would be especially welcome.

Re: Sem: New primitive for code understanding – not LSPs, but entities on top of Git

#39

Post by rohanucla »

throw1234567891 wrote: post #20
rohanucla wrote: post #3

Earlier quoted context omitted.

It can do that, but that's a small slice of what it does. sem parses your codebase into entities (functions, classes, methods) and builds a dependency graph across files. So instead of line level analysis the whole granularity of seeing changes and tracking thing shifts to entities. It helps in attention mapping of your agent and lets you track the changes faster. LSPs have been doing it for quite long but using tree…

The tool looks great! Thanks for sharing.

Appreciate it!

Re: Sem: New primitive for code understanding – not LSPs, but entities on top of Git

#40

Post by appplication »

gwerbin wrote: post #38
cpard wrote: post #34

This is really neat. I’m working on something similar but for data artifacts not just code. It’s very encouraging to see that this kind of tooling helps both humans and models, that was what made me starting to work on that.

There is still no good "data diff" tool that I can run on, say, a big pile of CSV or Parquet. Something with DVC integration would be especially welcome.

I would imagine because at scales where most folks use parquet files, you’re generally no longer really thinking in terms of individual diffs to your data (and also does imply some level of batch processing, vs e.g. a DB).

We have some custom data diff tools at my ultracorp that provide a browsable interface, but the customer tends to be more operations folk than engineers or DS etc who would be more familiar with actual version control concepts. But these work against the data store and not on something like csv or parquet.

Post reply on HN