Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Glean – System for collecting, deriving and querying facts about source code

glean.software

Post reply on HN
Print view View original

11–20 of 85 posts

Re: Glean – System for collecting, deriving and querying facts about source code

#11

Post by simonmar »

the_duke wrote: post #8
dons wrote: post #5

We use this to power things like find-references or jump-to-def, "symbol search" and autocomplete, or more complicated code queries and analysis (even across languages). Imagine rich LSPs without a local checkout, web-based code queries, or seeding fuzzers and static analyzers with entry points in code. Our focus has been on very large scale, multi-language code indexing, and then low latency (e.g. hundreds of micros…

This is really cool. Seems like there are only indexers for Flow and Hack though. Will there be more indexers built by Facebook, or will it rely on community contributions?

There will be more indexers: we have Python, C++/Objective C, Rust, Java and Haskell. It's just a case of getting them ready to open source. You can see the schemas for most of these already in the repo: https://github.com/facebookincubator/Glean/tree/main/glean/s...

Re: Glean – System for collecting, deriving and querying facts about source code

#12

Post by soonnow »

dons wrote: post #5

We use this to power things like find-references or jump-to-def, "symbol search" and autocomplete, or more complicated code queries and analysis (even across languages). Imagine rich LSPs without a local checkout, web-based code queries, or seeding fuzzers and static analyzers with entry points in code. Our focus has been on very large scale, multi-language code indexing, and then low latency (e.g. hundreds of micros…

Does that mean you are using the shell or how is it used to enable these functionalities?

Re: Glean – System for collecting, deriving and querying facts about source code

#14

Post by soonnow »

lazamar wrote: post #7
soonnow wrote: post #4

I had a look at the site and it seems to be parsing source code in multiple languages and storing the parsed "syntax trees" into a database for querying. I would love to know what the usecase for this tool is aside from maybe being a source for presentations? (We have 5 million if statements). How can this be used to improve code quality or any other aspect of the code lifecycle? Or is it solving problems in a comple…

Glean is focused on storing and querying data about the code. The idea is that you have your own program to collect that data, then you use Glean to store that compactly and to have snappy queries. You would create entries like "this is a declaration of X", "this is a use of X". Then you can query things like "give me all uses of X" in sub-millisecond time. You hook that up to an LSP server then you get almost zero-c…

> For very large or complex codebases, where it is just too expensive or too complicated to calculate this information locally a system like this becomes very useful.

Thanks I guess I get it now. But to enable this functionality you'd need to have some form of frontend or integration into the existing build lifecycle?

Or IDE integration I guess.

Re: Glean – System for collecting, deriving and querying facts about source code

#16

Post by dons »

soonnow wrote: post #12
dons wrote: post #5

We use this to power things like find-references or jump-to-def, "symbol search" and autocomplete, or more complicated code queries and analysis (even across languages). Imagine rich LSPs without a local checkout, web-based code queries, or seeding fuzzers and static analyzers with entry points in code. Our focus has been on very large scale, multi-language code indexing, and then low latency (e.g. hundreds of micros…

Does that mean you are using the shell or how is it used to enable these functionalities?

Most clients hit the Glean server via the network (thrift/JSON) and then mostly via language bindings to the Glean query language, Angle. The shell is more for debugging/exploration.

Imagine an IDE plugin that queries Glean over the network for symbol information about the current file, then shows that on hover. That sort of thing.

Re: Glean – System for collecting, deriving and querying facts about source code

#17

Post by progval »

dons wrote: post #5

We use this to power things like find-references or jump-to-def, "symbol search" and autocomplete, or more complicated code queries and analysis (even across languages). Imagine rich LSPs without a local checkout, web-based code queries, or seeding fuzzers and static analyzers with entry points in code. Our focus has been on very large scale, multi-language code indexing, and then low latency (e.g. hundreds of micros…

How would it perform for, say, 500TB of source code?

And what would be the disk and memory requirements for this? Could they be distributed across a handful of servers?

Re: Glean – System for collecting, deriving and querying facts about source code

#18

Post by soonnow »

dons wrote: post #16
soonnow wrote: post #12

Earlier quoted context omitted.

Does that mean you are using the shell or how is it used to enable these functionalities?

Most clients hit the Glean server via the network (thrift/JSON) and then mostly via language bindings to the Glean query language, Angle. The shell is more for debugging/exploration. Imagine an IDE plugin that queries Glean over the network for symbol information about the current file, then shows that on hover. That sort of thing.

Alright gotcha. Thanks for the clarification.

Re: Glean – System for collecting, deriving and querying facts about source code

#19

Post by pdpi »

dons wrote: post #5

We use this to power things like find-references or jump-to-def, "symbol search" and autocomplete, or more complicated code queries and analysis (even across languages). Imagine rich LSPs without a local checkout, web-based code queries, or seeding fuzzers and static analyzers with entry points in code. Our focus has been on very large scale, multi-language code indexing, and then low latency (e.g. hundreds of micros…

Been away from Fb for a few years. How does this relate to tbgs?
Post reply on HN