Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Glean – System for collecting, deriving and querying facts about source code

glean.software

Post reply on HN
Print view View original

71–80 of 85 posts

Re: Glean – System for collecting, deriving and querying facts about source code

#71

Post by fragmede »

Syzygies wrote: post #67
WastingMyTime89 wrote: post #58

Earlier quoted context omitted.

> It seems like a lot of hoops to jump through when Visual Studio (and Visual Studio Code) can index a very large codebase in a few seconds. I think you are not thinking large enough. An IDE absolutely can not index a very large codebase and allow users to make complex queries on it. Think multiple millions lines of code here. The use case is closer to "find me all the variables of this type or a type derived from it…

There's large, and there's scope. I use VSCode to dabble in dozens of projects across a dozen languages at a time, often coming back to fix things after years. VSCode is great at telling me what I did in the current project, but I can't remember library calls or even syntax without looking at something I wrote before. My efficiency is perhaps 50% at recalling where to look; a tool that kept my entire corpus at my fin…

If you've not had to deal with a codebase that takes VSCode longer than a few minutes to index, then you're probably outside their initial target market. If you've not had to setup a hosted code search tool (eg livegrep https://github.com/livegrep/livegrep ) because there's just too much code, you've been lucky. If your projects can be scoped, and not pull in code from dozens of libraries, across dozens of teams, many of which are on different continents, you're doing a better job of organizing code than I've been able to manage.

Re: Glean – System for collecting, deriving and querying facts about source code

#72

Post by X6S1x6Okd1st »

minxomat wrote: post #36
scns wrote: post #32

Earlier quoted context omitted.

Oh wow, mindblowing stuff. Glad to see tech like this being open sourced, fuels the imagination about possible future scenarios. Do you use it on the Linux Kernel?

A comparable, powerful system (CodeQL) was used recently on the kernel[1] and Chrome. You can learn more about it here: https://codeql.github.com/docs/codeql-overview/about-codeql/ (disclosure: I work at GH on CQL) [1] https://pwning.systems/posts/sequoia-variant-analysis/

Oof on the terms & conditions:

https://securitylab.github.com/tools/codeql/license/

Re: Glean – System for collecting, deriving and querying facts about source code

#75

Post by tclancy »

simonmar wrote: post #68
tclancy wrote: post #65

Getting a 401 when trying `docker pull ghcr.io/facebookincubator/glean/demo:latest` -- is that true for anyone else?

Sorry about that, the package was still set to private. Try again now?

All set, thanks!

Re: Glean – System for collecting, deriving and querying facts about source code

#76

Post by mrazomor »

simonmar wrote: post #31
coderdd wrote: post #21

Great to see this space moving! Any pointers on diff vs Kythe? Kythe has a mostly fixed schema, for one. One of the pain points using Kythe is wiring up the indexer to the build system. Would Glean indexers be easier to wire up for the common cases? Other is the index post-processing, which is not very scalable in the open source version (due to go-beam having rough Flunk support, for example). Third, how does it lin…

Kythe has one schema, whereas with Glean each language has its own schema with arbitrary amounts of language-specific detail. You can get a language-agnostic view by defining an abstraction layer as a schema. Our current (work in progress) language-agnostic layer is called "codemarkup" https://github.com/facebookincubator/Glean/blob/main/glean/s... For wiring up the indexer, there are various methods, it tends to dep…

In case using Kythe was an option, what was the rationale for not using it?

One major limitation of Kythe is handling different versions. For example, Kythe can produce a well connected index of Stackage, but a Hackage would have many holes (not all references would be found, as the unique reference name needs the library version). How Glean handles different library versions?

EDIT: the language agnostic view is already mentioned.

Re: Glean – System for collecting, deriving and querying facts about source code

#77

Post by jhayward »

fnord77 wrote: post #66
gwbas1c wrote: post #48

Earlier quoted context omitted.

I'm really struggling to understand what Glean does, and why I would use it. Most important: Your landing page should quickly show what Glean does that a typical IDE (Visual Studio, Visual Studio Code, Eclipse, ect, does.) Specifically, things like "Go to definition," and tab completion have been in industry-leading IDEs for at least 20 years. What's novel about Glean? It seems like a lot of hoops to jump through whe…

“Go to definition” has been around even longer, since at least the early 90s

I don't recall which version of Emacs first had "go to definition", but it was well before the 90's.

Re: Glean – System for collecting, deriving and querying facts about source code

#79

Post by rad_gruchalski »

gwbas1c wrote: post #48
dons wrote: post #5

We use this to power things like find-references or jump-to-def, "symbol search" and autocomplete, or more complicated code queries and analysis (even across languages). Imagine rich LSPs without a local checkout, web-based code queries, or seeding fuzzers and static analyzers with entry points in code. Our focus has been on very large scale, multi-language code indexing, and then low latency (e.g. hundreds of micros…

I'm really struggling to understand what Glean does, and why I would use it. Most important: Your landing page should quickly show what Glean does that a typical IDE (Visual Studio, Visual Studio Code, Eclipse, ect, does.) Specifically, things like "Go to definition," and tab completion have been in industry-leading IDEs for at least 20 years. What's novel about Glean? It seems like a lot of hoops to jump through whe…

Imagine that you pulled in all your dependencies in different languages in source + windows source and visual studio source. Now you want to click around that source. This is what this tool is for.

Re: Glean – System for collecting, deriving and querying facts about source code

#80

Post by rad_gruchalski »

jhayward wrote: post #77
fnord77 wrote: post #66

Earlier quoted context omitted.

“Go to definition” has been around even longer, since at least the early 90s

I don't recall which version of Emacs first had "go to definition", but it was well before the 90's.

How easy was it in 90’s to have a go to definition over, say, 10% of today’s maven central?
Post reply on HN