21–30 of 134 posts
Is there any work or planned work around enterprise authentication and access? For instance, indexing Sharepoint in such a way where a user of Danswer isn't exposed to sharepoint information they wouldn't otherwise have access to?
Are you using anything like Llamaindex internally or did you write it from scratch without the assistance form a helping wrapper like this?
We couldn't use the more involved pipelines because we needed significant custom logic to enforce permissions, filters (like time filter, source filter, document-set filters), and other complexities. At that point, it's easier to write from scratch rather than conform to expectations of these third party libraries.
I believe there are a variety of projects focused on that problem but would be good to latch onto one that handles and integrates externalities.
Speaking of which I did not see sentry in the list of connections or mentioned in an issue. Any plans there?
Congrats on the launch! Does it search within pdf files?
Is this something that you would find a lot of value in or is simple text processing of PDFs sufficient?
Are you using anything like Llamaindex internally or did you write it from scratch without the assistance form a helping wrapper like this?
We use LlamaIndex very sparingly. Specifically the context aware document chunking functionality is via LlamaIndex. We couldn't use the more involved pipelines because we needed significant custom logic to enforce permissions, filters (like time filter, source filter, document-set filters), and other complexities. At that point, it's easier to write from scratch rather than conform to expectations of these third part…
Another question. If I host this publicly for people that I work with on their data, how can I make sure that it's only them that can access the service? Do you have any form of auth?
We also do OIDC and SAML to integration with Identity Providers (IdPs) like Okta but that's part of the paid features. Ahhh please don't hate us!
Earlier quoted context omitted.
most of the files I have, I'm most interested in finding graphs and then updating relevant data. it looks like the best way to do that is understand the ooxml format in xlsx. it's all fairly easy to understand.
Ya, parsing the file is generally not bad at all. The problem comes with the fact that LLMs are notoriously bad with numbers and formatted data. So the current approach of passing relevant information to the LLM and asking it to generate answers will produce misleading information when larger tables are passed in. By asking the LLM to generate a formula though, it doesn't actually need to do any number crunching of i…
Your methodology is nice. Is there any work or planned work around enterprise authentication and access? For instance, indexing Sharepoint in such a way where a user of Danswer isn't exposed to sharepoint information they wouldn't otherwise have access to?
Yes, there are several options for user authentication (Basic Auth, Google OAuth, OIDC, SAML).
Currently the RBAC is managed via the Danswer and this controls who has access to which documents (it's done at the connector level as it would be untenable to assign access to documents individually).
We're also working on automatically sync-ing permissions from the sources. Basically seeing which emails have access to each doc and mapping it to the Danswer users.