Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Loki: An open-source tool for fact verification

github.com

Post reply on HN
Print view View original

31–40 of 74 posts

Re: Loki: An open-source tool for fact verification

#31

Post by Xudong »

vinni2 wrote: post #14
raycat7 wrote: post #13

Earlier quoted context omitted.

regarding your last concern, I found that yuxiaw is their COO[1], so it can't be considered a copy? [1] https://www.librai.tech/team

Ok but bigger issue is there is evidence that the LLMs are not better than specialized models for fact-checking. https://arxiv.org/abs/2402.12147

Hello vinni2, thank you for mentioning the paper. However, I noticed that it hasn't gone through peer review yet. Also, the paper suggests that fine-tuning may work better than in-context learning, but that's not a problem. You can fine-tune any LLMs like GPT-3.5 for this purpose and use them with this framework. Once you have fine-tuned GPT, for example, with specific data, you'll only need to modify the model name (https://github.com/Libr-AI/OpenFactVerification/blob/8fd1da9...). I believe this approach can lead to better results than what the paper suggests.

Re: Loki: An open-source tool for fact verification

#32

Post by lta »

xyst wrote: post #23
axegon_ wrote: post #17

Overall great idea though, I'll be definitely checking it back in the future. A few things that hit me out of the box: * The idea behind using Serper is great, however it would be cool if other search engines/data sources can be used instead, ie. Kagi or some private search engine/data. Reason for the latter: there are tons of people who are sourcing all sorts of information which will not immediately show up on goog…

I fully expect some sort of enshittification of openai at some point.

That's assuming it's not done already with their mission of being open completely forgotten

Re: Loki: An open-source tool for fact verification

#33

Post by Xudong »

axegon_ wrote: post #17

Overall great idea though, I'll be definitely checking it back in the future. A few things that hit me out of the box: * The idea behind using Serper is great, however it would be cool if other search engines/data sources can be used instead, ie. Kagi or some private search engine/data. Reason for the latter: there are tons of people who are sourcing all sorts of information which will not immediately show up on goog…

Thank you for your suggestions, axegon!!! We will definitely consider them and add the features in a future version shortly.

Regarding the first version, we are currently working on enabling customized evidence retrieval, including local files. Our plan is to integrate existing tools like LlamaIndex. Any suggestion is greatly appreciated!

Regarding the second point, we have found OpenAI's JSON mode to be greatly helpful, and have optimized our prompts to fully utilize these advances. However, we agree that it would be beneficial to enable the use of other models. As promised, we will add this feature soon.

Lastly, we appreciate your suggestion and will work on improving the installation process for the next version.

Re: Loki: An open-source tool for fact verification

#34

Post by dekervin »

I have a project where I take a different approach [0] . I basically extract statements , explicit or implicit , that should be accompanied by a reference to some data but aren't and I let user find the most relevant data for those statements.

[0] https://datum.alwaysdata.net/

Re: Loki: An open-source tool for fact verification

#35

Post by Xudong »

siffland wrote: post #30

When I saw Loki as the name, I instantly thought of Grafana Loki for logging. I click on the GitHub and get Libr-AI and OpenFactVerification. I am not commenting on the actual software and I know names are hard and often overlap, but with something as popular as Loki already used for logging I think it might get confusing.

Hi siffland! Thank you for your feedback. We understand your concern about the potential confusion given the popularity of Grafana Loki in the logging space. When naming our project, we sought a name that encapsulates our goal of combating misinformation. We chose Loki, inspired by the Norse god often associated with stories and trickery, to symbolize our commitment to unveiling the truth hidden within nonfactual information.

When we named our project, we were unaware of the overlap with Grafana Loki. We appreciate you bringing this to our attention! I will discuss this issue with my team in the next meeting, and figure out if there is a better way of solving this. If you have any suggestions or thoughts on how we can better differentiate our project, we would love to hear them.

Thank you again for your valuable input!

Re: Loki: An open-source tool for fact verification

#36

Post by Xudong »

swores wrote: post #18

Feedback on the example gif: at the moment it's almost comically useless. First you're bored watching the beginning 90% while commands are slowly being typed, and then the bit that's actually interesting and worth reading scrolls too fast and then resets to the beginning of the gif before there's a chance to read it.

Thanks for your feedback on the gif figure, swores! We will revise it soon.

Re: Loki: An open-source tool for fact verification

#37

Post by big_hacker »

Xudong wrote: post #33
axegon_ wrote: post #17

Overall great idea though, I'll be definitely checking it back in the future. A few things that hit me out of the box: * The idea behind using Serper is great, however it would be cool if other search engines/data sources can be used instead, ie. Kagi or some private search engine/data. Reason for the latter: there are tons of people who are sourcing all sorts of information which will not immediately show up on goog…

Thank you for your suggestions, axegon!!! We will definitely consider them and add the features in a future version shortly. Regarding the first version, we are currently working on enabling customized evidence retrieval, including local files. Our plan is to integrate existing tools like LlamaIndex. Any suggestion is greatly appreciated! Regarding the second point, we have found OpenAI's JSON mode to be greatly help…

Dead internet.

Re: Loki: An open-source tool for fact verification

#39

Post by smoyer »

martinbaun wrote: post #38

Maybe the name is not so fitting as Loki is a name in Norse Mythology. Known for deceiving and lying which is basically the opposite you're trying to do :)

It's also the name of a well-known open-source log collection system that's part of the LGTM stack (predominantly led by GrafanaCloud Labs.)

Re: Loki: An open-source tool for fact verification

#40

Post by croes »

martinbaun wrote: post #38

Maybe the name is not so fitting as Loki is a name in Norse Mythology. Known for deceiving and lying which is basically the opposite you're trying to do :)

Maybe it's on purpose.

Who could better know the patterns of liars than the god of lying.

Post reply on HN