Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Loki: An open-source tool for fact verification

github.com

Post reply on HN
Print view View original

41–50 of 74 posts

Re: Loki: An open-source tool for fact verification

#41

Post by westurner »

Xudong wrote: post #25
rjb7731 wrote: post #7

Isn't this similar to the Deepmind paper on long form factuality posted a few days ago? https://arxiv.org/abs/2403.18802 https://github.com/google-deepmind/long-form-factuality/tree...

Yes, they are similar. Actually, our initial paper was presented around five months ago ( https://arxiv.org/abs/2311.09000 ). Unfortunately, our paper isn't cited by the DeepMind paper, which you may see this discussion as an example: https://x.com/gregd_nlp/status/1773453723655696431 Compared with our initial version, we have mainly focused on its efficiency, with a 10X faster checking process without decreasing acc…

> We further construct an open-domain document-level factuality benchmark in three-level granularity: claim, sentence and document

A 2020 Meta paper [1] mentions FEVER [2], which was published in 2018.

[1] "Language models as fact checkers?" (2020) https://scholar.google.com/scholar?cites=3466959631133385664

[2] https://paperswithcode.com/dataset/fever

I've collected various ideas for publishing premises as linked data; "#StructuredPremises" "#nbmeta" https://www.google.com/search?q=%22structuredpremises%22

From "GenAI and erroneous medical references" https://news.ycombinator.com/item?id=39497333 :

>> Additional layers of these 'LLMs' could read the responses and determine whether their premises are valid and their logic is sound as necessary to support the presented conclusion(s), and then just suggest a different citation URL for the preceding text

> [...] "Find tests for this code"

> "Find citations for this bias"

From https://news.ycombinator.com/item?id=38353285 :

> "LLMs cannot find reasoning errors, but can correct them" https://news.ycombinator.com/item?id=38353285

> "Misalignment and [...]"

Re: Loki: An open-source tool for fact verification

#43

Post by antihipocrat »

big_hacker wrote: post #37
Xudong wrote: post #33

Earlier quoted context omitted.

Thank you for your suggestions, axegon!!! We will definitely consider them and add the features in a future version shortly. Regarding the first version, we are currently working on enabling customized evidence retrieval, including local files. Our plan is to integrate existing tools like LlamaIndex. Any suggestion is greatly appreciated! Regarding the second point, we have found OpenAI's JSON mode to be greatly help…

Dead internet.

Have to agree with you, every comment from the product creator reads like a chatGPT response.

Re: Loki: An open-source tool for fact verification

#44

Post by eMPee584 »

swores wrote: post #18

Feedback on the example gif: at the moment it's almost comically useless. First you're bored watching the beginning 90% while commands are slowly being typed, and then the bit that's actually interesting and worth reading scrolls too fast and then resets to the beginning of the gif before there's a chance to read it.

mpv ftw: playback speed control even for gifs..

Re: Loki: An open-source tool for fact verification

#45

Post by Xudong »

antihipocrat wrote: post #43
big_hacker wrote: post #37

Earlier quoted context omitted.

Dead internet.

Have to agree with you, every comment from the product creator reads like a chatGPT response.

I will take it as a compliment, lol. But I do hope ChatGPT or some agents could help me with this. Btw, our recent study on machine-generated text detection might be interesting to you.

https://arxiv.org/abs/2305.14902 https://arxiv.org/abs/2402.11175

Re: Loki: An open-source tool for fact verification

#46

Post by redder23 »

The name Loki is such a great fit! WOW!

This is some giant BS that is for sure. Some stupid, literally brain-dead AI searching things created by humans to determine what is a "fact". This is beyond dystopian crap.

We all know all the fact-checker orgs. used by big tech like Facebook and others are filled with hyper biased woke people who do not actually fact-check things but get off on having the power to enforce their beliefs, feelings and biases.

I can already tell this is total BS without even looking into it, what kinds of sources will it use? What ranking will they give them? Snopes? ROFL. Probably just uses some woke infested, censored and curated language model to determine a fact based on what has the most matches or THE MOST LIKELY because that how AI works. Has absolutely nothing to do with facts.

And it's even worse, we are literally in a time when AI hallucinates things that do not exist. I won't use a stupid AI to find me "facts".

Re: Loki: An open-source tool for fact verification

#47

Post by Uehreka »

croes wrote: post #40
martinbaun wrote: post #38

Maybe the name is not so fitting as Loki is a name in Norse Mythology. Known for deceiving and lying which is basically the opposite you're trying to do :)

Maybe it's on purpose. Who could better know the patterns of liars than the god of lying.

When coming up with a name for something, the significance of the name should be the first thing that comes to mind, not the opposite thing or a thing that requires explanation. When people say the name of your tool, you won’t always be there to explain it.

Re: Loki: An open-source tool for fact verification

#48

Post by jimmygrapes »

Uehreka wrote: post #47
croes wrote: post #40

Earlier quoted context omitted.

Maybe it's on purpose. Who could better know the patterns of liars than the god of lying.

When coming up with a name for something, the significance of the name should be the first thing that comes to mind, not the opposite thing or a thing that requires explanation. When people say the name of your tool, you won’t always be there to explain it.

Discord comes to mine although I know it's a play on Eris (Free Net), same as Slack is a nod to Uncle Bob

Re: Loki: An open-source tool for fact verification

#50

Post by martinbaun »

smoyer wrote: post #39
martinbaun wrote: post #38

Maybe the name is not so fitting as Loki is a name in Norse Mythology. Known for deceiving and lying which is basically the opposite you're trying to do :)

It's also the name of a well-known open-source log collection system that's part of the LGTM stack (predominantly led by GrafanaCloud Labs.)

Don't want to be that guy, but I still don't get it :) You want your logs to lie? :D
Post reply on HN