Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Bible Semantic Search

share.streamlit.io

Post reply on HN
Print view View original

71–80 of 93 posts

Re: Bible Semantic Search

#71

Post by xhrpost »

Syonyk wrote: post #34
chrislee973 wrote: post #29

Earlier quoted context omitted.

Hey, creator of the Bible Semantic Search app here. I 100% agree with you. I hacked together this prototype for the purposes of learning the Pinecone API rather than fine-tuning a language model, but I'm still pleasantly surprised by the quality of the search results despite all the shortcomings you mentioned. The results aren't great, but they're decent. I'm surprised how well SBERT works out of the box considering…

> Switching to a version that uses more modern English like the NIV is trivial so maybe I'll do that. ESV or CSB, please... How hard would it be to train it on the range of common modern translations? It would be an interesting stress test of the models to see how close searches in different translations are - they're theoretically all communicating the same thing, with different styles and emphasis, but I'd expect a…

The range[1] you speak of could definitely be interesting to add. There can already be quite a difference between translations on this spectrum which could help this tool pick up even more possibilities for particular meanings (particularly on the extreme end of functional/paraphrasing like The Message you mentioned or The Passion Translation). It's quite fascinating looking at a verse across this spectrum and seeing what the different translators chose to focus on or "pull out" of the underlying Greek/Hebrew.

I haven't personally read much of ESV or CSB, just curious, why the preference for those?

[1]: https://en.wikipedia.org/wiki/Dynamic_and_formal_equivalence

Re: Bible Semantic Search

#72

Post by yjftsjthsd-h »

xhrpost wrote: post #68
chrislee973 wrote: post #29

Earlier quoted context omitted.

Hey, creator of the Bible Semantic Search app here. I 100% agree with you. I hacked together this prototype for the purposes of learning the Pinecone API rather than fine-tuning a language model, but I'm still pleasantly surprised by the quality of the search results despite all the shortcomings you mentioned. The results aren't great, but they're decent. I'm surprised how well SBERT works out of the box considering…

Some modern translations might have copyright rules preventing this without express permission (which may or may not be difficult to acquire, obviously other sites/apps have done it). They all have different rules and limitations but I remember reading that the NET Bible was specifically built for more permissive copyright requirements in the Internet age.

https://ebible.org/t4t/ might work? It's available in ex. https://andbible.github.io/ which bodes well for its licensing. It's also in conventional English (minimal jargon), which is probably a positive.

Edit: Actually, better yet: https://crosswire.org/sword/modules/ModDisp.jsp?modType=Bibl... has licensing info for each listed item

Re: Bible Semantic Search

#73

Post by eckza »

chrislee973 wrote: post #29
lalaland1125 wrote: post #10

SBERT isn't trained on this type of archaic English and you can see it failing. It needs to be fine tuned or you should use a modern Bible translation. A clear example is the query: "homosexuality" This returns: > James 2:3 - And ye have respect to him that weareth the gay clothing, and say unto him, Sit thou here in a good place; and say to the poor, Stand thou there, or sit here under my footstool It's clearly seei…

Hey, creator of the Bible Semantic Search app here. I 100% agree with you. I hacked together this prototype for the purposes of learning the Pinecone API rather than fine-tuning a language model, but I'm still pleasantly surprised by the quality of the search results despite all the shortcomings you mentioned. The results aren't great, but they're decent. I'm surprised how well SBERT works out of the box considering…

Semantic search on several versions and show results for chapter and verse in my preferred version = best of both worlds.

This is awesome. Thank you for building it.

Re: Bible Semantic Search

#74

Post by mfer »

phgn wrote: post #66

FYI it's failing for me with: urllib3.exceptions.ProtocolError: This app has encountered an error. The original error message is redacted to prevent data leaks. Full error details have been recorded in the logs (if you're on Streamlit Cloud, click on 'Manage app' in the lower right of your app). Traceback: File "/home/appuser/venv/lib/python3.8/site-packages/streamlit/scriptrunner/script_runner.py", line 475, in _run…

It worked yesterday. Something broke in the meantime.

Re: Bible Semantic Search

#75

Post by JasonFruit »

chrislee973 wrote: post #30
kadoban wrote: post #15

Earlier quoted context omitted.

Couldn't they search in a modern translation and just refer back to the ancient one? Seems like the way bible verses are split up should remove the ability of translation to move anything around much.

Ah, that's a good idea. I was considering just fully replacing the KJV with the NIV, but for people who want the KJV, having a mapping between the two versions would work. Do the verses between the two versions map each other exactly? Like they cover the same exact thing, just written in a different style?

Since a few verses are in the KJV but not the NIV, some would never appear in searches.

Re: Bible Semantic Search

#76

Post by giancarlostoro »

phgn wrote: post #66

FYI it's failing for me with: urllib3.exceptions.ProtocolError: This app has encountered an error. The original error message is redacted to prevent data leaks. Full error details have been recorded in the logs (if you're on Streamlit Cloud, click on 'Manage app' in the lower right of your app). Traceback: File "/home/appuser/venv/lib/python3.8/site-packages/streamlit/scriptrunner/script_runner.py", line 475, in _run…

I'm having the same issue. My searched phrase was two words very simple "Judge not" for reference. I'll check back later, curious how this looks and behaves overall.

Re: Bible Semantic Search

#77

Post by swasheck »

chrislee973 wrote: post #29
lalaland1125 wrote: post #10

SBERT isn't trained on this type of archaic English and you can see it failing. It needs to be fine tuned or you should use a modern Bible translation. A clear example is the query: "homosexuality" This returns: > James 2:3 - And ye have respect to him that weareth the gay clothing, and say unto him, Sit thou here in a good place; and say to the poor, Stand thou there, or sit here under my footstool It's clearly seei…

Hey, creator of the Bible Semantic Search app here. I 100% agree with you. I hacked together this prototype for the purposes of learning the Pinecone API rather than fine-tuning a language model, but I'm still pleasantly surprised by the quality of the search results despite all the shortcomings you mentioned. The results aren't great, but they're decent. I'm surprised how well SBERT works out of the box considering…

sblgnt is a decent basis for literary analysis but it’s NT only. the non-trivial part of your work is to navigate the oddly byzantine bible licensing considerations

Re: Bible Semantic Search

#78

Post by michaelmior »

angrygoat wrote: post #35
gk1 wrote: post #7

Credit goes to Chris Lee: https://www.chrisleeportfolio.com Explanation when you click “Wat this?”: > This is a Streamlit app I prototyped for performing semantic search on the King James Bible. It conducts full text search as well as semantic search, which is useful for surfacing passages that are similar in meaning to the query, even if the passages don't explicitly contain the query keyword(s). Suppose you wanted…

How does pinecone compare to pgvector? https://github.com/pgvector/pgvector

Milvus[0] would be another interesting comparison.

[0] https://milvus.io/

Re: Bible Semantic Search

#79

Post by hushpuppy »

chrislee973 wrote: post #29
lalaland1125 wrote: post #10

SBERT isn't trained on this type of archaic English and you can see it failing. It needs to be fine tuned or you should use a modern Bible translation. A clear example is the query: "homosexuality" This returns: > James 2:3 - And ye have respect to him that weareth the gay clothing, and say unto him, Sit thou here in a good place; and say to the poor, Stand thou there, or sit here under my footstool It's clearly seei…

Hey, creator of the Bible Semantic Search app here. I 100% agree with you. I hacked together this prototype for the purposes of learning the Pinecone API rather than fine-tuning a language model, but I'm still pleasantly surprised by the quality of the search results despite all the shortcomings you mentioned. The results aren't great, but they're decent. I'm surprised how well SBERT works out of the box considering…

What makes the Bible unique in this context is that there are huge amount of resources that all reference the same basic structure.

One of the things that makes KJV version useful for scholars is that since it's been the "standard" for hundreds of years a great deal of other work references its structure. People use it because of this. It's less to do with it being a fabulously accurate translation or whatever. It's just newer bibles don't have this wealth of history and documentation that references it and much of them are pretty expensive to license, while KJV is public domain.

For example "Strong's Exhaustive Concordance". If you get a version of KJV with "Strong's Numbers" you can cross reference words and phrases in their original languages (greek/hewbrew/etc). This way students can understand some of the original meanings that go into difficult or disputed passages.

Also there is a large number of commentaries of all sorts of different types that reference specific passages.

Besides that KJV is just mostly valued in Protestant Christian dialects of Christianity. Other Christian religions such as various versions of Eastern Orthodox have different numbers of books or will arrange things in different orders. There have been different attempts to past scholars to arrange things in more chronological order, too.

This makes the Bible fairly unique when it comes to literature. Each verse of text can have dozens of different "back links" and "references".

so if somebody searches for the subject "Homosexuality" it will get hits in various commentaries. Those commentaries all directly reference verses in the Bible.

So you could show the version found and why it was selected. That way a reader would be shown "These authors think this verse is has to do with homosexuality" and they could click through and find out the justification for this, different translations, what those translations are likely based on, what other Christian sects feel this verse means, and so on and so forth.

I don't know if it would be useful for you, but there is a "Sword Project" that collects and cross references different Bibles and bible resources as well as tools.

https://www.crosswire.org/sword/index.jsp

Re: Bible Semantic Search

#80

Post by overthemoon »

the_only_law wrote: post #21
yosito wrote: post #17

Would have loved this back in my religious days, or in the brief period of time I was interested in criticizing religion. These days I'm not really interested at all in the Bible, but this is still a pretty cool idea. A little bit of an archaic choice to search the KJV though. Maybe it was chosen due to copyright issues.

Isn't the KJV popular among evangelical sects as well? I remember seeing more of those as a kid than any modernized version.

At least when I was growing up it was the New International Version. You were a fundie if you insisted on the KJV or the NKJV among people I knew.
Post reply on HN