Viewing profile — janchorowski

Yes, we hope that the YAMLs will provide a clear separation between configuration and code allowing easier deploys of many apps from same codebase and principled experiments. Thank…

Hello everyone! I am Jan, CTO and one of the creators of Pathway, the real-time data processing framework. I’m excited to share Pathway’s ready-to-use AI Pipelines, configurable wi…

An index that combines multiple indexing techniques, e.g. vector search with more classical information retrieval techniques such as BM25. We found that they have different strengt…

Thanks! Can you share some more details on the usecases and features used?

Thanks for the kind words!

Hi HN data folks, I am excited to share Pathway, a Python data processing framework we built for ETL and RAG pipelines. https://github.com/pathwaycom/pathway We started Pathway to …

Hey, I'm Jan, a Machine Learning researcher, and CTO of Pathway. I've been working on RAG for the last year and I'm excited to share a new RAG optimization strategy that adapts the…

Now you can use Airbyte source connectors to process data in memory with Python. We integrated Airbyte connectors with Pathway, a Python stream processing framework, using the airb…

Originally from 1998: https://web.archive.org/web/19980206223800/https://norvig.co... I didn't want to mislead with a newer date. Sorry if it confuses you.

That's a good idea, the deduplication criterion is easy to change, using an llm is faster to get started, but after a while a corpus of decisions is created and can be used to eith…

Sure: when a new response is produced because some source documents have changed we ask an LLM to compare the responses and tell if they are significantly different. Even a simplis…

Hi I am Jan, CTO @ Pathway. A use case we have been working on with LLMs is to let people know when an answer to their query changes due to revisions of source documents. Obviously…

An index is a software component building block, which becomes a database when wrapped with the data management system. We will see more and more traditional databases to add a vec…

To quickly get to the application sources please go to: - https://github.com/pathwaycom/llm-app/blob/main/llm_app/path... for the simplest contextless app - https://github.com/path…

This depends on the data source used. Some track updateable collections, some have a more "append-only" nature. For instance, tracing a database table using CDC+Debezium will suppo…

Pathway ( https://github.com/pathwaycom/pathway ) is a data processing framework we are developing that unifies stream and batch processing of large datasets. It lets developers co…

Hi HN, I am Jan, CTO and co-founder of Pathway.com. We’ve built a LLM microservice that answers questions about a corpus of documents, while automatically reacting to additions of …