Cool project
Thanks!
Show HN: ChainForge, a visual tool for prompt engineering and LLM evaluation
31–33 of 33 posts
Re: Show HN: ChainForge, a visual tool for prompt engineering and LLM evaluation
#32Earlier quoted context omitted.
It seems to be more powerful than langflow and flowise https://github.com/logspace-ai/langflow https://github.com/FlowiseAI/Flowise
k i need somebody to do a comparison table for us...
Yeah, I'd love to see these tools discussed in an upcoming Latent Space episode.
Re: Show HN: ChainForge, a visual tool for prompt engineering and LLM evaluation
#33I like it! Any plans to add Google Vertex AI support?
There is a long term vision of supporting fine-tuning through an existing evaluation flow. We originally created this because we were worried about how to evaluate ‘what changed’ between a fine-tuned LLM and its base model. I wonder if Vertex AI has an API that we could plug-in, though, or if it’s limited to the UI.
I meant for completion, chat and embedding. https://cloud.google.com/vertex-ai/docs/generative-ai/chat/t... . Some examples here.
Vertex AI has the same API as PaLM as far as I know. However, the authorization is through Google Cloud. So I use it like any other GCP API.
I love the idea of adding fine tuning as a node though. Here is the API for creating a model tuning job - https://cloud.google.com/vertex-ai/docs/generative-ai/models...
I wish I could use ChainForge nodes in Node Red.