Ask HN: LLM Prompt Engineering

#1

Post by Scotrix » Thu, Sep 18, 2025, 2:11 PM UTC

I’m working on a project where I need to extract user intents and move them to deterministic tool/function/api executions + afterwards refining/transforming the results by another set of tools. Since gathering the right intent and parameters (there are a lot of subtle differences in potential prompts) is quite challenging I’m using a long consecutive executed list of prompts to fine tune to gather exactly the right pieces of information needed to have somewhat reliable tool executions. I tried this with a bunch of agent frameworks (including langchain/langgraph) but it gets very messy very quickly and this messiness is creating a lot of side effects easily.

So I wonder if there is a tool, approach, anything to keep better control of chains of LLM executions which don’t end up in a messy configuration and/or code execution implementation? Maybe even something more visual, or am I the only struggling with this?

Re: Ask HN: LLM Prompt Engineering

#2

Post by thekuanysh » Thu, Sep 18, 2025, 2:12 PM UTC

What kind of IO do you have? JSON or plain language?

Re: Ask HN: LLM Prompt Engineering

#3

Post by Scotrix » Thu, Sep 18, 2025, 3:48 PM UTC

thekuanysh wrote: post #2

What kind of IO do you have? JSON or plain language?

I input text and preferably I output JSON but doesn’t matter much as long as it’s somewhat structured.

Ultimately I’d like to extract information like date ranges, specific indications of tool usages (e.g. I have a bunch of data apis with their own individual data and semantic meaning which need to be picked and then a combination of tools to transform the data)

Re: Ask HN: LLM Prompt Engineering

#4

Post by zerodayai » Thu, Sep 18, 2025, 5:13 PM UTC

Scotrix wrote: post #3

thekuanysh wrote: post #2

What kind of IO do you have? JSON or plain language?

I input text and preferably I output JSON but doesn’t matter much as long as it’s somewhat structured. Ultimately I’d like to extract information like date ranges, specific indications of tool usages (e.g. I have a bunch of data apis with their own individual data and semantic meaning which need to be picked and then a combination of tools to transform the data)

I am creating something along these lines, https://github.com/zero-day-ai, it's meant for security testing, but probably has most of the functionality you need (and you can write plugins fairly easily if not); you can create a prompt repository, defined by a schema that are organized my domains (again, security testing domains, but they can be expanded). If you have any features you'd like to see, or have an ideal workflow feel free to ping me: anthony@zero-day.ai

Re: Ask HN: LLM Prompt Engineering

#5

Post by Scotrix » Fri, Sep 19, 2025, 10:46 AM UTC

zerodayai wrote: post #4

Scotrix wrote: post #3

Earlier quoted context omitted.

I input text and preferably I output JSON but doesn’t matter much as long as it’s somewhat structured. Ultimately I’d like to extract information like date ranges, specific indications of tool usages (e.g. I have a bunch of data apis with their own individual data and semantic meaning which need to be picked and then a combination of tools to transform the data)

I am creating something along these lines, https://github.com/zero-day-ai , it's meant for security testing, but probably has most of the functionality you need (and you can write plugins fairly easily if not); you can create a prompt repository, defined by a schema that are organized my domains (again, security testing domains, but they can be expanded). If you have any features you'd like to see, or have an ideal w…

cool, i'll have a look. thanks for sharing.