Live data from Hacker News

AI for AWS Documentation

awsdocsgpt.com

61–70 of 134 posts

Re: AI for AWS Documentation

#61
I asked it to write some basic terraform:

- Make a VPC. - Add an Instance. - Abstract the region and AZ, into vars.

etc... every time I wanted to change the code, I asked the bot to do the refactor, and it did.

Overall, I'm impressed. It wasn't the most complicated thing, but it didn't dive off the deep end.

Re: AI for AWS Documentation

#62
I wonder if people will make DSLs specifically for LLMs.

For example the terseness / symbols of APL, Perl, or event set notation.

LLMs could train and output the shorter symbolic notation, and it could be expanded for human readability by another program at export.

Re: AI for AWS Documentation

#63

Earlier quoted context omitted.

Why? You can ask the same questions to ChatGPT and get the same or better answers. I also know from personal experience with ChatGPT, that you can use it to: - convert Python/boto3 to any language that has an AWS SDK - convert CloudFormation to Terraform or the CDK - write scripts that use the SDK You will get the occasional hallucination.

ChatGPT is not up to date unless you start using the plugins. This sort of indexing is based on vector databases and various intermediate prompting. If you want to get technical, the academic term is "Retrieval Augmented Generation".

Hallucination is unfortunately inevitable when it comes to any autoregressive model, even with RAG. You can minimize hallucination by prompting, but you'll still see some factually incorrect responses here and there (https://zilliz.com/blog/ChatGPT-VectorDB-Prompt-as-code).

I unfortunately don't think we'll be able to solve hallucination anytime soon. Maybe with the successor to the transformer architecture?

Re: AI for AWS Documentation

#64

What's the best current way to make a bunch of documents searchable via LLMs like this? I've tried the using OpenAI w embeddings (iirc), but this was slow, got expensive quickly, and it struggled to answer questions about the text accurately. Curious if there's better standard approaches now.

You could copy the steps listed in alpaca-lora with your own data. Try thebloke's superhot ggml models. Train it in lambda labs or replicate

Re: AI for AWS Documentation

#65

We did something similar for all the cloud whitepapers from AWS, Azure, GCP, CloudFlare, and CNCF at https://cloudwhitepapers.withcoherence.com/ These are fun projects!

You might have better outcomes with the API docs and official documentation. Whitepapers tend to have a lot of marketing, may not age gracefully.

Re: AI for AWS Documentation

#66

We did something similar for all the cloud whitepapers from AWS, Azure, GCP, CloudFlare, and CNCF at https://cloudwhitepapers.withcoherence.com/ These are fun projects!

This looks cool. How does this work?

We actually open sourced the project - see here https://www.withcoherence.com/post/create-your-langchain-and...

Re: AI for AWS Documentation

#67

We did something similar for all the cloud whitepapers from AWS, Azure, GCP, CloudFlare, and CNCF at https://cloudwhitepapers.withcoherence.com/ These are fun projects!

You might have better outcomes with the API docs and official documentation. Whitepapers tend to have a lot of marketing, may not age gracefully.

Thanks for the perspective!

Re: AI for AWS Documentation

#69
Unfortunately this is a good concept, but it's using gpt-3.5-turbo. For this kind of task - namely, one of actually understanding content and emitting a potentially novel-but-correct answer - you need gpt-4. But it's quite slow and you'll quickly run into rate limiting.

I ran into these issues when building this for my own company's docs, at least.

Post reply on HN