Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AI for AWS Documentation

awsdocsgpt.com

Post reply on HN
Print view View original

71–80 of 134 posts

Re: AI for AWS Documentation

#73

Post by scrum-treats »

8note wrote: post #54
scarface_74 wrote: post #40

Earlier quoted context omitted.

Asking the same question to Chatgpt To connect a Lambda function to an RDS (Relational Database Service) instance and query it safely, you can follow these general steps: 1. Configure your Lambda function's execution role: Ensure that your Lambda function has the necessary permissions to access the RDS instance. The role should include the `AWSLambdaVPCAccessExecutionRole` and the appropriate RDS permissions. 2. Plac…

This all looks about right, but how would somebody without aws experience be able to validate it without making a mess?

Same as pre-GPT: Labs. You still have to practice implementation (for now).

Re: AI for AWS Documentation

#74

Post by efitz »

I asked it two questions and it gave me very reasonable, complete answers:

How do I connect two VPCs from separate organizations (VPC peering)

How do I only allow authorized applications to access my S3 bucket (BPA, IAM policies, a lot of best practices docs)

Re: AI for AWS Documentation

#76

Post by Simorgh »

fzliu wrote: post #63
KRAKRISMOTT wrote: post #51

Earlier quoted context omitted.

ChatGPT is not up to date unless you start using the plugins. This sort of indexing is based on vector databases and various intermediate prompting. If you want to get technical, the academic term is "Retrieval Augmented Generation".

Hallucination is unfortunately inevitable when it comes to any autoregressive model, even with RAG. You can minimize hallucination by prompting, but you'll still see some factually incorrect responses here and there ( https://zilliz.com/blog/ChatGPT-VectorDB-Prompt-as-code ). I unfortunately don't think we'll be able to solve hallucination anytime soon. Maybe with the successor to the transformer architecture?

Hallucination is naturally a concern for anyone looking to depend upon LLM-generated answers.

We’ve been testing LLM responses with a CLI, we’re using it to generate accuracy statistics, which is especially useful when the use-case Q/A is limited.

If ‘confidence’ can be returned to the user, then at least they can have an indication if there is a higher quality-risk with a given response.

Re: AI for AWS Documentation

#77

Post by shaunpud »

sovietmudkipz wrote: post #31

This is exactly the kind of software that people should be making. Even if this one is bad, the concept is very sound. I want to have a specialist AI that is trained to help me learn how to use the software. 100% what should be happening. General AI should know how to do stuff too but having an specialist AI implies that the company/group of people/person is making sure to tune the model. Just an IMO.

I was looking at the CloudLinux[1] docs the other day and noticed an "Ask GPT?" feature which I found quite helpful.

[1] https://docs.cloudlinux.com

Re: AI for AWS Documentation

#78

Post by avereveard »

8note wrote: post #54
scarface_74 wrote: post #40

Earlier quoted context omitted.

Asking the same question to Chatgpt To connect a Lambda function to an RDS (Relational Database Service) instance and query it safely, you can follow these general steps: 1. Configure your Lambda function's execution role: Ensure that your Lambda function has the necessary permissions to access the RDS instance. The role should include the `AWSLambdaVPCAccessExecutionRole` and the appropriate RDS permissions. 2. Plac…

This all looks about right, but how would somebody without aws experience be able to validate it without making a mess?

It still isn't tho, this will work for testing, maybe, but each lambda will be it's own connection and that will exhaust db resources real fast, you're supposed to have a pooling proxy between lambdas and RDS.

Re: AI for AWS Documentation

#79

Post by avereveard »

underlines wrote: post #58

RAG is very difficult to do right. I am experimenting with various RAG projects from [1]. The main problems are: - Chunking can interfer with context boundaries - Content vectors can differ vastly from question vectors, for this you have to use hypothetical embeddings (they generate artificial questions and store them) - Instead of saving just one embedding per text-chuck you should store various (text chunk, hypothe…

To solve the question embedding issue I prefer another approach, you store document embeddings as normal, then for searching you let LLM hallucinate an answer and use the wrongish answer embedding to do the vector search.
Post reply on HN