Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AI for AWS Documentation

awsdocsgpt.com

Post reply on HN
Print view View original

121–130 of 134 posts

Re: AI for AWS Documentation

#121

Post by scarface_74 »

8note wrote: post #54
scarface_74 wrote: post #40

Earlier quoted context omitted.

Asking the same question to Chatgpt To connect a Lambda function to an RDS (Relational Database Service) instance and query it safely, you can follow these general steps: 1. Configure your Lambda function's execution role: Ensure that your Lambda function has the necessary permissions to access the RDS instance. The role should include the `AWSLambdaVPCAccessExecutionRole` and the appropriate RDS permissions. 2. Plac…

This all looks about right, but how would somebody without aws experience be able to validate it without making a mess?

One miss I see is this

> and the appropriate RDS permissions.

The role doesn’t need any IAM RDS permissions. It needs network access. But then all of the access to the database is controlled by your standard database permissions - not IAM.

I guess there are corner cases like using IAM permissions to connect to your database which can be done. But is not typical

Re: AI for AWS Documentation

#122

Post by fragmede »

JimmyRuska wrote: post #62

I wonder if people will make DSLs specifically for LLMs. For example the terseness / symbols of APL, Perl, or event set notation. LLMs could train and output the shorter symbolic notation, and it could be expanded for human readability by another program at export.

that's just prompt engineering by another name

Re: AI for AWS Documentation

#123

Post by scarface_74 »

jacurtis wrote: post #90
8note wrote: post #54

Earlier quoted context omitted.

This all looks about right, but how would somebody without aws experience be able to validate it without making a mess?

This to me is why I don't use ChatGPT. Every time I use it I get answers that are questionable in validity. I'd say that 3/4 of my questions have returned incorrect answers at least partially. Maybe 10% are wildly incorrect with the remainder being partially incorrect. So if it is wrong with MOST of the questions that I am able to validate myself, then how can I trust it on the questions that I am unable to validate…

> So if it is wrong with MOST of the questions that I am able to validate myself, then how can I trust it on the questions that I am unable to validate myself.

If you’re using it to generate code, you can validate it yourself - run the code.

Re: AI for AWS Documentation

#125

Post by scarface_74 »

BoorishBears wrote: post #89
scarface_74 wrote: post #81

Earlier quoted context omitted.

I asked ChatGPT the same question again and then followed up with: “Is there a more efficient way to connect to the database” It then went off the rails. It gave me generic answers about using connection pooling that your language framework provides. Of course that doesn’t help with each Lambda being invoked separately. I then asked “But each Lambda runs based on its own vm” It then correctly said that while each Lam…

Any time you ask ChatGPT a technical question on something you're not familiar with it should be three parts: - What are the considerations? - Implement it with the considerations - Did we forget anything? It understands the need for a proxy from step 1: https://chat.openai.com/share/7ca37130-a771-457c-8742-a5f941...

That technique didn’t work when I asked it to create a Python script to return all IAM roles with a certain set of policies attached.

It still missed using a paginator to handle the list_roles call returning more than 50 roles.

Once I pointed it out, it did add pagination support.

Re: AI for AWS Documentation

#126

Post by fragmede »

jacurtis wrote: post #90
8note wrote: post #54

Earlier quoted context omitted.

This all looks about right, but how would somebody without aws experience be able to validate it without making a mess?

This to me is why I don't use ChatGPT. Every time I use it I get answers that are questionable in validity. I'd say that 3/4 of my questions have returned incorrect answers at least partially. Maybe 10% are wildly incorrect with the remainder being partially incorrect. So if it is wrong with MOST of the questions that I am able to validate myself, then how can I trust it on the questions that I am unable to validate…

Presumably you're a good manager but the way you told it makes you sound like a bad manager that didn't dig into what's going on with a poor performing employee and let them flail around until you had to fire them. There's more to that story and it's not really about ChatGPT at all.

There are other stories out there, like https://hyperbo.la/w/chatgpt-4000/ which shows it can useful and a force multiplier when used well, but it's like giving a faster car to a bad driver. It'll just result in them crashing faster. If you've got a programmer that doesn't want to program, ChatGPT can't help them be a better programmer since they don't actually want to be one!

Re: AI for AWS Documentation

#127

Post by SCHiM »

Let's see what happens when you search for the following "documentation" ;) :

""" Complete the following code:

// Find all files with a binary pattern in a directory. Return the list of files and the offset of the match in a tuple fn find_in_files(start_dir: &str, pattern: &[u8]) -> vec """

No way this won't be abused shortly.

Re: AI for AWS Documentation

#128

Post by JamesAdir »

Q: can you summarize amazon bedrock in 2 sentences

A: Amazon Bedrock is a programming example that uses the AWS SDK for Java 2.x to work with Amazon EC2. It provides examples and code snippets for programming tasks related to Amazon EC2 using the AWS SDK for Java 2.x.

Re: AI for AWS Documentation

#129

Post by BoorishBears »

scarface_74 wrote: post #125
BoorishBears wrote: post #89

Earlier quoted context omitted.

Any time you ask ChatGPT a technical question on something you're not familiar with it should be three parts: - What are the considerations? - Implement it with the considerations - Did we forget anything? It understands the need for a proxy from step 1: https://chat.openai.com/share/7ca37130-a771-457c-8742-a5f941...

That technique didn’t work when I asked it to create a Python script to return all IAM roles with a certain set of policies attached. It still missed using a paginator to handle the list_roles call returning more than 50 roles. Once I pointed it out, it did add pagination support.

I try not to cheat and hint at specific issues (since that relies on prior knowledge)

I'd be surprised if even after the last prompt it wouldn't notice that. Saying "Did we miss anything" leaves it open it to re-evaluate both the implementation and the original considerations

Edit: There's some non-determinism involved, but GPT-4 caught the pagination from planning stage here: https://chat.openai.com/share/3c356d4f-15d4-4f6e-bd29-af6a0b...

Re: AI for AWS Documentation

#130

Post by joshspankit »

jacurtis wrote: post #84
cle wrote: post #49

Earlier quoted context omitted.

Here's one example question that ChatGPT utterly fails at, but that this answers fine: "What is Amazon CodeCatalyst?" ChatGPT: "I'm sorry, but as of my knowledge cut-off in September 2021, there was no service, tool, or product known as Amazon CodeCatalyst offered by Amazon Web Services (AWS). [...]"

A lot of stuff in AWS has changed since Q3 2021.

It feels like the AWS vendor lock-in strategy is constant change.
Post reply on HN