Using ChatGPT for AWS service questions is actually pretty good. For instance, I asked it for a Cloud Practitioner study guide (using a small set of crafted prompts), and GPT performed quite well. While I have yet to query GPT about Solutions Architect or DevOps material, I know I can feed a set of URLs and GPT will "learn" the material and summarize it in ways meaningful and relevant to my prompts. In this way, ChatGPT is quite a powerful assistant on its own.
AI for AWS Documentation
21–30 of 134 posts
Re: AI for AWS Documentation
#22Meh. It doesn't actually pull a valid response. We just upgraded a database that required an updated EC2 instance, and it failed to connect those dots.
Re: AI for AWS Documentation
#23Prompt: What is glacier? Glacier is a term that is not directly mentioned in the provided sources. Prompt: What is a glacier? A glacier is a large mass of ice that moves slowly over time due to the accumulation of snow, ice, and other forms of frozen precipitation. Seems like it’s just using a general model?
It's definitely just using standard semantic search (otherwise you wouldn't be getting links). "What is glacier storage" gives you a great response.
If they finetuned the model on AWS docs, would the embeddings, and thus the vector search improve?
Re: AI for AWS Documentation
#24It invented an answer for something that AWS actually can't do right now. https://imgur.com/a/0IYZ2WV
phind.com did not halucinate in this case: https://capture.dropbox.com/4gIUDuAxr14bnNIt
https://www.phind.com/search?cache=d0b3a85b-17f9-4def-b8d0-b...
Re: AI for AWS Documentation
#25Just use Phind.com for searching developer docs for most cases.
This is cool, thanks for linking
Re: AI for AWS Documentation
#26The problem is garbage in -> garbage out. When the docs are wrong or misleading you'll still get burned, even if the model doesn't hallucinate responses
Agreed, that's the problem eod. Was trying to set up an instance a few days ago and the docs for it hadn't been updated since '21
Re: AI for AWS Documentation
#27Nice work! Would be even more useful to be able to have a conversation with it.
We're in the process of doing just that and adding chat context/basically remembering your past questions.
Re: AI for AWS Documentation
#28Please I beg you: ask it how to take a snapshot of an EC2 instance and then how to restore it :)
Re: AI for AWS Documentation
#29Earlier quoted context omitted.
It's definitely just using standard semantic search (otherwise you wouldn't be getting links). "What is glacier storage" gives you a great response.
If they finetuned the model on AWS docs, would the embeddings, and thus the vector search improve?
Most likely, the model would be less inclined to answer questions/hallucinate for prompts not related to AWS—this is definitely be a future path for improvement
Re: AI for AWS Documentation
#30seems super useful, try this prompt: how do you use wrangler and glue to make athena tables using terraform
Awesome to hear that it was useful for you!!!