Testing GPT 4's code-writing capabilities with some real world problems

> simply know firsthand (i'm old, i have manager, cto, ceo friends who I go golf and play squash with) that people in data entry and programming have been let go in the past weeks because 1 person could take over their work using the gtp/chatgpt api's and do their work faster with less errors. I am recommending the same in my company as a lot of my colleagues are doing nothing anymore as the skilled seniors are doing it themselves with gpt now as it's faster, less communication etc. We feed jira issues into gpt and it generates code; we review and refine or fix ourselves. It works much much faster and with better results.

This isn't software engineering work, this is 21st century data entry with some code. This is exactly the type of menial work that should be automated by AI.

If you have small self contained problems like map X -> Y then sure, ChatGPT will be sufficient. Where I disagree with you is calling these jobs "programming" jobs. These are the type of tasks that should've been written in a transform language like JOLT. This shouldn't even be code.

> With humans this is a painful and slow process, especially with people who are fast and loose (broken education seems to made many of those graduate just by cheer production speed & volume instead of quality; gpt can do that better too...).

Humans suck at repetitive menial tasks like this. It's not education's fault.

ChatGPT is just a tool. With it you can create a service and fine-tune it to specific use cases. Let's imagine there's some cool startup called chris-the-it-consultant.ai that uses ChatGPT in the backend. Here's a prototype:

System Prompt: You are CHRIS the IT consultant. Once you take in a question, make sure you understand all the requirements. Don't write code until you have an explicit request to do so. First make sure you ask all the questions relevant to solving the problem and needed by the code. All the responses as CHRIS should be written as CHRIS: your response here.

    Sure, I understand. As an IT consultant, it's important to fully understand the problem and requirements before proposing a solution or writing any code. Please let me know what question you have and I'll do my best to ask all relevant questions to ensure I fully understand the problem.

> CHRIS, we need a new basic POST endpoint

    CHRIS: Great, I'd be happy to help you with that. Could you please provide more information about the purpose of this endpoint? What kind of data will be sent in the request body? Will this endpoint be used to create new data, update existing data, or perform some other action?

> Third-party company XXQ will let you know the details. They will be the ones calling this new endpoint. But in essence it should be very simple: just grab whatever they pass and save it in our db.

    CHRIS: Understood. It sounds like the purpose of this endpoint is to receive data from a third-party company, XXQ, and store it in your database. Here are some questions to help me better understand the requirements:
    What format will the data be in? Will it be JSON, XML, or some other format?
    What kind of data will be included in the request body? Will it be simple data types like strings and numbers, or more complex data structures like arrays and objects?
    Will there be any validation or transformation required on the data before it's stored in the database?
    Are there any security or authentication requirements that need to be considered for this endpoint, given that it will be accepting data from a third-party company?

> We got this contract here: { product_id: string, product_price: float, product_name: string }

    CHRIS: Great, thanks for providing that information. Based on that, here are the next steps to create the POST endpoint:
    We'll need to set up an API endpoint to listen for POST requests.
    We'll need to parse the request body, which should contain the product information in JSON format.
    We'll need to validate the data to ensure that it contains the required fields (product_id, product_price, and product_name) and that the data types are correct.
    Once the data has been validated, we can save it to the database.

This is a real ChatGPT response (current ChatGPT - GPT 3.5 turbo, not GPT-4, and not a real system-prompt). Not bad for a prototype! Now give CHRIS Github/Vercel/SSH access keys, SMTP/IMAP, Slack and a Whisper interface to Zoom/Meet/Teams and invite it to the 200 meetings where things will be changed around 1000x before being released. Raise some VC money and you are all set!

This is exactly what GPT is great for, accelerating learning.

As expected, LLMs don't actually think. This is not really a surprising result when you understand that it's a few billion Markov chains in a trenchcoat.