How well can LLMs write COBOL?
bloop.ai
How well can LLMs write COBOL?
1–10 of 63 posts
Re: How well can LLMs write COBOL?
#2I tried to get chatgpt to write 6502 assembler for the 1980s 8-bit home computer the BBC Micro. It was game, but clueless
Re: How well can LLMs write COBOL?
#3[dead]
Re: How well can LLMs write COBOL?
#4I tried to get chatgpt to write 6502 assembler for the 1980s 8-bit home computer the BBC Micro. It was game, but clueless
Try feeding it 8 pages of examples first? Something like this https://atariwiki.org/wiki/Wiki.jsp?page=Advanced%206502%20A...
Re: How well can LLMs write COBOL?
#5I suspect there were relatively few training examples for COBOL. It would be interesting to see the results for a system which had a significant number of such examples in the training set.
Re: How well can LLMs write COBOL?
#6It would be interesting to feed it a formal language specification of some language it hasn't seen and then ask it write code and see how it does.
That could be a test of reasoning and reading comprehension
Re: How well can LLMs write COBOL?
#7On one hand, it'd be super cool to solve the talent shortage in the field, also Cobol code isn't very pleasant to write.
On the other, I'm not sure I'd want tools known for poor code quality, and hallucinations, to write these super critical systems.
Guess there might be a copilotesque productivity booster for human developers, but I think these systems are some of the last places I'd want LLMs to contribute.
Re: How well can LLMs write COBOL?
#8I tried to get chatgpt to write 6502 assembler for the 1980s 8-bit home computer the BBC Micro. It was game, but clueless
That's a little surprising; 6502 assembly is a fairly popular hobbyist thing, so I would expect the data is out there. It's also mostly pretty simple, but you do have to watch out for quirks like rotating through the carry bit.
Re: How well can LLMs write COBOL?
#9I suspect there were relatively few training examples for COBOL. It would be interesting to see the results for a system which had a significant number of such examples in the training set.
There is probably a business opportunity for an AI company to build private LLMs for large enterprises trained on their own COBOL code bases. They won't find much available as open source, and individual companies tend to have significantly different coding styles tied to COBOL versions and database schemas.
Re: How well can LLMs write COBOL?
#10I tried to get chatgpt to write 6502 assembler for the 1980s 8-bit home computer the BBC Micro. It was game, but clueless
It depends on what you test for.
I am from the C64 demo scene and in this regard ChatGPT is pretty useless. VIC tricks, raster timing - nothing a LLM can help with at the moment judging from my experience with ChatGPT 4.0 so far.
Same goes for Amiga and simple blitter access for scrolling.
LLMs will be very limited here unless they receive sensory feedback repeatedly.
Pure algorithms like sorting for example may be doable by GPT, but the mentioned machines are very creatively used to come up with effects.
Here are techniques reused, not so much code itself, which gets modified and optimized for every demo and often times recombined with other techniques.
Most techniques for the VIC are pretty well documented, but the timing as well as recombining them is the heavy lifting.