Before, only the senior cobol programmers at the company understood and knew the codebase. Now, no one does. I can see the allure of moving away from a legacy cobol system. But an AI rewrite doesn't actually solve any of the issues with having a legacy cobol codebase. You just have a new system no one knows or understands.
Wasn't COBOL invented to read like business prose and be easily understood by programmers and non-programmers alike? If developers are able to write COBOL code no one understands than it failed its main objective. IMHO every large software system should be developed on the basis of clear and concise functional specifications so everyone can understand what the software does and how it does so at a higher functional l…
AI migrated legacy COBOL programs to Java, bugs included
81–90 of 108 posts
Re: AI migrated legacy COBOL programs to Java, bugs included
#82Earlier quoted context omitted.
Wow, their Java code example [0] makes COBOL look elegant and terse by comparison. [0] https://community.ibm.com/community/user/blogs/gregory-cerne...
I know it's not the only problem (but it is IMO the main one), but this is why I'm a proponent of allowing operator overloading in languages. Yes, you can't tell at a glance if `+` really still does what it should, but I find that problem no different from a library mis-naming a function and/or said function having strange side-effects. `+` should do addition, period . No, not even concatenation if possible (I do fin…
Re: AI migrated legacy COBOL programs to Java, bugs included
#83The biggest problem is not that bugs are migrated with COBOL, but that lots of new bugs are going to be introduced. AI is not deterministic, it will be making tons of mistakes. The only realistic low-error approach is incremental step-by-step migration using Cursor or similar tools. However, it requires much more time as each step must be prompted, tested and committed manually. Any hope that one-shot migraton of a l…
> AI is not deterministic, it will be making tons of mistakes. Most compilers are also not deterministic, at least by default, but they don't usually make mistakes. Determinism isn't an important quality here. And if it were, AI can be deterministic, it just isn't normally for much the same reason compilers typically aren't (hint: performance). I find it bizarre that I keep reading this here. Just one of those things…
can you get some arbitrary number of 9s for consistency? Yea. You can’t get 100% deterministic though.
Re: AI migrated legacy COBOL programs to Java, bugs included
#84Earlier quoted context omitted.
> AI is not deterministic, it will be making tons of mistakes. Most compilers are also not deterministic, at least by default, but they don't usually make mistakes. Determinism isn't an important quality here. And if it were, AI can be deterministic, it just isn't normally for much the same reason compilers typically aren't (hint: performance). I find it bizarre that I keep reading this here. Just one of those things…
They cannot be made entirely deterministic. You have cases where the next set of token choices have the same probability, and most people also are using cloud provider served models which can be served across different hardware implementations which can modify the math used in running the model. can you get some arbitrary number of 9s for consistency? Yea. You can’t get 100% deterministic though.
Computers are deterministic. When we talk about non-determinism we're just talking about where there are hidden inputs, but the fundamentals of computing means that all hidden inputs can become visible if you want them to be (although possibly at the cost of things like performance), so, yes, you can reach 100% determinism just fine if you wish to. It's just not worth the tradeoffs in most cases. But if a deterministic solution solved a problem here it would be worth it.
However, a deterministic LLM doesn't help here. LLMs don't "make mistakes" because they are typically non-deterministic. They would equally "make mistakes" when deterministic.
Re: AI migrated legacy COBOL programs to Java, bugs included
#85The biggest problem is not that bugs are migrated with COBOL, but that lots of new bugs are going to be introduced. AI is not deterministic, it will be making tons of mistakes. The only realistic low-error approach is incremental step-by-step migration using Cursor or similar tools. However, it requires much more time as each step must be prompted, tested and committed manually. Any hope that one-shot migraton of a l…
> AI is not deterministic, it will be making tons of mistakes. From the paper: > The COBOL source is passed through an internal deterministic Migrator to produce a generated Java target. Also, humans are not deterministic either. Give the same COBOL -> Java translation to multiple developers and each will come up with a different solution. Heck, even the same developer will produce a different output for the same tas…
Re: AI migrated legacy COBOL programs to Java, bugs included
#86The biggest problem is not that bugs are migrated with COBOL, but that lots of new bugs are going to be introduced. AI is not deterministic, it will be making tons of mistakes. The only realistic low-error approach is incremental step-by-step migration using Cursor or similar tools. However, it requires much more time as each step must be prompted, tested and committed manually. Any hope that one-shot migraton of a l…
AI doesn't need to be deterministic.
As a side note, I can't fathom why they chose Java. It's easier to teach COBOL to a competent engineer than to rewrite everything in a language that encourages onion architectures. Why not Go, for example?
Re: AI migrated legacy COBOL programs to Java, bugs included
#87This sounds like what the company Mechanical Orchard offers: https://www.mechanical-orchard.com/ Their pitch is like, "we take your COBOL code, and all the real-world data you can give us. Then we model your COBOL program as a graph, where each node has inputs and outputs. Then we use AI to port each node, making sure that it has the same (input => output) mapping for all the test data you gave us."
This is the secret sauce. You'll be responsible for any defects if you didn't prepare your test data to act like a massive unit test. Wonder how hard is to prepare the ideal test data vs writing the unit tests themselves in COBOL and verifying their translation. I guess if you throw enough data at it, the effort is minimal while the coverage becomes good enough to fix any bugs manually.
Re: AI migrated legacy COBOL programs to Java, bugs included
#88Before, only the senior cobol programmers at the company understood and knew the codebase. Now, no one does. I can see the allure of moving away from a legacy cobol system. But an AI rewrite doesn't actually solve any of the issues with having a legacy cobol codebase. You just have a new system no one knows or understands.
Re: AI migrated legacy COBOL programs to Java, bugs included
#89Earlier quoted context omitted.
You don’t need the entire codebase in context in every moment to migrate it. Also AI being non deterministic does not prevent it from one-shotting perfect solutions 100% of the time for simple enough problems. And every model generation brings this bar higher. So that’s really not a fundamental problem. And we can also implement llm inference deterministically if we want, it’s just that it’s not worth the loss in per…
> And we can also implement llm inference deterministically if we want, it’s just that it’s not worth the loss in performance to do it. My understanding is thats not possible (different from being practical), wonder if you have any literature, research to back up that claim?
Re: AI migrated legacy COBOL programs to Java, bugs included
#90COBOL will never die. Whatever this is will only result in more COBOL being written.
I can see it. "Let's just do this in COBOL for now to get it working. Add a ticket to use AI to migrate it to Java in the backlog."