Back in March 2023 I wrote the following blog post about using Generative AI and Large Language Models (LLMs) to write code: Are Chat-GPT and Google Bard The New Frontier For Writing DSP Code?
Since then, I have used these tools in many projects and have made a number of observations. In general, the more complex the task you are setting for the LLM, the more likely the performance of each is going to diverge and also the more likely it is that, as a programmer, you are going to have to test the code extensively to find the bugs.
Using these tools is a bit like an artist generating a preliminary sketch, rather than the final polished painting, with all of the correct detail.
I have found three main uses for Generative AI in coding:
- Writing code to meet a specification
- Documenting / commenting existing code
- Converting code from one language / system to another
I have tried all of the following: Gemini, Google Code Assist, Chat-GPT, Bing and Co-Pilot. I have had the best coding results with Gemini (Bard) however if I find that this is struggling then I will try them all because they all have strengths and weaknesses.
It is important that you know what you want to do because there is no guarantee you will receive a correct answer!
A useful trick is to try the same request multiple times because, unlike a traditional search engine, an LLM with give you a different responses each time. Handily, Gemini automatically generates 3 draft solutions and you can click on the tabs provide to review each.
I have observed that LLMs are much better at writing Python than lower level languages (C/C++ etc.). In Python, it will almost certainly produce a working solution using the Numpy/Scipy library functions, that may just need some final tuning.
If you are writing code for a lower level language then the best option is often to take a two stage approach:
- Generate Python/Numpy/Scipy code
- Convert the Python code to C - LLMs are very good at converting Numpy/Scipy functions to C
Generative AI is very good for converting between languages and Gemini will add comments to code that does not contain original comments. This is particularly useful if you work with a colleague who is not very dilligent with their code commenting ;-). It is worth noting, however, that comments are sometimes wrong due to AI misunderstanding the intention of the code.
Sometimes the conversion process will skip complex code sections, in a program, entirely so if this happens then the next step is to copy those sections and convert them separately.
Converting code from Python to C/C++ is generally very easy because they both use 0-based array indexing. Matlab, however, is more complex because it uses 1-based array indexing and this confuses the LLM. When converting Matlab code to Python or C/C+++ then I generally use the following request, which I then follow with the code section:
convert the following matlab code, with 1 based array indexing, to Python and Numpy, with 0 based array indexing
One final example of a gotcha is that Matlab uses FIR filter order whereas Scipy uses the number of coefficients.
As well as documenting code, LLMs are very good at debugging code however it is often important to explicitly specify the language in the request, rather than leaving it to the LLM to decide what language the code is written in.
Finally, Test! Test! Test!
Copyright © 2024 Delta Numerix
