I write code (mostly R) daily to carry out statistical analyses in epidemiology. I generally do not use generative AI to write such code. These are some reasons why.

(For the purposes of this note I am not focusing on the ethical aspects of using generative AI products. Ethical considerations may well be a singularly sufficient reason to not use such generative AI for any given task.)

1 The code base is not large

Programmers and software engineers can make use of agentic AI to create various parts of a code base, e.g. building databases and front ends swiftly and simultaneously. They can then review the code and integrate different parts into the overall system. Comparatively, the code base required for many statistical analyses is not particularly large. Even the more extensive and complex analyses I do require a modest quantity of code. There is little potential benefit of handing an initial code generating step over to an AI agent or chatbot.

2 It is already well abstracted

One reason why statistical analysis code bases are not large is the substantial amount of abstraction that exists. Much of the code I write is “high level” letting packages and functions take care of implementation details. Statistical methods, from the straightforward to the complex, are often only a few lines of code away. Tools like the tidyverse and data.table make data management steps easy and concise (and it’s often easy in base R too). I can also reuse my own abstractions within and between different projects. I don’t need another (stochastic, expensive, unreliable) layer of abstraction between me and the analysis.

3 Every character matters

When I’m not writing high level code, I am in the details. For the details of a statistical analysis, every character matters. A single character written, or not written, has the potential to change a key detail in an analysis which may impact the results. I can’t absolutely trust generative AI. (For example, I have seen generative AI produce absolutely incorrect statements about how missing values were handled in a line of code.) Of course, I would carefully review any analysis code produced by AI, but then it seems I may as well think about and implement the details myself.

4 An analysis plan sufficiently detailed is effectively just writing code

With vague instructions for a statistical analysis, AI will “fill in the gaps” in generated code. This will likely not match the intended analysis precisely. The solution to this would be a very detailed analysis plan, so that the AI is only “translating” from language to (code) language, and making no implicit analysis decisions. But to write such an analysis plan is akin to writing the code itself.

In fact, I can find it helpful to write code (using dummy data) for a pilot analysis in order to review the analysis plan. This forces me to think through crucial details of each data preparation and analysis step, leading to an improved, refined plan for analysis.

5 Writing code deepens understanding

Writing code myself improves my understanding of the code, the programming tools, the data and the analysis. Reviewing AI generated code to understand it is a worthwhile step, but developing understanding can be done simultaneous with writing code.

6 Writing code can be easier than text

I feel it is often the case that I can articulate my thoughts in code (or pseudo code) easier than in natural language. Then when I’m working in a programming language I’m familiar with (such as R) converting this to analysis code is straightforward.