A data scientist and her secret love affair with testing

Data scientists come from diverse backgrounds — they can be software engineers, mathematicians, physicists, economists, etc. In general, they often come from a research environment, and therefore, data science projects are sometimes understood and treated more as research than software development projects. This fact has many pros but also some cons. One of the negative aspects is that data scientists in many cases do not follow standard software engineering practices, like code reviews or testing.

So, let’s start with testing — I personally was always kind of indifferent when it came to testing. I tested during my software engineering career, did not test during my research career, whatever was needed at the time.

Recently, I came to the conclusion that testing in data science projects is even more important and should be much more sophisticated than I imagined. I will tell you my reasons, but first, let’s speak about what tests I am talking about.

There are several methods and forms of testing data scientists can do. But in this article, I would like to mention only three, which can help with the cases described below.

The first of them are integration tests. With integration tests, we are testing how well our model works within the whole project. If the piece of the puzzle works well in regards to the whole picture and does not break something.

Then we have unit tests. Unit tests are testing separately each function or logical unit.

The last ones are functional tests that we use for checking the functionality. With these tests, we validate if the model gives expected results. And why you ask are these tests are so important?

Data science projects are not only about well-functioning code, but also good numerical results. And many times, I have seen that a small change in an external or internal library caused a slight numeric change in the model. Or somebody changed a parameter in the code, resulting in different results, but nobody noticed. Functional tests greatly help as they can tell you that the results do not equal anymore to what you expect. Or even when you know you need to update part of code, plug a new library or just refactor your code, it is always good to have integration/functional/unit tests there to tell you if something is broken.
Functional tests can also help you with the optimization of the model. A lot of us are optimizing in quite a suboptimal way, writing code in a notebook, rewriting a parameter, and hitting run, over and over. The tests can be rerun with different parameters and therefore serve for automatic optimization.
As in many other projects, tests are a great form of documentation. Unlike written documentation, they have to be up-to-date. And it is much easier to understand code or return to what you did if you have some examples of how the code works.
Testing can also give us a perspective on the given problem. Personally, it always forces me to think about the flaws of the project. When I write unit tests, I always write them with the mindset of “how could I break this model”? And this kind of change in thinking can catch quite a number of flaws.

Hopefully, I’ve made a case for writing tests in data science projects. But keep in mind that tests are not a one-time thing.

To serve their purpose well, they need to be kept up-to-date, extended as the functionality grows, and run regularly, ideally at each push to a version control system.

Thank you :)

A data scientist and her secret love affair with testing was originally published in ableneo Technology on Medium, where people are continuing the conversation by highlighting and responding to this story.