Psychological Network Analysis is Not a Predictive Method



The Amazing Capacity of Vision

Most of the time, I talk with other clinical psychologists. However, every once in a while, I get the chance to talk to a vision scientist. If you get the chance to do the same, I highly recommend it — you might be surprised what you hear.

Vision scientists tend not to see vision as something “real”. This can be jarring for the rest of us, but hear me out — I think they make some good points. A vision scientist will explain that the information entering your eyeballs daily is absurdly complex — somewhere around the magnitude of 10^14 photons per second — and what we actually see is surprisingly unrepresentative of the raw information.

Instead, what we see has a lot to do with heuristics that are very important to human survival. We are good at seeing edges and movement. We are good at parsing edges into objects. We are even quite good at gauging the depth of these objects relative to one another. Amazingly, we are pretty good at parsing these objects in terms of their threat, reward, and even their social relationships to one another– all before we’ve even realized that we’ve seen them.

What does this anecdote have to do with anything? Well, vision is an amazing capacity for understanding. The amount of data you have taken in through your eyes today is extraordinary, yet you understood it seamlessly. If you’re even remotely interested in the acquisition or the production of knowledge, you should care very much how you managed to do this.

Vision is not the truth — vision is better than the truth. It is an understandable approximation of the truth, and remarkably unlike the raw truth, it is actually useful to us.

Predictive and Explanatory Statistics

Most of the time, I talk with other clinical psychologists. However, every once in a while, I get the chance to talk to a statistician or data scientist. If you get the chance to do the same, I highly recommend it — you might be surprised what you hear.

Statisticians tend not to see statistics as something “real”. This can be jarring for the rest of us who work in applied settings. Statisticians know — have always known — that statistics are just something humans made up to roughly force complex datasets into our very limited human brains.

According to Yarkoni & Westfall, statistical methods fall roughly into two camps: predictive and explanatory. Predictive methods are good at turning inputs into powerful predictions, but they are often opaque to human understanding. On the other hand, explanatory methods enhance human understanding, but often possess relatively limited predictive power.

Psychological Networks are Explanatory, not Predictive

I have thought long and hard about the issue, and feel quite certain that psychological networks fall firmly in the explanatory camp. Why do I think so? There are a few reasons:

  • When I talk to experts in predictive methods (i.e., machine learning), they generally do not consider network analysis to be a predictive method
  • When compared to other statistical benchmarks in the literature, psychological networks aren’t great predictors
  • Psychological networks are highly interpretable. Most of the parameters estimated are of the same type. This makes them very easy to understand, and also very inflexible. This is a hallmark of a good explanatory model (and a limited predictive model).
  • When networks are predictive in other fields, it’s usually because they have a near 1:1 measurement connection with reality (for example, physical electrical networks). In these cases, networks are both explanatory and predictive, not just predictive.
  • Psychological networks were devised as a way for scientists to understand psychological phenomena.
  • Researchers don’t use psychological network analysis like a predictive method. Allow me to explain what I mean. When a machine learner wants to predict an outcome, they typically engage in a workflow of trying out 5-10 different types of models, cross-validating, and deciding which model has the best out-of-sample prediction. They have not a care in the world for which model is most accurate to the real-world phenomenon, as long as it is predictive. In contrast, researchers who use network analysis do so specifically because they believe that the model is a good way of understanding the phenomenon they study.
  • Researchers who use psychological network analyses care a lot about ‘getting it right’. They worry about assumptions that might cause misinterpretation. This assumes of course that interpretation is one of the primary goals.

Good Explanatory Statistics are Overly Simplistic

Just like good vision, explanatory statistics are essentially heuristics that help translate the complexity of the world into something that humans can understand and care about.

Just like vision, explanatory statistics are not the truth. Just like vision, good statistics are better than the truth — they are an understandable approximation of the truth.

Something that most good statisticians tend to appreciate is that over-simplicity is a really great thing. Statisticians appreciate this because they understand that all statistics are an oversimplification — and this means that if you care about explanation, a fluently understandable model that explains 50% of the variance is much better than a mumbo-jumbo-model that explains 55% (for explanation).

To the Point: Network Models

I think that psychometric networks are a wonderful innovation that will play a huge role in advancing psychology, particularly clinical psychology.

There are lots of others who agree with me about psychological networks — perhaps yourself– but I’m afraid that they agree with me for the wrong reasons. They might think that they are going to use EMA data to make networks that precisely predict outcomes. They might think that as soon as we are able to deal with X assumption or Y assumption, we will have a model that matches reality. Even more problematic, they might think that network models are the best way to make treatment decisions or to predict important clinical outcomes.

I believe that many of these thoughts are based in the assumption that we can have our cake and eat it too: have an explanatory model that also provides the best predictions. I’m doubtful.

Psychological networks are wonderful, but I think they are wonderful mostly for their explanatory power, not their predictive power.

Psychological networks are just not good predictive models. In the vast majority of cases, they are just far, far, too simplistic and rigid to every compete with some of the powerhouses in modern machine learning — random forests, neural nets, bagging, boosting, etc. (with a million more well on the way).

But network models are far superior to all of these predictive methods in one very important way — they can make pretty OK predictions, and we can understand why they are making these pretty OK predictions. This makes them extremely valuable. Because many of us scientists care quite a bit about prediction — but we also care quite a bit about explanation.

Far after the fanciest new predictive methods of today are considered outdated, linear regression will continue to be used by those who care about explanation. I hope that network analysis can be there alongside it.

Using Networks for Explanation

I had a very jarring recent experience when a family member asked me about one of my projects. They asked me what the main takeaways were, and I forgot. After hours of building complex models, computing centrality statistics, and so forth, I literally forgot the results I came up with. And if I’m forgetting, what is the chance that someone else will read my article, remember better than I did, and actually use that knowledge to produce some real-world good? Unfortunately for me, not good. I focused on complexity, and I focused on accuracy, but I forgot to focus on explanation.

Network models are heuristics. Network models are explanatory. And explanatory heuristics are only useful when they are easily understood and applied.

Something science journalists tend to realize but actual scientists tend to forget is that humans just can’t deal well with complexity. We don’t have the time to understand everything. We don’t have the energy to understand everything. In fact, in my experience, when our models become complex enough, even we ourselves don’t take the time to really understand them.

I want to conclude with two suggestions:

  1. If you care mostly about predicting things, consider branching out from network models. Seriously. There are much better predictive models available. Let a computer crunch your data and classify your decision for you. And if you absolutely have to use network models, at least settle on 1 outcome you care about, compare with non-network approaches, and take some super basic steps such as cross-validation.
  2. If you care mostly about explaining things, reevaluate the methods you’re using to explain. How you’re writing. Complexity can be valuable, but humans have limits. If you’re going to explain, it’s worth first making sure that there is someone who cares enough to take the time to understand.



Source link


Please enter your comment!
Please enter your name here