This is my second blog, which has now turned into a series, and while not entirely videogame related, it is inspired by the recent studies and research which have been hitting the front page and various cblogs. I do not claim that any of these studies in particular are well researched nor poorly researched, and am using them more as an example of what to look for when you decide whether a study is believable or not. The main aim of this blog is to look at how to better critique studies and the questions to ask yourself so you can get a better idea what the study is about or where the holes are. For those of you who have done any critical appraisal of scientific papers, a lot of this stuff will be old hat.

This became quite long as I was writing it, so I'm splitting it up into more managable chunks. I don't really want too many tl;dr comments, or wall of text images.

The first part was 'What is the study trying to say, and what does it actually say??'. Now on to the second part.



How do they come to that conclusion??

Inference is the biggest problem when it comes to studies, but the fact remains that correlation is not the same as causation. Just because we can show Nature Park use has decreased as Videogame use has increased does not in fact make them linked.

To take this type of argument to rediculous levels we merely have to pick other things that have been in a constant state of decline or increase since the late 80's. Maybe we could pick AIDS cases in the US, they have been on the increase over a similar time period to what was studied in the Nature Park study. By using the Nature Park logic, we could say that AIDS cases have decreased peoples interest in Nature Parks, or as per the title of this blog, that videogames caused AIDS. Clearly this is unlikely.

Unfortunately, I can only find the abstract to this study, but this is the sort of science that strikes me as non-science. It appears the authors have come into this knowing that nature park attendance is down, formulated some ideas about why it might be, and then developing a questionnaire that 'surprisingly' found exactly what they were looking for. You don't need to be overly intelligent to figure out that videogames or TV watching have increased since the late 80's. If they want to prove this is the cause they need to show a distinct difference between those who played videogames and those who didn't with respect to Nature Park attendance rates. In fact the abstract concludes, all major lines of evidence point to an ongoing and fundamental shift away from nature-based recreation. That is all that has actually been shown.

Unfortunately this now needs to go a little bit into study methods, and types of study, I'll try to make it brief. The best type of study is to take a group of people and split them randomly into two groups, a randomised controlled trial. Due to randomisation, all traits should be similar except for the factor you're trying to control, in this case videogame play vs no videogame play, and then follow these people for a set amount of time, and measure the factor you're interested in, Nature Park attendence. Unfortunately, it is not ethical to do these sort of studies on people, as they generally like to decide for themselves whether or not to play video games.

So we have to fall back on statistically worse studies, which do not correct as well for confounding variables. These are variables that are closely linked to the variable being studied (and not necessarily obviously), which may confuse results. Examples being race and socioeconomic status, or smoking cigarettes and drinking alcohol. These studies are called longitudinal studies. These can either by prospective or retrospective.

In the Nature Park study, they have taken a group of people, and followed them over a period of time collecting information at various points, which is a prospective study. Prospective studies are limited by the way the questionairre is originally set up, prior to the years it will run. If for instance half way through the study period, Hologram Nature Park 2010 came out, making faux Nature Park attendence easy and affordable, and driving real Nature Park business down, you can not just add it into the questionairre as it skews all the results you have collected from previous periods. This is why a lot of studies have pilot studies first. There is also the issue of people knowing what you are studying and modifying their own behaviour. An example would be during a dietary or exercise study, people generally exercise more or eat a better diet in an attempt to please the researcher.

The final study design is retrospective studies, which are worse again than prospective studies. These are studies in which the researcher looks back over time attempting to establish causality from past events. This is where the a lot of the school shooters 'statistics' come from. In particular, it seems a lot of these people have played videogames. This highlights one of the main problems with retrospective studies, researchers can find exactly what they're looking for and generally these are the sort of studies used to further ones own agenda. If we take American adolescents as a population, most studies tend to agree on a figure of around 80-90% of adolecents have exposure to a videogame system. Is it really that surprising that the school shooters have mostly played them?? This is like looking retrospectively and saying, hang on a minute, these shooters mostly ate meat, meat must cause you to become violent, ban meat. Or the most obvious connection, all of these shooters attended school.

Again, this has become more unwieldly than I originally thought, I'll finish this part here, we'll start again tomorrow (or maybe the next day) with the question where does the information come from??

I hope this isn't getting too boring for you all, I appreciate those of you who took the time to read it.