How to Read a Research Article: Part III
This is the third part in a series about understanding research articles. See Part I here.
Method(s)
The method section of the research report should clearly explain how the study was conducted. However, for the layperson, this will be the most difficult section to read. The methods section may be divided into subsequent sub-headers, such as: participants (subjects), equipment, procedure(s), and data analysis.
Participants
This section is presented to give readers a basic overview of the participants and is typically done in terms of age, gender, height, weight and other demographic characteristics of value. Many times participant demographics are presented in the form of a table to enhance viewing. If a particular characteristic (e.g. disease of osteoporosis, status as an elite athlete) is required for inclusion in the study, those criteria for inclusion should be explained. It is preferred that groups of participants are not different (especially significantly) from each other; however, when comparing men and women, many of these differences are inherent.
Selection of participants is very important to the study; how participants are selected and how many are selected can have major impacts on the research quality. Terms like “randomly”, “counter-balanced”, and “cross-over” usually indicate a better research design. “Randomized” means that participants are a representative sample of the larger population and the other two terms show that the research design has controlled for uncontrollable differences in participants. The size of the sample also plays a role in the research quality. Generally, smaller samples provide less knowledge about the general population. While there are statistical analyses to correct for small sample sizes, it is generally thought that individual studies or groups within a study should be no smaller than 8-10 individuals.
In all likelihood, the participant sample will be comprised of individuals who the investigator has easy access too. This is typical of the university study that recruits students. However, the results from a research study are not as easily applied to the general population since the sample was not randomly selected from the general population. The reader should be weary of results that are generalized beyond the sample of the study.
Equipment
The equipment used to provide treatment or measure variables should be described in great detail. While most non-scientific readers will generally ignore this section, those interested in Whole Body Vibration (WBV) will want to be sure to see which vibration machine was used for training or treatment.
Procedure(s)
This section should explain, in detail, how the steps of the study were performed, the order in which they were performed, and how the data were collected. Readers who have a clear idea of how the research was conducted may also have a clear idea of how to apply the results or determine if they can accept the author’s conclusions. Readers should be satisfied the changes noted during the study are the result of the device(s) being studied and not the result of a sloppy procedure.
While this section is very important to researchers, it is another section that is often overlooked by laypeople. The items that non-scientists will want to take note of (specifically those interested in WBV) are specific exercises and vibration characteristics (frequency, amplitude, acceleration/g-force/magnitude/intensity, duration of exposure, number of reps, rest time etc).
Data Analysis
The data analysis section is typically filled with statistical language not understood unless the reader has taken a statistics course. The item that should be paid attention to though, is the p-value. A p-value (or alpha [a]) of 0.05 is most commonly used. This means that there is a 5% chance of error in the results (i.e. the results happened by chance). Whereas, use of a p-value of 0.10 means that there is a greater chance of an error and the use of p=0.01 indicates only 1% error. Most studies will indicate whether or not their results were equal to or less than a certain p-value.
Results
The results section should be solely the results of the data analysis, absent of commentary about those results. This section will typically indicate if a measure was statistically significant (i.e. their results were less than their selected p-value). This is very important; however, some results may statistically significant but not practically or clinically significant.
For example, an increase in vertical jump height of 2 mm is not very practically significant, especially if it is over a long period of time. While there are much more detailed items that could be scrutinized in this section, these are the most important to those without an advanced knowledge of statistics. Non-scientific readers should concentrate on the tables and graphs instead of the statistical jargon in writing. This will help the reader understand the results of the study better without getting lost in the language.
Discussion
In the discussion, the author should cite the main findings if the study and whether or not those findings accepted or rejected the hypothesis. Discussions of the reasons for the results should be extensive and discussion in this section should be tied to the introduction. While reviewing this section, readers should think back to the logic of the arguments presented and consider the issues related to the original problem. Has the author discussed broader implications of their findings? Readers should use this section to able to judge the knowledge and insight of the investigator.
Readers also should be wary of unsupported discussion and conclusions based on insignificant results (e.g. stating that if more data were collected, the results would become significant). Drawing conclusions from future experiments is fraught with suspicious bias. Most research is not of a dramatic, profound, profession-changing nature and usually creates more questions than it answers. Readers must judge if the researcher has conducted fair and objective research.
Since all research has limitations, these should be discussed. Limitations of the study may be presented as an independent section, but are many times also discussed throughout the text of the discussion section. A quality study is always presented with its limitations whether or not they are inherent in the design or procedures.
Lastly, the author should provide a basis on which to apply future research and directions that are important for future research to pursue.
Conclusion
The conclusion section of a research article contains a brief restatement of the experimental results and describes the implications of the study. The conclusions should not be taken outside of the context of the study. Most research studies are very specific in what they are looking at, so application to large-picture ideas is difficult. Remember that the reader’s understanding of the conclusions should be related to the study design, results, and discussion. An attempt to be a neutral reader should be made so that preconceived notions have limited affect on interpretation.
Happy Reading!
References
1. Lunsford, T.R. and Lunsford, B.R. (1996) Research Forum—How to Critically Read a Journal Research Article. Journal of Prosthetics & Orthotics 8(1):24-31 (Accessed at http://www.oandp.org/jpo/library/1996_01_024.asp).
2. Accessed at http://unilearning.uow.edu.au/reading/1d.html.
3. Accessed at http://www.brockport.edu/sociology/journal.html.
Written by David M. Bazett-Jones on August 29th, 2007
14 Comments
- John Weatherly
August 31st, 2007I agree with Lloyd. As I mentioned in a post on the prior section (Part 2), I haven’t seen ANY studies validating vibration equipment. Shouldn’t researchers make sure the equipment they use is actually measuring what they think it is measuring before they do a study? And these are people (at least the majority of them) with PhDs?????
Not a single study has investigated the exact variances involved ( I start at 9 with a basic negative response platform ) in the actual vibration. True it is not their job to actually build the machines but understanding these variances would allow them to create true theoreticals . Without these I fail to see how the research can be replicated or move forward.
I have been doing this from day one and could never have built better machines or matching programs without them.
And an obvious question..
Why don’t I just give them the answers ?
(1) I believe they should be at a certain level of understanding BEFORE they start playing with this technology. I could never trust a researcher that found it accepatable to start off any other way.
(2) It could and would be misused by marketers and maunfacturors who want to replicate without understanding , which I see as like giving a gun to a monkey.
- parisa
September 21st, 2007effected vibration training on reduce body fat. tank you
efected vibration training on body fat. tanks
Just received an email from David Bazzett-Jones full of statements like this…
“I would have given… Power Plate …my data if they requested it ”
” I never reported to … Power Plate…. my findings as I was never required to do so”
” I have no control over anything ”
“I am a researcher, not a product tester”
In response to my question on why he didn’t live up to his promise to readers of this site, of approaching Power Plate regarding his machine not coming up to spec on his own tests.
He goes on to suggest if I threw money at him and his mates I would somehow become more ethical and usefull like Power Plate.
I said thanks but no thanks.
Nothing more to say besides , that’s how Power Plate get away with lying to the public time and time again. With help . It’s a shame but its an economic reality I suppose.
On the brighter side I can use this as a prime example on my new IVTRB website to show the public how the truth is kept from them ,and why its so important for us to fight against corruption.
I know it sounds as though I am angry at David, but the truth is I am just upset.
Upset because I believe what we are doing can help a lot of people and this is a definitive moment in the way we look after our health. What has just happened does not help us , it makes the task just that little bit harder.
I know some people in certain positions do not understand this yet , but later on when all the bad things Power Plate and other companies like it have done is common knowledge, they will know they helped them in their own small way. Because without people ” just by not saying something” companies would be forced to front up and start being honest.
The David Bazzett- Jones research story so far for those of you new to this forum…….
Comments made by David Bazzett-Jones on vibrationtraining.net over a number of days regarding his views on valid and repeatable research. Please read them carefully so you obtain a clear idea of his opinions.
“That is why I believe that research should be very descriptive the settings that they are using. In addition, the vibration acceleration *should* be measured independent of the manufacturer’s report (my idealism). Also, product testing must be done by independent laboratories to give both the research community and consumers the knowledge to be able to choose quality machines that will give consistent results.”
“As far as having “engineering reports” done on machines, I would argue that it is the ethical responsibility of the manufacturer to do this testing and report the results. The blame is not on the researchers but the manufactures ”
“As far as my ( Power Plate ) study in 2005, we only measured the accelerations (which were different than the manufacturer’s claimed accelerations). This was done without an individual standing on the plate.”
” I also feel that there are some researchers (myself included) that would like to perform product testing so that consumers can be informed of the true specs.”
” I own the results and will publish them how I like, whether or not Power Plate likes the results or not.”
“We are testing it loaded and unloaded. Believe it or not, you are not the only one who has wondered if the Power Plate, being made of a softer material than steel, affects the vibration characteristics.”
“I agree that all plates should be tested before (and while, to go above and beyond) they are being used for a research study…..I can only wish that I thought of this prior to doing this study”
“I am more concerned about the errors/limitations of my research than you are. This was years of planning, testing, and analyzing that I put in to this project. I could not account for limitations in the study that I was not aware of at that time.”
“Just because a bit of research was done on a “faulty unit” doesn’t mean that the research is worthless.”
During these conversations we made it clear that our main concern was that even though David clearly was not happy at present he had been lied to ,in time this fact would be buried with no strong recommendations given to Power Plate . That Power Plate would in fact never be taken to task over this recurring unethical behavior and other researchers would find themselves in the same situation if nothing was done. They would do this by dangling grant money in front of researchers to keep them quite.
David said he was not like that , the truth would be told loud and clear and we could trust him, he even added I could trust him enough to share my work with him……..
“I have lots of room in my e-mail and would love to go over your equations and theories. I will even sign a contract that I will not duplicate or share them”
I told him I would wait and see if he could be trusted , actions speak loader than words.
But recently……….
A comment by a top Power Plate representative was made that made us think David had indeed not reported the problem ……….
Scott Hopson
Director of Power Plate Education and Research
07-18-2008“I can even tell you that I spent a couple of years in great communication with David Bazzett Jones and not once has he reported such unethical concerns. Nowhere in his paper did it mention product fault or incorrect magnitudes of force, etc.”
He added….
” Indeed David has continuously expressed gratitude for the product, support and communication with Power Plate - even requesting more support as his studies progress.”( Eg…more grant money )
After contacting David about this apparent about face on the subject this is what was said……..
“I would have given… Power Plate …my data if they requested it ”
” I never reported to … Power Plate…. my findings as I was never required to do so”
” I have no control over anything ”
“I am a researcher, not a product tester”
After reading his paper at http://www.jssm.org/vol7/n1/21/v7n1-21pdf.pdf
He has indeed listed the machine as working correctly at 30Hz 35Hz 40Hz and 50Hz he even says they are ” defined by the manufacturers settings ” No mention of the variations in advertised specs he promised to report on . He does mention g-forces used in the tests but does not mention that they do not match Power Plates advertised g-force ratings .
I will leave it up to the reader to come to their own conclusion as to what happened . All I know is the truth is still not being told by the people we trust to tell it.
P.S. For those of you who find this conversation controversial , David came on here voluntarily and engaged us in a debate on this exact subject. So it is only right it plays out in public on the same site.
- Ritchie
August 28th, 2008This is the problem with our industry, when you have people like David, (who could choose to be a pioneer), choosing a path like this). Im disappointed because the industry needs not just good companies but also good researchers. I feel there will come a day when he will regret this decision.
- Ritchie
August 28th, 2008Another thing that disappoints me is that David is such an intelligent person, yet he’s sold out.
Reading the research paper several times, I have so many questions to ask David. I wonder if he’ll come back in here as I thought he would when the study was published. Also I couldn’t find the number of days/times the tests were done.
David has obviously focused on the human involvement (movement, warmup etc) and has (conveniently?) overlooked the machine discrepancies. It does seem that they don’t matter for the purpose and results of his study BUT he did tell us that they mattered to him and that machine specs need to be as advertised by the manufacturer/marketer and that he would like be a product tester.
Reading your correspondence Lloyd and the study article, David seems to have sold out and just got on with his tests with considerable loyalty to those who are supporting him financially (PowerPlate). This is very sad after the time he spent on here saying he was aware of discrepancies and also being able to read here of other problems. David, we’d love to have some answers from you.
Di , why do you think he didnt voluntarily come on here to tell us he had released his paper. If he was proud of his actions all we would have heard is..
” I told you I could be trusted ”
He opened up to us once before with the truth and look at the position he is in now. On top of that in his e-mail to me he replied to my request for him to come on here and explain….
“I find it interesting that you called VT.net “our” site since it seems from observation that you are in control of the site content”
Because he lives in a currupt world he is convinced everybody is the same.
Well Lloyd, I know that you aren’t in control of this site’s content as some of my comments, over time, have been moderated and you didn’t even know I’d made them. Articles? Well anyone can submit those, and not to you, to the site owners/managers. You are “loud and persuasive” at times but most of us wouldn’t have it any other way as we’ve learnt to trust your judgement even if, sometimes at first, we don’t think we agree with you. David will find this all out, in the end.
Aside from that I wish he would come back as I have a few questions about his study, from a “laymans” understanding, like how many times was the test performed as I didn’t see that in the paper; maybe I missed it. Also why he chose 45 seconds as the test time, which isn’t the 30 sec or 60 sec that PP and others use in studios and home use. Also why dynamic tests when he must have read on here and other sites and studies that most “work” on Vibration platforms is static. Why he used a warm-up, as most people can jump without a warm-up and the Vibration Platform acts as a warmup for muscles anyway. I wonder if the warm-up helped the pre-vibration jumps and degraded the post-vibration jumps. But hey, I’m not an academic, even if I was awarded an honorary doctorate once, (just in fun, not real).
August 29th, 2007
I find this statement ambiguous and primed for abuse.
“The equipment used to provide treatment or measure variables should be described in great detail. While most non-scientific readers will generally ignore this section, those interested in Whole Body Vibration (WBV) will want to be sure to see which vibration machine was used for training or treatment.”
This does not go anywhere near far enough, with the name of the unit not as important as exact specs and complete desciption of the unit.