Survey of editors and reviewers of high-impact psychology journals: statistical and research design problems in submitted manuscripts. 2011

Alex Harris, and Rachelle Reeder, and Jenny Hyun
Veterans Affairs Palo Alto Health Care System, Center for Health Care Evaluation, Willow Road (MPD-152), Menlo Park, CA 94025, USA. alexander.harris2@va.gov

The authors surveyed 21 editors and reviewers from major psychology journals to identify and describe the statistical and design errors they encounter most often and to get their advice regarding prevention of these problems. Content analysis of the text responses revealed themes in 3 major areas: (a) problems with research design and reporting (e.g., lack of an a priori power analysis, lack of congruence between research questions and study design/analysis, failure to adequately describe statistical procedures); (b) inappropriate data analysis (e.g., improper use of analysis of variance, too many statistical tests without adjustments, inadequate strategy for addressing missing data); and (c) misinterpretation of results. If researchers attended to these common methodological and analytic issues, the scientific quality of manuscripts submitted to high-impact psychology journals might be significantly improved.

UI MeSH Term Description Entries
D008373 Manuscripts as Topic Works about compositions written by hand, as one written before the invention or adoption of printing.
D010506 Periodicals as Topic Works about publications issued at stated, more or less regular, intervals. Journals as Topic,Magazines,Newsletters,Magazine,Newsletter
D011584 Psychology The science dealing with the study of mental processes and behavior in man and animals. Factors, Psychological,Psychological Factors,Psychological Side Effects,Psychologists,Psychosocial Factors,Side Effects, Psychological,Factor, Psychological,Factor, Psychosocial,Factors, Psychosocial,Psychological Factor,Psychological Side Effect,Psychologist,Psychosocial Factor,Side Effect, Psychological
D011786 Quality Control A system for verifying and maintaining a desired level of quality in a product or process by careful planning, use of proper equipment, continued inspection, and corrective action as required. (Random House Unabridged Dictionary, 2d ed) Control, Quality,Controls, Quality,Quality Controls
D012107 Research Design A plan for collecting and utilizing data so that desired information can be obtained with sufficient precision or so that an hypothesis can be tested properly. Experimental Design,Data Adjustment,Data Reporting,Design, Experimental,Designs, Experimental,Error Sources,Experimental Designs,Matched Groups,Methodology, Research,Problem Formulation,Research Methodology,Research Proposal,Research Strategy,Research Technics,Research Techniques,Scoring Methods,Adjustment, Data,Adjustments, Data,Data Adjustments,Design, Research,Designs, Research,Error Source,Formulation, Problem,Formulations, Problem,Group, Matched,Groups, Matched,Matched Group,Method, Scoring,Methods, Scoring,Problem Formulations,Proposal, Research,Proposals, Research,Reporting, Data,Research Designs,Research Proposals,Research Strategies,Research Technic,Research Technique,Scoring Method,Source, Error,Sources, Error,Strategies, Research,Strategy, Research,Technic, Research,Technics, Research,Technique, Research,Techniques, Research
D003625 Data Collection Systematic gathering of data for a particular purpose from various sources, including questionnaires, interviews, observation, existing records, and electronic devices. The process is usually preliminary to statistical analysis of the data. Data Collection Methods,Dual Data Collection,Collection Method, Data,Collection Methods, Data,Collection, Data,Collection, Dual Data,Data Collection Method,Method, Data Collection,Methods, Data Collection
D003627 Data Interpretation, Statistical Application of statistical procedures to analyze specific observed or assumed facts from a particular study. Data Analysis, Statistical,Data Interpretations, Statistical,Interpretation, Statistical Data,Statistical Data Analysis,Statistical Data Interpretation,Analyses, Statistical Data,Analysis, Statistical Data,Data Analyses, Statistical,Interpretations, Statistical Data,Statistical Data Analyses,Statistical Data Interpretations
D006801 Humans Members of the species Homo sapiens. Homo sapiens,Man (Taxonomy),Human,Man, Modern,Modern Man
D017712 Peer Review, Research The evaluation by experts of the quality and pertinence of research or research proposals of other experts in the same field. Peer review is used by editors in deciding which submissions warrant publication, by granting agencies to determine which proposals should be funded, and by academic institutions in tenure decisions. Grants, Peer Review,Peer Review, Grants,Peer Review, Publishing,Grant, Peer Review,Grants Peer Review,Grants Peer Reviews,Peer Review Grant,Peer Review Grants,Peer Reviews, Grants,Peer Reviews, Publishing,Peer Reviews, Research,Publishing Peer Review,Publishing Peer Reviews,Research Peer Review,Research Peer Reviews,Review Grant, Peer,Review Grants, Peer,Review, Grants Peer,Review, Publishing Peer,Review, Research Peer,Reviews, Grants Peer,Reviews, Publishing Peer,Reviews, Research Peer
D055811 Journal Impact Factor A quantitative measure of the frequency on average with which articles in a journal have been cited in a given period of time. Factor, Journal Impact,Factors, Journal Impact,Impact Factor, Journal,Impact Factors, Journal,Journal Impact Factors

Related Publications

Alex Harris, and Rachelle Reeder, and Jenny Hyun
May 2003, Journal of wound care,
Alex Harris, and Rachelle Reeder, and Jenny Hyun
June 1998, Revista espanola de cardiologia,
Alex Harris, and Rachelle Reeder, and Jenny Hyun
January 1991, Revue des maladies respiratoires,
Alex Harris, and Rachelle Reeder, and Jenny Hyun
January 1998, Revista de gastroenterologia de Mexico,
Alex Harris, and Rachelle Reeder, and Jenny Hyun
January 1988, Revista de gastroenterologia de Mexico,
Alex Harris, and Rachelle Reeder, and Jenny Hyun
March 2008, Aesthetic plastic surgery,
Alex Harris, and Rachelle Reeder, and Jenny Hyun
January 2009, Enfermeria clinica,
Alex Harris, and Rachelle Reeder, and Jenny Hyun
February 1991, BMJ (Clinical research ed.),
Alex Harris, and Rachelle Reeder, and Jenny Hyun
June 1982, British medical journal (Clinical research ed.),
Copied contents to your clipboard!