Sociology 331 Research Methods: Evaluation of a Peer-Reviewed Article, assignment help

Note: the instructions Below

Requirements:

Your evaluation should include

–  ALL sections 1-5,

–  1 section from 6-10 (based on design in your article), and

–  1 section from 11-12 (based on analysis in your article).

SOCIOLOGY
331

RESEARCH METHODS

HW1:
EVALUATION OF A PEER-REVIEWED ARTICLE (25 points):

Due
no later than 11:55p ET on Sunday of Week 3 – submit using link in Sakai AND to
turnitin.com

PURPOSE: The purpose of exercise is to conduct
a detailed, critical evaluation of
the research design, methods and analysis of a study written up and published
in a peer-reviewed journal.  Students
will be using Wolfer’s (2007) critical questions for evaluating written
research to evaluate an article; these questions are provided below.  Tips on Finding a Peer Reviewed Journal
Article may be found under Lessons>Course Materials>Resources.

Article selection tips:

• select an
  article from a peer-reviewed Sociology Journal

• the
  article should clearly be written about a study the author conducted

  • a
    meta-analysis of multiple studies is not appropriate for this assignment

  • secondary
    data analysis of an existing national data set is OK, but be sure you
    understand what you are reading

    Requirements:

Your evaluation should
include

– 
ALL sections 1-5,

– 
1 section from 6-10 (based on design in
your article), and

– 
1 section from 11-12 (based on analysis
in your article).

This assignment does NOT
have to be written in essay format.  You
may organize it by section/chapter number, question number with question text,
and then your answer.

APA formatting should be
used throughout.  Any time your
paraphrase or directly quote a source (such as your article), in-text citations
should be used.  A full APA-formatted reference
should be included at the beginning or end of assignment.  See Course Materials>Resources for APA
Tips.

1.Title (3 pts)

1. Is the title specific enough to differentiate it from
   other related topics?

2. Do subtitles, if present, provide important information
   regarding the research?

3. Are the main variables expressed in the title?

4. Are the terms in the title easily understood by most
   people?

5. Does the title avoid any reference to the study’s
   results?

6. Overall, is this a good title?  Why or why not?

   2. Ethical Evaluation
   (2.5 pts)

7. Are the steps the researcher took to honor ethical
   responsibilities to individuals clear? 
   Are they appropriate? Are they enough?

8. If there were any findings (based on your readings of
   tables or other means of data presentation) that refuted the researcher’s
   hypothesis, did he address these findings?

9. If any results were unexpected, did the researcher
   discuss any explanations for the unexpected effects?

10. Did
    the researcher adequately acknowledge the limitations of the research?

11. Overall,
    has the researcher adequately fulfilled his ethical obligations?

    3. Literature Review
    (4 pts)

12. Is
    the material presented in the literature review relevant to your research
    interests?

13. Is
    the special problem area identified in the first paragraph or two of the
    report?

14. Does
    the researcher establish the importance of the research problem?

15. Has
    the researcher been appropriately selective in deciding what studies to include
    in the literature review?

16. Is
    the research cited recent?

17. Is
    the literature review critical?

18. Is
    the researcher clear as to what is research, theory and opinion?

19. Overall,
    do you think this is an adequate literature review?  Why or why not?

    4. Operationalization
    and Measurement (5.5 pts)

20. Is
    the conceptualization suitably specific?

21. Are
    the definitions productive?

22. How
    many different dimensions are being measured at once?

23. Are
    the various dimensions sufficient?

24. Are
    the actual questions (or a sample of them) provided?

25. Is
    the response format clear, or, when not already clear, does the researcher
    provide information on the response format? 
    Is there any information on restrictions in respondents’ responses?

26. If
    the researcher is using a published instrument, does he or she cite sources
    where additional information can be found?

27. Has
    the researcher avoided overstating the preciseness of the measurement?

28. Does
    the researcher provide some measure of reliability?  What type of reliability is established?  Do the measures indicate adequate reliability
    for your purposes?

29. Does
    the research provide some measure of validity? 
    What measures of validity are presented and are they adequate for your
    purposes?

30. Overall,
    is the measurement appropriate and adequate given the research purpose?

    5. Sample Strategy (3
    pts)

31. Does
    the research goal lend itself to generalization? Is the broad sampling method
    appropriate for the research goal?

32. Does
    the researcher provide information regarding the study population?  The sample?

33. Is
    the exact sampling method (e.g. simple random, purposive) specified?  Remember, it is not sufficient for a
    researcher to simply state that a sample was selected ‘randomly.’

34. Is
    the sample size sufficient, given the research goals, the degree of accuracy
    the researcher desires, and the nature of the population studied?  Given the nature of the research, is the
    sample size sufficient?

35. If
    the researcher uses a probability sample, does he or she generalize the
    findings to the appropriate population? 
    If the researcher uses a non-probability sample, does he or she refrain
    from generalizing to a wider population?

36. Overall,
    is the sampling appropriate?

    Your evaluation
    should include ONE of the following sections (6-10) (4 pts):

    6. Experiments

37. Can
    you identify a treatment variable that indicates that an experiment is the
    method of observation?

38. How
    many groups were studied?

    1. If there were two or more groups, did the researcher
       use random assignments

    2. If the researcher did not use random assignment, did the researcher present evidence that
       the groups were similar regarding key variables at the beginning of the study?

39. Is
    the treatment and any pre- or posttests described in sufficient detail that
    facilitates replication?

40. Is
    deception necessary?

    1. If so, is the deception within the parameters of the
       research topic?

    2. Have the participants been debriefed so they know the
       true nature of the study (and can enact their right to privacy by declining to
       participate after the fact?)

41. Based
    on the description of treatment and experimental procedure, do you see any red
    flags regarding ethical issues?

42. Did
    the researcher use assistants?

    1. If so, did the researcher state that they were properly
       trained?

    2. If so, did the researcher specify any special measures
       to make sure that the assistants administered the treatment properly?

43. Is
    the setting natural or artificial (in a laboratory)?

    1. If it’s in a laboratory, does the researcher recognize that
       external validity may be weak?

    2. If it’s in a natural setting, does the researcher
       recognize that there may be some differences in the environments of the various
       groups?

    3. Overall, do you think the experimental design is sound?

       7. Survey

44. Is
    the research topic worded appropriately for survey research?

45. Did
    the researcher specifically state which type of survey method was used?

46. Do
    the survey questions adequately address the topic?

47. Are
    the survey questions constructed correctly?

48. Did
    the researcher provide any information about the response rate? Did the
    researcher provide any information about follow-up mailings or other ways of
    increasing response rate? What are the implications of the response rate?

49. Did
    the researcher explain how he or she guaranteed anonymity or confidentiality?

50. Overall,
    is the survey methodology effective and appropriate?

    8. Field Research

51. Does
    the research describe the selected site? Does the research provide some
    explanation as to how that site was chosen?

52. Did
    the researchers explain how they addressed gatekeepers?

53. Did
    the researcher address how he developed field relations?  If conflict arose, did the researcher make
    any comment about how personal or research problems in the field were
    addressed?

54. Did
    the researcher adequately protect the identity of the respondents? Did the
    researcher address other ethical considerations?

55. Did
    the researcher describe, at least in passing, his method of note taking?  Does the method seem adequate?

56. In
    the analysis, does the researcher present general patterns of behavior and
    support those patterns with data such as quoted comments? Does the researcher
    use quotes selectively?

57. Does
    the researcher make any mention of issues of validity and/or reliability?

58. Overall,
    is the research adequate?

    9. Unobtrusive Measures

59. What
    is the researcher’s research purpose or hypothesis?  Is content analysis an appropriate method of
    observation?

60. What
    are the researcher’s units of analysis? 
    What are the units of observation (if they are different than the units
    of analysis)?

61. Is
    the researcher studying a population or a sample of these units?  If the researcher is studying a sample, is it
    a probability sample?  If so, was it
    correctly drawn?  If the researcher is
    not studying a population or a probability sample, is he or she appropriately
    cautious about the nature of any conclusions?

62. Does
    the researcher identify the characteristics and level of content being
    analyzed?  Does the researcher explain
    how material is coded, especially for issues of latent content?

63. Did
    the researcher do any type of pretest with other coders to test for
    reliability?  Where they any tests for
    validity?

64. Are
    the conclusions consistent with the units of analysis?

65. Are
    the results clearly presented and the conclusions appropriate?

66. Generally,
    is the method of observation done appropriately?

    10. Evaluation
    Research

67. What
    is the purpose of the evaluation presented?

68. Is
    the nature of the program described in detail?

69. Are
    the goals presented and can the goals that the author presents be evaluated?

70. What
    type of observation method is used? Is it appropriate, given the real-life
    restrictions of evaluation research?

71. Is
    a control group used? If so, how has the researcher tried to show that it is
    equivalent to the experimental group?  If
    not, does the researcher adequately explain its omission?

72. How
    are people selected for program participation? Does this affect the
    interpretation of findings, and, if so, does the researcher discuss this?

73. Are
    the results clearly explained?

74. How
    does the article address the other areas of evaluation discussed in earlier
    chapters?

    Your evaluation
    should include ONE of the following sections (3 pts):

    11. Qualitative
    Analysis

75. Is
    the results section a cohesive essay?

76. Does
    the researcher connect the results to any general research questions or goals?

77. Is
    the perspective of the results presentation appropriate? Does it match the
    research technique?

78. Has
    the writer presented enough examples to support the conclusions?  Do the examples make the readers ‘believe’
    the researcher’s points?

79. Do
    you have reason to believe that the presence of the researcher influenced the
    actions or statements of other group members? If this is possible, has the
    researcher addressed it in the research?

80. Especially
    in field research (although this may be an issue to a lesser degree in other
    forms of qualitative data gathering), does the researcher discuss how he or she
    interacted with subjects in the field, what problems arose, and how the
    researcher addressed them?

    12. Quantitative
    Analysis

81. Is
    the results section a cohesive essay with the important findings highlighted?

82. In
    the essay, does the researcher tie the results to the research hypotheses or
    goals stated in the introduction?

83. If
    there are tables or graphs, are they clearly presented?

84. Does
    the researcher present any descriptive statistics?

85. Are
    the statistics appropriate for the level of measurement?

86. Are
    the conclusions the researcher draws appropriate for the statistical
    information?

87. In
    the discussion section, does the researcher briefly summarize the research
    purposes, methodologies, and key findings (in a non-statistical manner)?

88. Does
    the researcher acknowledge any methodological or statistical weaknesses?

89. Are
    the implications of the research or suggestions for future research discussed?

90. Overall,
    is the results section adequate?

91. Overall,
    is the discussion section adequate?