And the Question Is... Evaluating the Validity of a Survey

Student Objectives

Session 1: Understanding Surveys

Session 2: Survey Result Analysis

Session 3: Survey Validity

Extensions

Student Assessment/Reflections

 

STUDENT OBJECTIVES

Students will

• Explore the meaning and purpose of surveys in writing and through discussions
  
  
• Improve comprehension of how surveys work by learning what types of questions are asked when conducting surveys
  
  
• Determine the validity of a survey and its conclusions by reading, discussing, and analyzing the results
  
  
• Improve skills in critical thinking, respectful dialogue, and reflective writing

back to top

 

Session 1: Understanding Surveys

1.	Ask students to respond in their journals to the prompt questions (see Preparation, Step 1). After a few minutes, invite students to share their answers with the class.
2.	Have students go to the Encyclopedia of Educational Technology: Types of Survey Questions website to learn about surveys and the types of questions asked. Allow time for them to read and digest this information.
3.	Lead a class discussion, asking students to share what they have learned from the reading. Ask students to explain how the information on the website agreed with or differed from the question responses they wrote in their journals. For example: What is the purpose and meaning of a survey? What kinds of questions are used in surveys? Have you learned any new information from the reading?
4.	Have students get into small groups of no more than four. (Note: You can let students choose their own groups, although you want them to be as heterogeneous as possible and may need to intervene accordingly.) Ask each group to visit the Indian Women & Gender Roles survey website.
5.	Ask students in each group to read the Introduction, Data/Methods, and Questions of the survey. Ask them to think about the following while reading: What is the purpose of the survey? How do you know? What type of questions were asked? Tell students that, if necessary, they can return to the Encyclopedia of Educational Technology: Types of Survey Questions. Encourage students to refer to the chart outlining which types of questions work best for particular research goals.
6.	Ask the groups to share their findings in a whole-class discussion. Points to consider include the purpose of the survey and the types of questions asked.

Homework: Students should record their new findings in their journals. Journal entries might consider the following:

• What I knew before this session
  
  
• What I learned from this session

back to top

 

Session 2: Survey Result Analysis

1.	Have students return to the groups from Session 1. Ask each group to visit the Survey How To website. This site gives a very thorough explanation of survey methods.
2.	Distribute copies of the survey students started analyzing in Session 1 and post the following questions on the board or overhead projector: How was the data analyzed? What is the survey telling you? Do you think the types of questions asked were appropriate for this kind of survey? Why? Why not? Would you have added or eliminated questions? Which ones? Ask students to discuss these questions in their small groups.
3.	Circulate among the groups and answer any questions students might have. Make sure students understand that charts were the format used for data comparison.

back to top

 

Session 3: Survey Validity

1.	On the board, write the word validity and ask students what they think it means. Then allow them to look up the definition in the dictionary. Discuss.
2.	Ask students how, in their opinions, validity applies in the case of surveys. How do we determine the validity of a survey? Let students discuss this question freely in their small groups. If necessary, provide the answer to the question: Validity is the degree to which a study accurately reflects or assesses the specific idea that the researcher is measuring.
3.	In terms of the particular survey students are analyzing, ask them to consider whether it really assesses Indian women and gender roles. Have students work in pairs to analyze the purpose, subjects, questions, conclusions, and validity of the survey using the Survey Evaluation Form. Allow a few minutes for this activity, and then ask students to share their opinions with the class.
4.	As a concluding activity, lead a whole-class discussion about the importance of media literacy, using the following prompts for the discussion: Why is it important to use statistics such as survey data? What bias might affect how the data is gathered, analyzed, or presented? Think about the survey you have just analyzed. Next time you analyze a survey, how will you know whether the survey is valid? What elements will you consider in this analysis?
5.	Ask students to reflect in their journals on what they think they have learned from this lesson and how it might affect their own research in the future.

back to top

 

EXTENSIONS

• Conduct the companion lesson "And the Question Is... Writing Good Survey Questions" to enable students to continue examining effective survey questions and to create their own survey to administer to classmates.
  
  
• To reinforce real-life application of the knowledge and skills attained in this lesson, have students analyze other surveys. Some examples are available from The Pew Research Center for the People and the Press.
  
  
• Ask students to analyze a news broadcast (perhaps using an online transcript) to determine what could have been added or eliminated to present the information differently. Also, what kind of survey information might have contributed to more information for the broadcast?

back to top

 

STUDENT ASSESSMENT/REFLECTIONS

• Monitor students’ participation in class and group discussions.
  
  
• Review students’ journals and the Survey Evaluation Form to ensure that they can identify the purpose, types of questions, and validity of a survey. Were students able to critically evaluate a survey? Did they understand the terminology?
  
  
• Assess what students learned using the Survey Analysis Rubric.

back to top