Lesson_Nine

Lesson 9 : 

Lesson 9 How Good Is The Evidence? Part II

Personal Observation as Evidence : 

Personal Observation as Evidence Personal observation is the basis for a great deal of scientific research. People trust what they can see. Personal observations are vivid and we rely upon them as evidence.

Slide 3: 

Problems with personal observation as evidence: People see/hear what they want. People are biased. We should be wary of relying on evidence based on one person’s observations in situations where multiple observers might have different observations.

Slide 4: 

Personal observation is most reliable as evidence when it is based on: Recent observations. Observations made by several people. Who have no apparent bias related to the event observed. Who were in a good position to see what happened. Examples: T.V. program Email to friend

Research Studies as Evidence : 

Research Studies as Evidence A systematic collection of observations by people trained to do scientific research. Scientific Method Avoid many of the biases in our observations. Seeks information in the form of publicly verifiable data. Multiple qualified people make their own observation. Level of control. Reduces error in observation and in the interpretation of research findings. Easier to achieve in physical sciences.

Slide 6: 

Scientific Method Tries to be precise and consistent in its use of language. Helped us achieve observations that are more dependable. Faults are not totally done away with.

Problems with Research Findings : 

Problems with Research Findings When communicators appeal to research as evidence, you should remember these important points (pgs. 105-107): Not all research is created equal. Research evidence must be understood in context. Research findings are not solid proof of conclusions. Not all researchers are the same (biases, etc). Communicators sometimes distort or over-simplify findings.

Problems with Research Findings : 

Problems with Research Findings When communicators appeal to research as evidence, you should remember these important points (pgs. 105-107): Research findings change over time (social sciences). Research can vary by how controlled or artificial it is. Research can be affected by the need for financial gain, status, security, etc.

Slide 9: 

Always ask the following critical questions about research evidence…(pg 108) What is the quality of the source of the research? Has the research been replicated? How selective has the communicator been in choosing studies? Is there any evidence of strong-sense critical thinking?

Slide 10: 

Always ask the following critical questions about research evidence…(pg 108) Is there any reason for someone to have distorted the research? Are conditions in the research artificial and therefore distorted? How far can we generalize, given the research sample? We will discuss this more in a moment. Are there any biases or distortions in the surveys, questionnaires, ratings, or other measurements that the researcher uses?

Slide 11: 

Such questions help us avoid: Blind acceptance. The impossible certainty fallacy. Assuming that a research conclusion should be rejected if it is not 100% certain. All research is going to have some doubt associated with it.

Generalizing from the Research Sample : 

Generalizing from the Research Sample Communicators will often use research reports to support generalizations. The dependability of such generalizations depends upon: Number of people or events studied The variety (or breadth) of people or events studied The randomness of the events or people studied.

Slide 13: 

Sampling The process of selecting events or persons to be studied (sample groups). Is used because researches can never study all events or people about which they are studying. Use sample group to generalize about the wider population of persons or events. Not all sample groups are created equal. How do we know if a sample group is dependable?

Slide 14: 

By examining the three items we just listed: number, breadth, and randomness. The sample size must be large enough to justify the generalization or conclusion. Usually the more people or events that are studied, the more dependable the conclusion. The sample size must contain as much diversity as the type of events/people about which conclusions are being drawn. The more random the sample size, the better. Randomness helps insure that all events/people about which they are drawing conclusions have an equal chance of being part of the sample group.

Slide 15: 

Critical questions of all research studies: How many events or people were sampled? How diverse (breadth) is the sample group? How random was the sample group? If we don’t answer these questions we might be guilty of: Overgeneralization or hasty generalization. Study results are much broader than the findings allow.

Example (pg. 109-110) : 

Example (pg. 109-110) “Alcohol consumption is at an all-time high at colleges nationwide. A recent survey conducted by Drinksville University found that of the 250 people surveyed, 89 percent drink on a semi-regular basis.” What problems do you see with that paragraph?

Biased Surveys : 

Biased Surveys How dependable are surveys and questionnaires used to measure people’s attitudes and beliefs? We have to be aware of various influences that may affect the dependability of such surveys. Are the questions answered honestly? Are the questions ambiguous and thus subject to multiple interpretations? How were the questions worded? Do the questions contain built-in bias (wording, etc)?

Slide 18: 

Example Page 110 “A college professor found that 86 percent of respondents believe that President Bush has failed the American people with respect to his handling of the war in Iraq” What was the survey question used in the poll? “What do you think about the President’s misguided efforts in the war in Iraq” The poll question has a clear built-in bias. Responses obtained are not an accurate indicator of attitudes concerning President Bush’s handling of the war in Iraq.

Slide 19: 

Another example from Page 111 Question: Should poor people who refuse to get a job be allowed to receive welfare benefits? Conclusion: 93% of people responding believe poor people should not receive welfare benefits. Survey results can also be affected by where the question is located in the survey, the length of the survey, the topic of the survey, etc. As critical thinkers we should examine survey procedures carefully before accepting survey results.

Group Activity : 

Group Activity Review the example on pages 112-113 Work through Passage 3 on page 119. What are the problems with the research survey mentioned there?

Case Examples as Evidence : 

Case Examples as Evidence Case examples are detailed descriptions of one or several individuals or events that are used to support a conclusion. Case examples can be very compelling because they are usually easy to visualize. Dramatic stories appeal to our emotions and therefore often distract us from seeking more relevant research evidence.

Slide 22: 

We should ask critical questions in order to determine the dependability of case examples as evidence: Is the example typical? Are there powerful counterexamples? Are there biases in how the examples are reported?

Analogies as Evidence : 

Analogies as Evidence Analogies compare two known things to allow the reader to better understand the relationship to something that is unfamiliar. The first part of an analogy involves a familiar object or concept. The second part of the analogy contains the relationship between the familiar objects or concepts. This relationship is used to create a principle that can be used to assist the understanding of a different object or concept. Finally, the new or less familiar object or concept is described in the same format as the known object or concept.

Slide 24: 

“Adults cannot learn all of the intricacies of new computer technology. Trying to teach adults new computer systems is like trying to teach an old dog new tricks.” “As an educator it is important to weed out problem students early and take care of the problems they present because one bad egg ruins the omelet.”

Slide 25: 

Reasoning by analogy is a common way of presenting evidence to support a conclusion. But how is it for evidence? Accurate analogies are powerful evidence, but they are often difficult for people to evaluate (assess). Overly emotional analogies cloud the real issues in argument and prevent meaningful discussion.

Evaluating Analogies : 

Evaluating Analogies To evaluate the quality of an analogy, we need to focus on two things: The number of ways the two things being compared are similar and different. The relevance of the similarities and the differences. Dependable analogies will be ones in which the two things we compare possess relevant similarities and lack relevant differences. When an analogy is proposed in which there are important relevant differences it is known as a Faulty Analogy.

Class Assignment : 

Class Assignment Read chapter 10 for next week. Evaluating an Historical Website Handout. Lesson 9 The End