675-6 Research Quality Using Transformational Conversations

Category: Education

Presentation Description

Issues of Reliability, Validity, Usability in transformational conversations action research


Presentation Transcript

Quality of Research:

Quality of Research How we can be sure that research is not rubbish?


RELIABILITY Can be estimated, though not exactly defined

What is the difference between Reliability and Validity? :

What is the difference between Reliability and Validity? Validity has to do with how accurate is your analysis . In action research it also has to do with how usable is it for social change Reliability: Is it consistent against time, repeatable

Which slide shows Reliability and Validity? :

Which slide shows Reliability and Validity?

Types of Reliability :

Types of Reliability The degree to which a research instrument produces consistent results E.g. Our bathroom scales weigh my bag as 35 KG then 28 Kg then 42 Kg. Are they reliable?


Test-retest Mostly you will do this with qualitative work You compare the results of the first survey or questionnaire with the second and identify the differences. By reworking it, you then compare the third, gradually iterating to a process that works well.


INTERNAL CONSISTENCY If all rate a professor equally that is highly reliable If half rate the professor positively and half hate him and rate him negatively, then the interelationship seems to be unreliable.

Parallel consistency:

Parallel consistency Groups complete similar versions of questions These are compared.


Validity Helps us make meaning? IS this research related tt eh topic. How accurate is the measurement Face Validity: does it look right Truthfulness: Have we labeled the idea accurately Criterion-related validity: e.g. SAT measures literacy, writng and problem solving. They compare against criteria based on an enormous experience of tesitng , and then these predict university success.

Logic and Validity:

Logic and Validity In a valid argument it is logically impossible to say that the premises of the argument and the conclusions are false. Cats have fleas Dogs scratch fleas All dogs are cats The flow of your logic must be clear. Demonstrate each link with clarity

Every piece is logical:

Every piece is logical Blah, blah blah . Blah blah blah. Blah, blah, blah. Blah, blah, blah. Blayh , blyah , blyh . Balh , bylh , byhl . Blahlo , blah, blah Blah blah blah premise linkage Connection Expansion of premise conclusion Expansion of conclusion

Theological validity:

Theological validity The validity of a theological perspective can only be considered within the totality of Biblical theology, the integration from cover to cover. The validity of the city conversation can only be determined within the totality of the urban conversation. Slums are but a microcosm within an urban world.

Missiological Research:

Missiological Research Acceptabe Research: Trust and Truth not Reliability (repeatability) and Validity(right data collection, right way) From Van Engen , Mission on the Way: Issues in Mission Theology

Missiological Questions: Trust:

Missiological Questions: Trust Trust Did the reader read the right authors, the accepted sources? Did the researcher read widely enough to gain a breadth of perspectives of the issue? Did the researcher read other viewpoints correctly? Did the researcher understand what was read? Are there internal contradictions either in the use (and understanding) of the authors or in their application of the issue at hand?


Truth Is there adequate biblical foundation for the statements being affirmed? Is there an appropriate continuity of the researcher’s statements with theological affirmations made by other thinkers down through the history of the church? W here contradictions or qualifications of arise, does the mission theologian’s work adequately support the particular theological directions being advocated in the study? Are the dialectical tensions and seeming contradictions allowed to stand, as they should, given what we know and do not know of the mystery of God’s revealed hiddenness as it impacts our understanding of missio D ei ?

Criteria to evaluate missiology:

Criteria to evaluate missiology Revelatory- grounded in scripture Coherent – holds together around an integrating idea Consistent – no glaring contradictions Simple – reduced to the basics of the issue at hand Supportable- logical, historically, experimentally, praxeologically affirmed and supported. Externally confirmable - – other significant thinkers, theological communities or traditions lens support t the thesis Contextual – interfaces appropriately with the context Doable – can be translated into missional action Transformational Productive of Appropriate consequences - prescriptive

4 Ways to Check Quality:

4 Ways to Check Quality Research is knowledge s tructured in a socially approved manner. Reliability: If the research was done again by someone else with different people would we get the same results? Subject error (different answers on different days) Subject bias (results are changed to match up with your research) this is mitigated by your incarnational involvement Observer error and bias

Internal Validity:

Internal Validity Does the evidence reflect the reality under investigation? Do the results match up? Threats to internal validity Historical threats: e .g. the World Cup interferes with social involvement of favelados in Brazil Testing: raising the question itself begins to bias people Instrumentation: the researchers are using certain approaches that change peoples words. Regression to the mean – if you repeat tests, they will move towards the mean from extremes.

Cause and Effect Validation Errors:

Cause and Effect Validation Errors Cause and effect Is there a relationship? V2 not dependent on V1 V2 influenced more significantly by other variables Changes over time of the relationship E.g. the Word Cup in Brazil affecting favelados involvement in social action.

Who was Jurgen Habermas? :

Who was Jurgen Habermas ? German sociologist and philosopher about critical theory ( a vein of post-Marx Marxism) and pragmatism.

Habermas 4 Criteria for Social Validity applied to Action Research:

Habermas 4 Criteria for Social Validity applied to Action Research Jurgen Habermas His Theory of Communicative Action Understanding through Dialogue Based on ideas, experiences and information But dialogue requires validation if people are to come to an understanding How to validate? Comprehensive Truthfulness Appropriate Authentic Applied to: Practitioner Values Language of Scholarship

Generalizability (External Validity):

Generalizability (External Validity) Going from the sample to a wider grouping (the whole people) What relevance do the results have beyond the situation investigated? Threats Atypical Selection (specific to this group – not generic) Snowball sampling, because these are the easiest volunteers for you to access – but it may only be who you can connect with. Atypical Setting (not a generalizable setting. E.g. Costa Rican slums are marginalized Nicaraguans) Atypical History (particular past experiences that are unique) Construct effects (only this group has particular constructs)


Credibility Who did it? Background training? Way they approached? Who paid for the research?


References Based on Shipman. (1997) The Limitations of Social Research. University of California Graham Gibbs. (2012) Reliability, validity, generalizability and Credibility , part 1, youtube video. http://youtu.be/4NQHeI8GD54 Habermas , Jurgen (1984) The Theory of Communicative Action. Beacon Press Van Engen , Charles and P aul Hiebert . (1996). Mission on the Way: Issues in Mission Theology. Baker Academic.

authorStream Live Help