Arnoff

Views:
 
Category: Entertainment
     
 

Presentation Description

No description available.

Comments

Presentation Transcript

Data Rich, Information Poor Understanding Data Warehousing and Data Mining So School Library Media Specialists Are Invited To Sit At The Accountability Table: 

Data Rich, Information Poor Understanding Data Warehousing and Data Mining So School Library Media Specialists Are Invited To Sit At The Accountability Table CEMA November 6, 2006 © S. Arnoff, 2006. All rights reserved.

Data, data everywhere …: 

Data, data everywhere …

But not a lot of useable information …: 

But not a lot of useable information …

Creating A “Buzz”: 

Creating A “Buzz” What is a “buzz”? A “buzz” is the very definition of public relations in the Internet era. It is word-of-mouth and word-of-press that elevates a topic into the minds of millions.

The Education Buzz: 

The Education Buzz In education today the buzz is all about: Data-Driven Decision-Making (D3M), Data Warehousing and Data Mining Gathering Data Storing Data Accessing Data to identify information and knowledge that will help improve student achievement

The Education Buzz: 

The Education Buzz The belief is that: In order to improve student learning and to achieve state and federal yearly progress guidelines educators at every level need to become more reliant on data and the research or evidence-based practice that data supports.

School Library Media Centers: 

School Library Media Centers The important question before us is: How can school libraries and library media specialists significantly contribute to the effort of using data to improve student achievement?

Essential Questions: 

Essential Questions To understand D3M, data warehousing and data mining we need to answer three essential questions: 1. What data do we need? 2. Where do we “house” or put the data? 3. How do we “mine” or access the data?

Slide9: 

D3M requires that we are able to effectively and efficiently: gather, store (warehouse), access (mine), and analyze quantitative (numeric) and qualitative (text) data about teaching and student learning outcomes. What is needed?

Slide10: 

The data that we can gather, warehouse, and mine for education decision-making can be defined by the following six categories: What is needed?

Slide13: 

What we intend to teach: Standards Benchmarks Curriculum maps (calendar-based) Standardized tests and assessments What is needed? 1. Curriculum Information

Slide14: 

What was actually taught or implemented: Units and lessons Resources and strategies Differentiated instruction Creating and using rubrics What is needed? 2. Instruction Information

Slide15: 

What students learned or attained: Homework, quizzes, and tests Grades and anecdotal information Perceptions Standards-based reports What is needed? 3. Assessment Information

Slide16: 

Student characteristics: School, grade, classes Gender, ethnicity Language proficiency Learning accommodations What is needed? 4. Demographic Information

Slide17: 

Information that explores program cost benefit analysis: Staff Materials and resources Facilities What is needed? 5. Budget Information

Slide18: 

Information about professional staff: Number of teachers Teaching experience Professional development participation What is needed? 6. Professional Staff Information

Slide19: 

What curriculum, instruction, assessment and student demographic data is available? Is any of the data available in electronic form? What equipment and programs might be needed? Do any staff members have experience analyzing student data? Questions For Thought Questions

Slide20: 

Systems There are two types of information systems: Operational Informational Operational versus Informational Systems

Slide21: 

Operational Systems Operational systems are systems that help run the day-to-day enterprise. A library automation system is an example of an operational system. What important data might this system be able to provide? Operational Systems

Slide22: 

Informational Systems Informational systems support the planning, forecasting, and managing of the school and school system. They contain transactional information from operational systems. Informational Systems “Informational systems have to do with analyzing data and making decisions, often major decisions, about how the enterprise will operate now and in the future.” (Orr, 2000)

Slide23: 

Informational Systems Information from the library automation system can be used to identify subjects areas that are used more heavily by students and teachers than other areas. Non-fiction versus fiction Biographies versus science fiction For Example

Slide24: 

What operational systems does the school or district have? What data may these systems hold? Of the available data, what data sets could help to provide information that will improve student achievement? Questions For Thought Questions

Slide25: 

Data Warehouse There are commercial software products called data warehouses. A true data warehouse, however, is more than one software application. These commercial products provide a place to put a variety of information and a way to search and display results. How do we “house” the data?

Slide26: 

Data Warehouse A true data warehouse is a central repository that brings together what might otherwise seem to be disparate information about the education process. How do we “house” the data? This information is often “jailed” or locked up in the personal computers of individual teachers creating unconnected information islands.

Slide27: 

Data Warehouse Bringing together data residing in computer systems and applications throughout the school system in a data warehouse provides a way to overcome the problem of seeing every tree, but never the forest. How do we “house” the data?

Slide28: 

Data Warehouse Data Warehouse Components This can be graphically represented:

Slide29: 

Data Warehouse The library automation software is an operational database from which information can be exported to a data warehouse. With longitudinal data from the library database, information from the student information system and the language arts program about a student’s reading level; we could look to see if a student’s pleasure reading level of difficulty increases over time. Is the student challenging him or herself? For Example

Slide30: 

Does the district have a data warehouse? Should the district build or buy a commercially developed product? Can the district staff support the development of a data warehouse or should it be outsourced? What should be in the data warehouse? Questions For Thought Questions

Slide31: 

What is Data Mining Data mining is the term used to describe the application of computer technology to attain beneficial information from what is often a vast and at first glance enormous quantity of seemingly unrelated data.

Slide32: 

What is Data Mining Data mining is a way to: answer questions that educators are interested in answering and to identify and answer questions that educators did not know enough to ask.

Slide33: 

For Example Are student demographic characteristics a factor in the type of pleasure reading that students enjoy? Are students that transfer into the school after grade 2 reading at the same level as students that have been in the school since kindergarten? Data mining can answer questions such as:

Slide34: 

How will data be mined for information? What will we do with the results of our data mining? Questions For Thought Questions

Slide35: 

The Knowledge Pyramid Zelenky, M. (1987) “Management Support Systems: Towards Integrating Knowledge Management”, Human Systems Management, Vol. 7. No.1. pp. 58-70. Ackoff, R. L. (1989) “From Data to Wisdom”, journal of Applied Systems Analysis, Vol. 16. pp. 3-9.

Slide36: 

The Knowledge Pyramid shows how raw data, as a result of data analysis, can become information. Information helps us to see and understand relationships. Information over time and with maturity can create knowledge that helps us to see and understand patterns of teaching and learning. The Knowledge Pyramid

Slide37: 

Raw data such as a test score, in and itself, may have little significance beyond its own existence. Information, however, is interpreted data that has meaning and purpose. The Knowledge Pyramid Knowledge takes information one step further and provides insight into impact, interactions, and influences.

Slide38: 

For Example Are students effectively transferring the search strategies taught in the library program to other disciplines? Have information literacy skills improved students’ abilities to find, retrieve, analyze, and use information in English Language Arts assessments? On Math tests? Are students demonstrating the skills of knowing when and where to effectively and efficiently locate information when taking high-stakes tests?

Slide39: 

For Example How can the library program improve student scores in situations where all students, or a subgroup of students, score significantly below the state or district average on items assessing: Understanding Text. Identify the main idea.* Nonfiction. Draw conclusion to summarize ideas. Understanding Text. What is the author implying? Identify facts: what is the advantage? Understand and interpret a metaphor: brief candle. Identify meaning in context: report – gossip. Understand the purpose of repetition. Recognizing bias and propaganda. *Source MCAS 10th grade ELA exam.

Slide40: 

Three manifestations of curricula defined by the SIMS (Travers and Westbury, 1989). © S. Arnoff 2005. All rights reserved. A Data-driven Decision-making Model©

Slide41: 

Two and Three Way Interactions

Slide42: 

Demographic and Curriculum Components Demographic Information School Class Student name Grade Ethnicity Gender Learning provision(s) Language Proficiency Characteristics Curriculum Information Standards Benchmarks Curriculum mapping Standardized tests State assessments District assessments Intended

Demographic and Curriculum Questions: 

Demographic and Curriculum Questions Which groups of students are (or are not) achieving the benchmarks and/or standards? Which benchmarks and/or standards are most troublesome for specific groups of students? What similarities do the students that are (or are not) achieving the benchmarks and/or standards exhibit? What differences do the students that are (or are not) not achieving the benchmarks and/or standards exhibit?

Slide44: 

Curriculum Information Standards Benchmarks Curriculum mapping Standardized tests State assessments District assessments Intended Instruction Information Units Lessons Materials Strategies Differentiated Instruction Authentic assessment Rubrics Professional Development Implemented Curriculum and Instruction Information

Curriculum and Instruction Questions: 

Curriculum and Instruction Questions Is the information specified in the intended curriculum covered in the textbook and class instruction? Are the intended and implemented curriculums aligned? Do the units and lessons taught in specific grade levels adequately cover the material that will be measured by summative assessments for those grade levels? What content is taught that is not tested? Are teachers using appropriate teaching methods and materials to adequately cover the quantity of content in the available instructional time? Have we decided what content is most important to cover? Have appropriate and adequate professional development opportunities been provided to teachers?

Slide46: 

Curriculum and Student Information

Curriculum and Student Questions: 

Curriculum and Student Questions To what degree have students achieved (or not achieved) the benchmarks and/or standards? On which specific benchmarks and/or standards have students achieved (or have not achieved) proficiency? Are teacher-created tests effectively connected to measuring standards achievement?

Slide48: 

Student Information Homework Quizzes Tests Grades Perceptions Standards-based reports Attained Instruction Information Units Lessons Materials Strategies Differentiated Instruction Authentic assessment Rubrics Professional Development Implemented Student and Instruction Information

Student and Instruction Questions: 

Student and Instruction Questions To what extent did students learn (or not learn) the content that was taught? If the teacher uses differentiated instruction techniques, have the differentiated instruction techniques improved student achievement? What demonstrable examples of differentiated instruction improving student achievement are available? What assessment techniques does the teacher use? Are teacher assessment techniques effectively measuring student achievement?

Slide50: 

Demographic Information School Class Student name Grade Ethnicity Gender Learning provision(s) Language Proficiency Characteristics Student Information Homework Quizzes Tests Grades Perceptions Standards-based reports Attained Demographic and Student Information

Demographic and Student Questions: 

Demographic and Student Questions Is there a difference in the achievement of boys and girls? (Gender) students with different ethnicities? (Ethnicity) students that come from homes where English is not the language spoken? (ESL) students with and without instructional modifications? (504 Plans) disabled and non-disabled students? (IEPs) [We will look at this question in more depth.]

Slide52: 

Demographic, Curriculum, and Instruction Information

Demographic, Curriculum, and Questions: 

Demographic, Curriculum, and Questions How did specific groups of students perform on standardized assessments relative the content that was covered in the classroom? Are the intended and implemented curriculum aligned for all groups of students? Are there specific groups of students that performed at or above the proficient standards level as a result of specific instruction interventions? Are there student characteristics that are associated with performing at basic standards level as a result of the presence or absence of specific instruction interventions? Based on summative assessment results, are there instruction interventions that were more successful with specific student groups?

Slide54: 

Curriculum, Instruction, and Student Information

Curriculum, Instruction, and Student Questions: 

Curriculum, Instruction, and Student Questions Are there areas of the intended curriculum that are not included in the implemented curriculum? To what degree are they aligned? How well did students perform on the areas that are (and are not) aligned Based on student achievement, are there areas of the implemented curriculum that can be reduced to provide more instructional time for areas where fewer students achieve proficiency? Are there instructional interventions that are more (or less) successful with specific curriculum benchmarks and/or standards? Are there professional development opportunities that would improve the teachers’ ability to cover more of the intended curriculum resulting in more students achieving the proficient and advanced standards levels? Are there student assessment techniques for specific benchmarks and/or standards that would provide more timely formative assessment information?

Slide56: 

Demographic, Instruction, and Student Information

Demographic, Instruction, and Student Questions: 

Demographic, Instruction, and Student Questions To what degree are specific groups of students learning (or not learning) the content that was taught? To what degree are specific instructional methodologies effective (or not effective) with specific groups of students? To what degree are assessments effectively (or not effectively) measuring the achievement of specific student groups? To what degree have professional development opportunities changed classroom instructional methodologies and is there any observable improvement in student achievement for specific groups of students?

Slide58: 

Demographic Information Curriculum Information School Class Student name Grade Ethnicity Gender Learning provision(s) Language Proficiency Characteristics Student Information Homework Quizzes Tests Grades Perceptions Standards-based reports Attained Standards Benchmarks Curriculum mapping Standardized tests State assessments District assessments Intended Demographic, Curriculum, and Student Information

Demographic, Curriculum, and Student Questions: 

Demographic, Curriculum, and Student Questions Which specific groups of students are (or are not) achieving the proficient level on the benchmarks and/or standards criteria? In what areas are specific groups of students achieving (or not achieving) the proficient level on the benchmarks and/or standards criteria? To what degree do achievement levels for specific groups of students need to be improved to attain the proficient and/or advanced level on specific benchmarks and/or standards? To what degree are assessment measures effective or not effective for providing achievement information for specific groups of students?

Slide60: 

Four-way Interaction Analysis

Four-way Interaction Analysis: 

Four-way Interaction Analysis Four-way interaction analysis uses all components of the model to explore how to best achieve the core purpose of the school: to improve student learning. Using formative assessment data to improve the instructional process and summative assessment data to measure student achievement outcomes disaggregated by demographic data provides useful information about specific student groups. It is by learning to use each of these information sets as a part of the instructional curricula revision process that teachers working together with Library Media Specialists will improve individual student achievement.

How does this work IRL? (in real life): 

How does this work IRL? (in real life) MCAS – Levels of Proficiency Level 1 – Failure/Warning Level 2 – Needs Improvement Level 3 – Proficient Level 4 – Advanced

Comparing Disabled to Non-disabled MCAS Subgroups (real data): 

Comparing Disabled to Non-disabled MCAS Subgroups (real data) 5% in 2002; 22 of 440 11% in 2003; 47 of 433 15% in 2004; 67 0f 454 17% in 2005; 83 of 475 Is the sample size significant enough for us to analyze and draw conclusions? What can we learn from the data and how can our library program help improve student achievement?

Disabled MCAS 02-05: 

Disabled MCAS 02-05 ELA Performance – Level 3 Proficient decreases over the 4-year period.

Non-Disabled MCAS 02-05: 

Non-Disabled MCAS 02-05 ELA Performance - Level 3 Proficient increases over the 4-year period.

Disabled MCAS Gap 02-05: 

Disabled MCAS Gap 02-05 ELA %age Reading Level 3 or Above Compared to Non-Disabled the Gap

Item Analysis by School Report MCAS Grade 10 2005 Non-disabled Subgroup (bold = correct): 

Item Analysis by School Report MCAS Grade 10 2005 Non-disabled Subgroup (bold = correct)

Item Analysis by School Report MCAS Grade 10 2005 Disabled Subgroup [N] = Correct in non-disabled subgroup: 

Item Analysis by School Report MCAS Grade 10 2005 Disabled Subgroup [N] = Correct in non-disabled subgroup

In The Future: 

In The Future There are people who believe that there may be an easier way to achieve these goals. Such approaches, however, are still somewhat experimental.

Slide70: 

For example, if we could wire our students directly to the computer we could teach, test, and remediate with precise accuracy!

authorStream Live Help