Download Scales of Measurement and Data Collection in Educational Research and more Study notes History of Education in PDF only on Docsity!
Gathering Research Data
Stephen E. Brock, Ph.D., NCSP
California State University, Sacramento
What is āDataā?
The different types of data gathered as part of an
empirical study are referred to as variables.
All variables have two or more values or scores.
Variables can be either categorical (e.g., eye color,
gender, etc.) or quantitative (e.g., rankings, test scores,
etc.).
Variables take the form of at least one of four āScales of
Measurement.ā
Different scales will require different types of statistical
analysis.
Scales of Measurement
Art skill level
Grades
Rankings
Observations are ranked in order of
magnitude.
Numerical ranks express a āgreater thanā
relationship, but with no implication
about how much greater.
Ordinal
Quantitative
Variables
Eye color
Gender
Ethnicity
Type of school
Qualitative categories only.
Observations sorted into categories by
principle of equivalence.
Scale categories differ from one another
only in a qualitative sense.
Nominal
Qualitative
Variables
Scale Properties Examples
Ordinal
1 = Tallest 6ā7āā
6 = Smallest 5ā
Scales of Measurement
Length
Weight
Scale has all properties of an interval
scale, and has an absolute zero
point.
Ratio
Quantitative
Variables
Numerical value assigned indicates Educational Tests
order of merit and meaningfully
reflects relative distances between
points along the scale.
A given interval between measures
has the same meaning at any point in
the scale.
Interval
Quantitative
Variables
Scale Properties Examples
Scales of Measurement
Family Income and Student Reading Test Scores
Ā How is one quantitative (ratio) variable related to
another quantitative (interval) variable?
Ā Correlation study
Gender and Student Reading Test Performance
Ā How is one Categorical (nominal) variable related to
another quantitative (interval) variable?
Ā Correlation study
Homework vs. Longer Classes and Math Test Scores
Ā How does one Categorical (nominal) variable affect
another quantitative (interval) variable?
Ā Ex-Post Facto or Experimental Study.
Scales of Measurement
Principal Ethnicity and Political Party Affiliation
Ā How is one Categorical (nominal) variable related to
another Categorical (nominal) variable?
Ā Correlation study
ADHD and Reading comprehension
Ā How is one Categorical (nominal) variable affect
another quantitative (interval) variable?
Ā Ex-Post Facto study
Methods of Data Collection
Standardized measures
Ā e.g., published tests
Ā These should be in your portfolio
Experimental measures
Ā i.e., measures developed by the researcher.
Coding
Ā of observations and records.
Methods of Data Collection
Standardized measures
Ā e.g., published tests.
Types of Measuring Instruments
Cognitive Tests (what people know and how they think).
Ā Achievement Tests
Ā Aptitude Tests
Affective Tests (what people believe and feel, and what
they perceive).
Ā Attitude Scales
Ā Interest Inventories
Ā Personality Inventories
Methods of Data Collection
Experimental measures
Ā i.e., measures developed by the researcher.
Ā e.g., reading comprehension test (See handout)
Methods of Data Collection
Coding
Ā of observations and records.
Ā e.g., infant smiling code (see handout)
Ā e.g., systematic behavior observation
techniques (see subsequent slides)
Systematic Observation:
Data Collection
Event Frequency Data
Ā Definition : Number of occurrences of behavior that has a
clear beginning and end, measured over a specified time
period.
Ā Example of behaviors measured : A punch; runs from room;
shouts out response, words read per minute, hand raises,
number of problems completed, eye blinks, questions
answered correctly, self-injurious acts with a clear beginning
and ending.
Ā Advantages: Easy to record. A small golf counter is often
used to collect this type of data.
Reference: Sulzer-Araroff, B., & Mayer, G. R. (1991). A guide to selecting behavior recording techniques. Behavior Analysis for Lasting change. New York: Holt, Rinehart & Winston.
Systematic Observation:
Data Collection
Duration Data
Behavioral event to be counted and timed
DATE: DATE: DATE: DATE: DATE:
Start: Stop: Duration:
Start: Stop: Duration:
Start: Stop: Duration:
Start: Stop: Duration:
Start: Stop: Duration: Start: Stop: Duration:
Start: Stop: Duration:
Start: Stop: Duration:
Start: Stop: Duration:
Start: Stop: Duration: Start: Stop: Duration:
Start: Stop: Duration:
Start: Stop: Duration:
Start: Stop: Duration:
Start: Stop: Duration: Start: Stop: Duration:
Start: Stop: Duration:
Start: Stop: Duration:
Start: Stop: Duration:
Start: Stop: Duration: Start: Stop: Duration:
Start: Stop: Duration:
Start: Stop: Duration:
Start: Stop: Duration:
Start: Stop: Duration:
Systematic Observation:
Data Collection
Permanent Product Data
Ā Definition : The enduring outcome of the behavior.
Ā Example of behaviors measured : Number of
problems or number of assignments completed,
windows broken. Activities with discrete, countable
segments.
Ā Advantages: Reliability, Can be collected after the
fact in some cases (e.g., by looking a teacher
grade books).
Reference: Sulzer-Araroff, B., & Mayer, G. R. (1991). A guide to selecting behavior recording techniques. Behavior Analysis for Lasting change. New York: Holt, Rinehart & Winston.
Systematic Observation:
Data Collection
Permanent Product Data
Behavioral outcome (or product) to be counted
Data Collection
Date Frequency Notes
Systematic Observation:
Data Collection
Interval Data
Ā Definition : Number of time intervals in which the behavior
occurs at least once. Total observation time is divided into
equal intervals and noting the behaviorās presence or
absence during that time. If the behavior occurs frequently
(at least once every 15 minutes), then this is the preferred
data source.
Ā Example of behaviors measured : Thumb sucking,
on/off-task, gestures, stereotypical behavior
Ā Advantages: Records behaviors that are not clearly
discrete (not have real clear beginnings and endings).
Reference: Sulzer-Araroff, B., & Mayer, G. R. (1991). A guide to selecting behavior recording techniques. Behavior Analysis for Lasting change. New York: Holt, Rinehart & Winston.
Systematic Observation:
Time Sampling Techniques
ā¢ Whole-interval time sampling. Records the response when
displayed throughout the entire interval. Can be used to
measure on-task behavior. Tends to underestimate occurrences
of behavior. Useful when it is important to know that the
behavior has not been interrupted.
ā¢ Partial-interval time sampling. Records the response when a
single instance is displayed at any time during the interval.
Can be used to measure swearing or bizarre gestures. Tends to
overestimate occurrences of behavior. Used to record behaviors
that are fleeting.
ā¢ Momentary-interval time sampling. Records the response if it
is displayed at the end for a specific interval. Can be used to
measure in-seat behavior or frequent stereotypic behavior.
Useful to record behaviors that are apt to persist for a while.
Reference: Sulzer-Araroff, B., & Mayer, G. R. (1991). A guide to selecting behavior recording techniques. Behavior Analysis for Lasting change. New York: Holt, Rinehart & Winston.
Systematic Observation:
Data Collection
Interval Data (whole, partial, momentary)
Behavioral event to be counted
Interval: DATE: DATE: DATE: DATE: DATE: 8: 8: 8: 9: 9: 9: 9: 10: 10: 10: 10: 11: 11: 11: 11: 12:
Evaluating Tests: Reliability
Stability (test-retest reliability)
Ā changes over time.
Equivalence (alternate form reliability)
Ā Similarity of two versions of the same test.
Internal Consistency
Ā Similarity of items within a test.
Scorer/Rater
Ā Agreement of independent scores/raters.
Standard Error of Measure
Allows us to estimate how much
difference there probably is between a
personās obtained and ā true ā scores.
The size of the difference is a function of
the tests reliability. Big differences
indicate low reliability.
Reporting scores as falling within a given
range takes SEM into account.
Evaluating Tests: Validity
The degree to which a test measures
what it is supposed to measure and,
consequently, permits appropriate
interpretations.
Valid for specific purposes and
populations.
A matter of degree.
Evaluating Tests: Validity
Content Validity
Ā Degree to which the test measures the intended
content area.
Ā Includes both item (item relevance to content area)
and sampling (sample of total content area) validity.
Ā Determined by expert judgment.
Ā e.g., the content validity of a science test would be
determined by a group of experienced science teachers
Evaluating Tests: Validity
Criterion-Related Validity.
Ā The degree to which a test (the predictor) correlates
with a second measure (the criterion).
Ā Concurrent Validity. Both measures are administered in
the same time frame. How well the measure reflects
current functioning.
Ā e.g., the correlation between the experimenterās 7th science
test results and student grades given by their 7th grade
science teacher.
Ā Predictive Validity. Both tests area administered at
different times. How well the measure predicts future
performance.
Ā e.g., the correlation between the experimenterās 7th grade
science test results and student grades given by their 8th
grade science teacher.
Evaluating Tests
Construct Validity
Ā The extent to which the test reflects the construct it
is intended to measure. It requires a series of
studies (including studies to determine content and
criterion-related validity research), and is the most
important form of validity.
Ā Does the test measure what it is supposed to
measure.
Ā e.g., the experimenterās 7th grade science test positively
correlates with other 7th grade science achievement test
results. In addition, the experimenters science test
correlates to a higher degree with other science tests than it
does with tests of other academic areas.
Portfolio Activity #
Identify at least three (3) standardized
measures relevant to areas of research
interest. The following information should be
included for each measure: (a) the name,
publisher, and cost of the measure; (b) a brief
description of what the measure purports to
measure, (c) a brief summary of the measureās
reliability and validity data.
Small group discussions
Portfolio Activities #7 & #8:
Mini-proposals 1 & 2
Briefly describe a survey research
project relevant to one of their your
research topics.
Briefly describe a correlational research
project relevant to one of your identified
research topics
Next Week
Descriptive Research
Read Educational Research Chapters 7
Portfolio Elements Due:
(7) Mini-proposal 1.
(8) Mini-proposal 2.
