ÍÃ×ÓÏÈÉú

Following Planning and Design, you can begin to think about “How” you will implement your research or evaluation study. Using the research questions that you have developed and refined, you can generate a model for your study, and select methods to carry out and collect study data. Part of this process involves selecting or developing Instruments and Measures for your work.

Two key considerations for selecting instruments and measures are to ensure that: (1) your methods and instruments are well-matched to your research questions, and (2) you select and/or develop valid, reliable measures that are appropriate for your study or evaluation participants. You also will need to consider what types of analyses would best respond to your research questions—the type of analyses you choose should guide the development of instruments and what data will be collected. Important questions to ask, when selecting, modifying, and/or developing instruments include:

• When using an existing instrument:
  • Do I have permission to use this instrument, and how should I cite or acknowledge it?
• When developing or modifying an instrument:
  • How many, and what types of items do I need to include in my instrument (e.g., single-choice, multiple-choice, open-ended)?
  • What is the most appropriate measurement-scale for my items (e.g., nominal, ordinal, interval, ratio)? Can I group items on my instrument into subscales to better respond to my research questions?
• When pilot testing a new or modified instrument:
  • How do I ensure that the instrument is measuring what it intends to measure accurately and consistently?

Tools for Developing Instruments and Measures

There are a plethora of resources and tools that can assist you in developing instruments and measures, as part of your work.  

Guidelines for Research Design and Development

Many state and federal agencies (e.g., National Science Foundation – Directorate for Education and Human Resources [EHR]; U.S. Department of Education - Institute of Educational Science [IES]) provide guidelines for designing effective education research. Some of these agencies collaboratively have identified 6 key types of study designs: 1) foundational research, 2) early-stage or exploratory research, 3) design and development research, 4) efficacy research, 5) effectiveness research, and 6) scale-up research.

Common Guidelines for Educational Research and Development (see )

Research Procedures and Standards Handbook, V3.0 (see )

Instrument Validity and Reliability Testing

Validity and reliability are two important factors to consider when developing and testing any instrument. Validity refers to the degree to which an instrument accurately measures what it intends to measure. Three common types of validity for researchers and evaluators to consider are content, construct, and criterion validities. Reliability refers to the degree to which an instrument yields consistent results. Common measures of reliability include internal consistency, test-retest, and inter-rater reliabilities.