Whether social science involved questionnaires or behavioral measures, it is common practice to use established measures from previous research, and sometimes to modify and adapt them to new studies. This process can produce several variants of popular measures used in different settings and reported in different papers. A lack of standardization in empirical procedures across studies can threaten the objectivity, reliability, and validity of the research literature.

Sometimes, measures lend themselves to adaptation not for different research questions and settings, but for the more problematic purpose of increasing the likelihood of producing a desired result (e.g., a "significant" effect), and, thus, increase the number of spurious findings in the research literature. Other times, excessive flexibility across studies could imply that they're not even informing us about the same construct, process, or other variable of interest.

Given that problematic analyses using flexible measures can be difficult to identify in any given individual paper, FlexibleMeasures.com is dedicated to collecting, aggregating, and communicating examples of flexible measures in the literature to encourage reflection about both the use of flexible measures in research and the validity of studies that rely on them.

So far, FlexibleMeasures.com reports extensive analyses on two measure (the Competitive Reaction Time Task and the Go/No-Go Task), but hopefully the database will grow - collaborations are welcome!

Flexible Measures