Means to valuable exploration II: How to explore data to modify existing claims and create new ones

Abstract

Transparent exploration in science invites novel discoveries by stimulating new or modified claims about hypotheses, models, and theories. In this second article of two consecutive parts, we outline how to explore data patterns that inform such claims. Transparent exploration should be guided by two contrasting goals: comprehensiveness and efficiency. Comprehensivenes calls for a thorough search across all variables and possible analyses as to not to miss anything that might be hidden in the data. Efficiency adds that new and modified claims should withstand severe testing with new data and give rise to relevant new knowledge. Efficiency aims to reduce false positive claims, which is better achieved if a bunch of results is reduced into a few claims. Means for increasing efficiency are methods for filtering local data patterns (e.g., only interpreting associations that pass statistical tests or using cross-validation) and for smoothing global data patterns (e.g., reducing associations to relations between a few latent variables). We suggest that researchers should condense their results with filtering and smoothing before publication. Coming up with just a few most promising claims saves resources for confirmation trials and keeps scientific communication lean. This should foster the acceptance of transparent exploration. We end with recommendations derived from the considerations in both parts: an exploratory research agenda and suggestions for stakeholders such as journal editors on how to implement more valuable exploration. These include special journal sections or entire journals dedicated to explorative research and a mandatory separate listing of the confirmed and new claims in a paper’s abstract.