The impact of open access scholarship has been studied and under review since at least 2004 when Stevan Harnad & Tim Brody published the seminal article: Comparing the Impact of Open Access (OA) vs. Non-OA articles in the Same Journals. More recent work has looked at impact more broadly as evidenced by the article published by Heather Piwowar, et. al. in 2018, The State of OA: a Large Scale Analysis of the Prevalence and Impact of Open Access Articles in the journal PeerJ.
Concerns regarding the historic metrics used to evaluate research impact are exemplified by the San Francisco Declaration on Research Assessment. The intent of this initiative is to change the evaluation of research outputs including how scholarship is evaluated within academic institutions. The general recommendations are to try to move beyond journal impact factors and evaluate research more on its own merits and shift the focus to impact on policy and practice. Part of the call of the declaration is for publishers to remove any limitations placed on references whether the article has been published open access or not. Making the citations used for any article readily available expands the reach of cited content. Since its inception in 2012, 19,210 individuals and organizations from 145 countries have signed on to follow the DORA guidelines.
Identifying and explaining your impact with open access scholarship goes beyond standard citation metric analysis. Emily Ford, Urban & Public Affairs Librarian, developed the Demonstrate Your Impact Guide to outline the various ways your impact can be both qualified as well as quantified through various tools.