Geoff Maslen15 October 2014 Issue No:339
An academic’s reputation plays a key role in generating increases in a
scientific paper’s citation count early in its citation life cycle,
before a tipping point, after which his or her reputation has much less
influence relative to the paper’s citation count. This is the intriguing
finding from a study by a team of collaborating social science analysts
in Belgium, Finland, Italy and the US.
Using data compiled by Thomson Reuters Web of Science, the team studied 450 highly cited scientists, nearly 84,000 articles in scientific publication, and 7.6 million citations tracked over the equivalent of 387,000 publication years!
In a paper published in the US journal Proceedings of the National Academy of Sciences, the team describe how they analysed, for each central scientist, his or her production as measured by the number of publications in a year, the cumulative number of citations received by publications, and a quantitative reputation measure defined as the net citations aggregated across all publications.
Combining several empirical features of the analysis, the researchers then investigated the role of the reputation effect, showing that author reputation accounted for a significant increase in the number of citations an academic could attract.
“Over a scientist’s career, a reputation is developed, a standing within a research community, based largely on the quantity and quality of his or her publications. Here, we develop a framework for quantifying the influence author reputation has on a publication’s future impact,” they write.
“We find author reputation plays a key role in driving a paper’s citation count early in its citation life cycle, before a tipping point, after which reputation has much less influence relative to the paper’s citation count. In science, perceived quality, and decisions made based on those perceptions, is increasingly linked to citation counts. Shedding light on the complex mechanisms driving these quantitative measures facilitates not only better evaluation of scientific outputs but also a more transparent evaluation of the scientists producing them.”
The researchers say that scientific reputation has emerged as a key “signalling mechanism” to address the dilemma of excessive information that arises in tasks such as evaluating, comparing, and ranking publication profiles in academic competitions. As well, a scientist’s reputation plays an important role as a signal of trustworthiness and quality, “a role that addresses directly the ‘agency problem’ characterising the reward system in science”.
“With little time to read every paper on a given topic, this trustworthiness signal is anecdotally consistent with the common practice of perusing the author names when preliminarily evaluating the relevance of a newly found publication,” the team says.
“In the past, an author’s identity and associated reputation was mainly linked to reference lists and personal interactions. Nowadays, an author’s reputation is becoming increasingly visible through searchable publication databases, laboratory websites, press, and other media, in addition to citations.”
In their introduction to the paper, the researchers say that citation counts are widely used to judge the impact of both scientists and their publications. Although many factors outside the pure merit of the research or the authors influence such counts, few investigations have been made into identifying and quantifying the role of author-specific factors.
They note it is also likely that institutional affiliation and journal reputation play a role in the citation dynamics. But they suggest that disentangling the interaction between the multiple reputation sources will likely be challenging and remains an open avenue for investigation.
Nevertheless, as measures are becoming increasingly common in evaluation scenarios throughout science, the researchers say it is crucial to better understand what the citation measures actually represent in the context of scientists’ careers.
“Moreover, how does reputation affect a scientist’s access to key resources, the incentives to publish quality over quantity, and other key decisions along the career path?” they say. “In addition, what role does reputation play in the mentor-matching process within academic institutions, in the effectiveness of single/double blinding in peer review, and in the reward system of science?”
They note that it is also important to consider the role of reputation in light of the increasing orientation of science around team endeavours with “multiple levels of hierarchy and division of labour”. But, because it is difficult to evaluate and assign credit to individual contributors, they say there may be an increase in the role and strength of the reputation in overcoming the problem associated with asymmetric and incomplete information.
Young scientists lacking a reputation can be negatively affected by the social stratification in science, the researchers say, adding that the “appealing competitive advantage gained by working with a prestigious mentor may be countered by the possibility that it may not be the ideal mentor-advisee match”.
Nevertheless, they say that their results have broad implications across the scientific community, given the numerous careers that interact with top scientists via collaboration or mentorship.
Using data compiled by Thomson Reuters Web of Science, the team studied 450 highly cited scientists, nearly 84,000 articles in scientific publication, and 7.6 million citations tracked over the equivalent of 387,000 publication years!
The scientists studied included 100 top-cited physicists, 100 highly prolific physicists, 100 assistant professors in physics, 100 top-cited cell biologists, and 50 top-cited pure mathematicians.
In a paper published in the US journal Proceedings of the National Academy of Sciences, the team describe how they analysed, for each central scientist, his or her production as measured by the number of publications in a year, the cumulative number of citations received by publications, and a quantitative reputation measure defined as the net citations aggregated across all publications.
Combining several empirical features of the analysis, the researchers then investigated the role of the reputation effect, showing that author reputation accounted for a significant increase in the number of citations an academic could attract.
“Over a scientist’s career, a reputation is developed, a standing within a research community, based largely on the quantity and quality of his or her publications. Here, we develop a framework for quantifying the influence author reputation has on a publication’s future impact,” they write.
“We find author reputation plays a key role in driving a paper’s citation count early in its citation life cycle, before a tipping point, after which reputation has much less influence relative to the paper’s citation count. In science, perceived quality, and decisions made based on those perceptions, is increasingly linked to citation counts. Shedding light on the complex mechanisms driving these quantitative measures facilitates not only better evaluation of scientific outputs but also a more transparent evaluation of the scientists producing them.”
The researchers say that scientific reputation has emerged as a key “signalling mechanism” to address the dilemma of excessive information that arises in tasks such as evaluating, comparing, and ranking publication profiles in academic competitions. As well, a scientist’s reputation plays an important role as a signal of trustworthiness and quality, “a role that addresses directly the ‘agency problem’ characterising the reward system in science”.
“With little time to read every paper on a given topic, this trustworthiness signal is anecdotally consistent with the common practice of perusing the author names when preliminarily evaluating the relevance of a newly found publication,” the team says.
“In the past, an author’s identity and associated reputation was mainly linked to reference lists and personal interactions. Nowadays, an author’s reputation is becoming increasingly visible through searchable publication databases, laboratory websites, press, and other media, in addition to citations.”
In their introduction to the paper, the researchers say that citation counts are widely used to judge the impact of both scientists and their publications. Although many factors outside the pure merit of the research or the authors influence such counts, few investigations have been made into identifying and quantifying the role of author-specific factors.
They note it is also likely that institutional affiliation and journal reputation play a role in the citation dynamics. But they suggest that disentangling the interaction between the multiple reputation sources will likely be challenging and remains an open avenue for investigation.
Nevertheless, as measures are becoming increasingly common in evaluation scenarios throughout science, the researchers say it is crucial to better understand what the citation measures actually represent in the context of scientists’ careers.
“Moreover, how does reputation affect a scientist’s access to key resources, the incentives to publish quality over quantity, and other key decisions along the career path?” they say. “In addition, what role does reputation play in the mentor-matching process within academic institutions, in the effectiveness of single/double blinding in peer review, and in the reward system of science?”
They note that it is also important to consider the role of reputation in light of the increasing orientation of science around team endeavours with “multiple levels of hierarchy and division of labour”. But, because it is difficult to evaluate and assign credit to individual contributors, they say there may be an increase in the role and strength of the reputation in overcoming the problem associated with asymmetric and incomplete information.
“In addition to the collaboration network, reputation also plays a key role in numerous other scientific inputs such as money, labour, knowledge, and so on, that inevitably affect the overall quantity and quality of scientific outputs.”
Young scientists lacking a reputation can be negatively affected by the social stratification in science, the researchers say, adding that the “appealing competitive advantage gained by working with a prestigious mentor may be countered by the possibility that it may not be the ideal mentor-advisee match”.
Nevertheless, they say that their results have broad implications across the scientific community, given the numerous careers that interact with top scientists via collaboration or mentorship.
No comments:
Post a Comment