The Matthew effect and cumulative advantage. See links on the course website to get access to the readings. The film is on reserve in Green Media Center under “The Persuaders”.
Robert K. Merton. 1968. “The Matthew Effect in Science.” Science 159(January 5): 56-63.
Derek de Solla Price. 1976. “A General Theory of Bibliometric and Other Cumulative Advantage Processes.” Journal of the American Society for Information Science (September-October): 292-306
Robert K. Merton. 1988. “The Matthew Effect in Science, II: Cumulative Advantage and the Symbolism of Intellectual Property.”Isis 79(December):, 606-623.
Paul A. David. 1994. “Positive Feedbacks and Research Productivity in Science: Reopening Another Black Box.” In O. Grandstrand, ed.Economics and Technology. Amsterdam: Elsevier.
Matthew J. Salganik, Peter Sheridan Dodds, and Duncan J. Watts. 2006. “Experimental Study of Inequality and Unpredictability in an Artificial Cultural Market.” Science 311(February 10): 854-856.
Film: Frontline: The Persuaders (screened April 18 )
Entries (RSS)
April 25th, 2007 at 4:48 pm
For me, the underlying message of the readings this week is that value is ultimately relational. Normally, when we think about what makes a song a good song, we think about qualities of the song itself and how it sounds. However, the ultimate judge of whether a song is “good” or not is it’s relationship with people — whether people listen to it or not. The same is true of scientific discoveries. What ultimately makes a scientific discovery valuable is not its insight or elegance but how well it is incorporated with the rest of science.
One of the insights of holism is that the relationships between elements have value. However, I think an insight of holism that is just as important is that these relationships can have further relations between them which also have value. To use the example of songs, the value of a song lies in its relationship with the listener. However, the value of that relationship is in turn very strongly affected by social influences, or the relations between that listener and other listeners. Or to use a particularly interesting example from the first article on the Matthew Effect, the Committee of the British Association for the Advancement of Science turned down a paper, only to reverse their decision when they found out who authored it. The relationship between the Committee and the author changed the value of the relationship between the Committee and the paper. After the author was known, the paper gained value because people would be more willing to read it and incorporate it into current scientific knowledge.
I think that the holistic insight that relations about relations matter is what connects holism to feedback. Relations between relationships that then affect those relationships are simply feedback loops. And I think that some of the insights about feedback from last week’s readings, such as the ones about hubs and nodes, could be incorporated into the readings from this week. For example, we could take individual scientists as nodes, and the visibility of scientists to each other are the connections between nodes. As nodes form more connections with other nodes, it becomes easier for them to form even more connections, and through a runaway feedback loop the nodes become hubs. As scientists become more visible, more attention is paid to their work, which makes it easier for them to produce more work, which in turn makes them more visible.
April 25th, 2007 at 5:34 pm
So, I was instantly reminded of the Chappie’s Daily this year that scared so many people (including several faculty members…). As you probably all remember, the headline was “Stanford drops to #13″ with a dramatic image of us dropping from #3 or #4 to #13. Anyone who I talked to about it admitted some fear or anger over the issue, even though we repeatedly tell ourselves that we don’t care about rankings? Why? Because a plummet from #4 to #13 really would reduce the qualty of our students: good students don’t want to go to #13 schools, and so we would plummet further. What’s of course stranger still is that the rankings are only approximations based on a rather limited and poorly weighted feature set: test scores, graduation rates, funding, etc, etc. However, because people pay attention to them, the quality of the university’s students and the funding it receives actually does decrease, as long as a number of other factors that make a university good in this ever escalating echo-chamber.
This is of course the flip side of the “Matthew Effect”, which isn’t really adequately modeled by Price at all with the cumulative advantage distribution. While he is right that not being published is not an event, a long swathe of non-events can be taken as a failure. A more adequate model of his Urn model would be to periodically add a failure ball in addition to whatever else happened. Thus, the more successes you had, the more likely you were to have another one because you added more success balls, but the fewer you had, the sooner your “urn” would become overwhelmed by failure. We have to allow this because lack of success leads to fewer opportunities to succeed: fewer top students, less money, less invitations to compete. This is the second half of “the rich get richer, the poor get poorer”, which Price seems to ignore.
Finally, a “Positive Feedback Loop” that should have been in “The Persuaders”, but wasn’t because I suppose it was a year or two too early is the Google AdWords system, and I think it’s worth addressing, mostly because of its simplicity. With adwords, you bid for a pay-per-click for your ads, but google makes money on a per-impression basis, since the more they display the ad, the more important it is that people click it. Moreover, ads that are closer to the top of the page are clicked on far more often, so Google moves the ones that are clicked the most often to the top of the page, creating an echo effect, with the “best” ads getting reinforced and the “worst” ads getting constantly shoved to the bottom of the pile or not displayed at all. All the while of course, Google is raking in cash. So we might want to readjust the Matthew Principle for Internet Advertising to be “For unto every one that hath shall be given, and he shall have abundance: but from him that hath not shall be taken away even that which he hath, and Google shall be given even more.”
April 25th, 2007 at 5:49 pm
The articles about the reputation effect in science are interesting, although much of the information seems intuitive. Clearly there are extraordinarily qualified people who are not rewarded for their work by chance or anomaly (think of the pidgeonhole principle in connection with the number of noteworthy individuals versus awards). Stanford tour guides routinely assure potential Stanford applicants there they could fill at least two more freshman classes with qualified applicants. To some extent, it’s chance that determines the group of students that is admitted to Stanford. Yet over time, being surrounded by other brilliant students and gaining access to additional resources will cause these Stanford students to become more qualified over time. Eventually we would be tempted to conclude that the class of students admitted to Stanford was more talented than other students who were not admitted at the time. However, this is actually just an example of small variance in abilities bubbling up to create larger gaps in accomplishment.
The Merton article seems to depict assessment as a strange outside force that acts in often unfortunate ways on the field of science. Young scientists are not adequately recognized for their work. Replication is taken more seriously if done by established scientists. Homogeneity is encouraged in science because young scientists must do accepted work in order to build their credibility. Yet I would argue that feedback is what distinguishes us from others and, in some sense, creates division in capability. Children start out with at least similar levels of awareness, intelligence, etc. Yet it is the stratification of such things – the quantification of an IQ score that delineates what is “smart” from what is “average” or the difference between the last starter on a soccer team and the first bench warmer that actualizes our potential and guides us in directions where our self-improvement will be most meaningful. In fact, our expectation of our own ability may influence success more than ability itself.
In the same year that Merton wrote about the self-reinforcement of reputation in science, Rosenthal and Jacobson published a book entitled “Pygmalion in the Classroom.” In a now famous experiment, the two researchers administered an IQ test to ten classrooms of first graders. For some, they returned real results, but for the experimental group they chose several students at random to identify as “hidden academic superstars.” When they re-administered tests at the end of the year, these students’ performance had risen sharply as compared with the control group. The idea that they were somehow more talented than others was all it took to propel their performance to extraordinary levels.
In some sense, then, scientific reputation must be somewhat self-fulfilling. To quote Merton, “Scientists that receive recognition for research done early in their careers are more productive later on than those who do not.” In other words, scientists who are quickly singled out as high-performing will go on to prove their supporters right. They succeed because they are expected to. On the other hand, their success must be at least partially due to the objective accuracy of the scientific community. Some of these scientists must have gotten credit early in their careers because they legitimately were brilliant experimenters. I would almost compare this trend to the grading system we’re using in this class. Several people were concerned that the grade matching system simply encourages people to conform to the teacher’s established method of evaluation. Yet it’s also clear that this standard more or less objectively good, and in this respect, the system promotes fair and unbiased assessment. The degree to which perceptions of success are skewed depends on how much we subscribe to the existing standard of comparison.
In the end, I am once again surprised at the strength of the effect of chance and conformity when assessing the merits of average to above average entities. Although at some level, whether or not the Descarteses of the world are inducted into the French academy, the merit of their thinking will speak for itself. I do feel that there is a limit to the self-perpetuating circle of reputation and preference for what is already established.
On a somewhat unrelated note, I very much enjoyed The Persuaders. It’s interesting to think about that movie in connection with the study done on the unpredictability of the popularity of artificial culture. Several people in the movie seemed to feel strongly that they had discovered what facilitates people’s connections with brand names or political candidates, yet this does not align with the Salganik article that claims that the preferences of large groups are completely unpredictable. The conflict leaves me even more eager to know whether corporations’ understanding of what luxury means translates into consumers’ attachment to real products. As the film glibly stated, “Half the money I spend on advertising is wasted; the trouble is, I don’t know which half.”
In the end, though, the most powerful part of the movie for me was the claim that the clamor for advertising is pushing our society towards the inoffensive and vaguely pleasurable. True art in film doesn’t take a break to sell BMWs. Although I don’t have any profound responses to this sentiment, I will say that the movie emphasizes the importance of reflecting on how we as individuals interact with marketers and culture.
April 25th, 2007 at 5:58 pm
Merton’s discussion(s) of the Matthew Effect ultimately rang true with my research experiences thus far, though I have a few qualms with some of the generalizations he makes. Having had the opportunity to work with a few renowned professors, who—while not laureates—are certainly considered eminent in Merton’s terms, I can personally attest to the frequency with which they receive the predominant credit for the research conducted by their graduate students. This does, in a sense, “trickle-down,” as their graduate students have also received credit for my own research ideas. For this reason, I suggest that credit is not only given to the better-known researcher, but as Merton echoes in his discussion of J.B.S. Haldane and S.K. Roy, credit is also given first to those with better credentials—though these two criterions generally coincide (Matthew Effect II). As a large portion of this week’s readings focus on the frequency at which esteemed researchers publish publications, I found it interesting that the pressure to produce papers—specifically in the university setting—in order to maintain positions of status was not explicitly discussed. As far as I understand, to maintain positions here at Stanford, tenured professors annually must submit a list of their recent citations for the given year, including any collaborative works. And, therefore, even after making the esteemed professor status, the number of citations one makes still matters. While in Merton’s first paper, he recounts laureates who oscillate over whether or not to include their names on their students’ papers, for most professors, at least at Stanford, this question does not seem to arise. Merton’s greater point, that students’ papers with their esteemed professor’s names on them will undergo a greater distribution, is certainly the case; it is simply in the advantage of both the student and the professor to include both names on the work.
One issue I had with Merton’s discussion pertained to his “accumulation of advantage” argument. While I would not think to argue with the notion that more privileged students have an advantage in obtaining more advanced degrees, (and also have the financial means to permit “late blooming” that less privileged students do not), he nevertheless appears to assume that all students come from an equal footing. He assumes poor precocious youths would have the means to have their talents recognized and thus have the ability to sustain grants and scholarships to further their academic interests. But this assumption itself rests on these students having equal access to knowledge about available resources, or qualified educators to notice their talent, which has empirically been shown to not be the case. On the other end of the spectrum, of those students who make it into elite institutions, like the ones Merton suggests tend to generate Nobel laureates and other relatively successful individuals, those who tend to pursue advanced degrees are not consistently the most precocious. According to Derek Bok and William Bowen in The Shape of The River, as many as 34% of students pursuing professional or doctoral degrees come from the bottom third of their college classes. This statistic may suggest that the best and brightest are not only the ones who go on to pursue research in one way or another. Yet, this statistic may nevertheless confirm an accumulation of advantage argument as those students who do tend to go own to pursue advanced degrees tend to come from home environments that value higher education, which is in itself a sort of advantage.
Finally, in thinking about the Matthew Effect and its related applications, I couldn’t help but picture two positive feedback loops perpetuating cycles of advantage and disadvantage (which of course have their limits, as Paul A. David suggests), which can be applied not only to the scientific research, the music industry, and income inequality, but even fundamental psychological processes—how we take in information and decide what to process and what to reject. Given our relatively simple capacities to categorize and process information, much like the scientist’s (in)ability to read the thousands upon thousands of journal articles constantly in print, we must, as humans, devise a means of processes the complicated stream of information to which we are constantly exposed. Forgive the social psychology, but I found a lot of parallels between the Matthew Effect and how we come to form stereotypes, which in a sense is a lot like how we train neural networks. Upon repeated exposure to one view or idea, we come to view that idea as more credible than an idea or view to which we are not as frequently exposed; when we see something contrary to our established view, then, we discard it as an oddity rather than consider its significance. This is much like the scientist being more likely to read the article of the already famous researcher than the article of some unknown scientist. As a related point, consider the negative racial stereotype about African Americans committing the majority of crimes in this country. In actuality, crime has been continually decreasing over the past decade (not increasing as the media would have you believe), and crime statistics show that whites commit more crimes than any other racial group (which makes sense, given their population size). Yet, because the media shows images of blacks committing crimes far more often than whites, when we see a white criminal on TV we tend to reject this as an anomaly, while we automatically picture perpetrators of crimes as being black. Returning to Merton’s article, this sort of process is similar to the one the scientist reading the journal article undergoes. When the scientist reads a paper by a group of people, she automatically attributes the work to the name she recognizes as having done good work. Likewise, when we see a student collaborating with an esteemed Professor, we would assume the work belongs primarily to the student as the reverse would be an anomaly. Thus, to generalize, we tend to take in knowledge/ideas similar to what we already know/believe, and reject those that tend to differ. And, as an attempt to categorize complex data in simple ways, we tend to group like things into similar categories, and therefore misattribute qualities of things in the process.
April 25th, 2007 at 7:08 pm
I want to bounce of what David said about the Matthew Effect of AdWords and reflect on the far more pernicious influence of the Matthew Effect of PageRank. PageRank was revolutionary in Internet search because it provided a much more useful metric of the importance of a page than any other previously used metric (earlier attempts were -very- primitive, such as the number of occurrences of the keyword on the page). As such, it provided very useful judgments about the importance of pages. However, as Google gained popularity and influence, to the point where today a vast number of people access the Internet mainly through Google, those pages that have a high PageRank will get more exposure among the Internet population, which will in turn link to the more popular pages, which will in turn further increase their PageRank. A PhD student in the C.S. department has explored the issue, and come up with an algorithm she calls DonkeySort, which probabilistically assigns positions on the results page based on PageRank, increasing the “diversity” of the results (and hopefully mitigating the winner-takes-all effects of the first result returned by Google).
On a related vein, I was fascinated by the methodology of the Salganik et al study on the effects of social influence on music downloads. I think their Web based approach (much larger scale, the effects of variables on aggregate choices) is a fascinating way to conduct economics experiments. The Web makes it much easier to create small, artificial “marketplaces” (artificial, yet nevertheless governed by the laws of human economic behavior), where certain variables and influences can be tracked and measured. Does anyone know of any other experiments of this kind?
And my one comment about the Merton paper on the Matthew effect in science is that the described behaviors are all individually rational (with the exception of rejecting a paper and then accepting it once the authorship becomes known). Consider a publication that is co-authored by a renowned scientist and a previously unknown collaborator. When judging the relative merit of each person’s contributions to the paper, we can establish prior probabilities on the abilities of both co-authors. In the first case, we will assign the renowned scientist a high prior probability of ability, because that ability has been corroborated in the past. In the second case, we will assign the unknown co-author a much more modest prior probability of ability, because we know nothing about this ability. Given the priors, the assignation of relative merit follows quite easily: it’s much more likely that the renowned scientist contributed more to the publication. Likewise, when the previously unknown co-author independently publishes a ground-breaking (second) work, then we can revise the prior probability of his or her ability, which changes our opinion of the first work.
None of the above, of course, makes any contentions against Merton’s arguments on the Matthew effect in science. I merely wish to point out that the Matthew effect is individually rational, even though collectively it may have suboptimal consequences (as Merton describes).
April 28th, 2007 at 10:46 am
I think this first set of readings, “applied” readings, if you will, demonstrates a paradoxical self-awareness of systems thinking in general, one that we began alluding to at the beginning of class. On the one hand is a very lucid, general explanatory power. On the other hand, though, the limitations of these simplified explanatory models are immediately apparent.
There were a few ideas that came up in the readings that I think would be interesting to pursue further.
(1) Buyer relationships. Salganik et al studied the effect of peer-opinions on buying habits. That is, they studied how the “funders” in a simple economic binary affect each other’s choices. This formed a closed feedback loop of it’s own, and suggested that more peer-related information led to more chaotic system behavior. This information should be applied to the academic setting. If different funding institutions are affected by popular trends, which I suspect they are, this model would help explain the discrepancies we are trying to understand.
(2) Scale. Both Merton and later David mentioned in passing how the increase in scientists is leading to more economic competition and a more apparent divide. One thing worth modeling would be how change in size alone affects the success and behavior of a system. For example, David began modeling academic funding using the classic n-armed bandit problem. However, the nature of the problem changes when n (here the number of potential researchers) goes from tangibly finite to effectively infinite. Some of the anomaly seen in the academic world may be a result of prior practices being applied to a problem of rapidly changing scale.
(3) Funder / receiver relationships. David discussed, at the end of his article, that the act of funding, or not funding, may have a positive or negative influence on the return of the researcher. Similarly, the choice of buying a record, say, may adversely or negatively affect the music output of a particular band. The consequences are not always the same, surely; the important point is that buyers cannot necessarily assume objective qualities in the products they shop. This negotiation seems complex, and I at least have never seen this type of feedback modeled in a problem. That said, I think this relationship is crucial to understanding the behavior of the overall system that is encapsulated by academic funding, or other economic problems.
With respect to the readings on research funding, these are all new or different levels of complexity that were necessarily “simplified out” of the discussions we read. Studying these phenomena and then conflating the results may provide more insight into the complex problem of academic funding.