The End of Good Science?

On his Web site, Pedro Merino reveals to the cybersurfing world that he cycles, swims, plays tennis, and likes classical music. He also notes that he likes to use the Internet to report his chemical research. That may not sound as cool as an extreme sport like helicopter skiing, but it’s a practice so radical that it could alter the fabric of the chemistry community.

Merino, an organic chemist at the University of Zaragoza in Spain, has submitted four research papers to a new online venue called the Chemistry Preprint Server, which is part of a virtual club called ChemWeb.com that boasts 200,000 members. Launched last summer, the server is the first major repository of chemists’ works in progress and research articles that have not been reviewed by a few selected chemists, as they are in published journals. Contributors can return to the site again and again to update their submissions, while the server records an electronic trail of their changes. ChemWeb visitors can rank the postings and contribute their opinions of them.

So far, chemists eager to share their results and garner helpful comments have uploaded, without charge, more than 120 documents to the ChemWeb server. This might sound like small potatoes, considering that a single print publisher—the Washington, D.C.-based American Chemical Society—annually publishes 32 peer-reviewed journals containing some 20,000 papers.

But consider this: Another preprint server with humble beginnings, based at the Los Alamos (N.M.) National Laboratory, racked up just 225 submissions in its first four months in 1991. That venture, which posts physics preprints, thrived to become the benchmark for later electronic preprint efforts. A decade later, the physics server houses more than 150,000 papers, with an estimated 31,000 added in the past year alone.

So, when Merino—a fan of computerized scientific communication—wrote a research paper last year, he not only submitted his work to the ACS’s Journal of Organic Chemistry (JOC) for the traditional sequence of scientific review followed by author revisions and then publication. He also uploaded it to the Chemistry Preprint Server for direct, immediate dissemination.

This double submission landed him on a mountain of conflict.

Merino had unknowingly stumbled into a spirited debate in which preprint servers aren’t just convenient communication devices, as some researchers see them. Opponents of the electronic preprint system say that uploading a paper to such a server constitutes publication—and many journals, such as JOC, don’t accept work that has been published elsewhere.

After some uncertainty, Merino’s case was resolved when just the abstract of his contested preprint was left on the server but linked to its final, peer-reviewed article on the ACS Web site, say the director of ACS publications, Robert D. Bovenschulte, and the director of ChemWeb operations, Bill Town.

Still, JOC and many other publications as a general rule, won’t be accepting a research paper if a preprint has been available online. A statement from some 30 ACS journal editors released last month makes this position even more firm.

So, what does the future hold? Will chemistry preprint servers accelerate the pace and dissemination of discovery? Will they undermine reliable research?

Or, will they die in their infancy?

A matter of semantics

At first glance, the controversy seems a minor matter of semantics that article authors and journal editors could easily wrangle over outside the public eye. But those who are made squeamish by archives of unrefereed papers say that there’s something much bigger at stake here: the accuracy and validity of science.

Anyone can post research to an electronic server, whether or not the work is any good. Many chemists ask, Are we on a downward spiral into a world where no one knows which scientific results are valuable?

Journal publishers provide a service, many researchers believe, by evaluating articles before they publish them. The journals select topics of interest to their readers and have scientists in the relevant field comment on the submissions. They consider, for example, whether the method and the analysis employed were appropriate.

After this assessment, the journal editors select articles for publication, often only a small percentage of those submitted. Scientists often regard work published in the most selective journals as the most valuable.

Peter J. Stang, editor of JOC and a chemist at the University of Utah in Salt Lake City, said at an ACS meeting last August that he sees dangers in trading this peer review for wider, quicker dissemination. If chemists could post whatever they chose, it would be difficult to identify valuable information in a mountain of data. “How do we extract the signal from the noise?” he asked.

Speaking on the same panel, librarian Ann Wolpert from the Massachusetts Institute of Technology in Cambridge pointed out that it’s difficult to organize fluid, changing research papers from such a server. “If I sit here before you with gray hair today, it’s because librarians spend an inordinate amount of time helping people track down real references that are flawed,” Wolpert said. “Can you imagine what the world will be like with preprint references?”

Furthermore, in questioning whether the new tool automatically improves scientific communication, Wolpert asked, “Is it progress if a cannibal eats with a knife and fork?”

Meanwhile, supporters of preprint servers consider the technology to be an inevitable development of the ever-more-online world and one that enriches science rather than depletes it.

In these venues, researchers communicate their results immediately, reach a broad audience, and obtain input from peers around the world. ChemWeb, for example, has received submissions not just from the United States and western Europe, but Romania, Bulgaria, Russia, the Czech Republic, Argentina, Brazil, Iran, Morocco, India, and Japan.

Scientific developments become accessible to anyone who can connect to the Internet�including researchers in poorly funded university departments and less developed countries who can’t afford a bevy of hefty subscription fees.

“Work can be shared with other scientists who can offer valuable comments before publication,” says Merino. “And it is a very good choice for communicating research results in a very fast way.”

It just makes conceptual sense to offer research freely to as many readers as possible, says physicist Paul Ginsparg, founder of the Los Alamos preprint server. The advantages of having the information immediately available outweigh any disadvantages, he says.

Preprint servers

The physics community has already traversed the growing pains of developing an electronic preprint server. Chemists, however, face a different set of circumstances.

For one thing, the physicists got their preprint server working before large journals had much of an online presence, and those journals didn’t oppose the new preprint server. Getting the Los Alamos server running before physics journals ventured online “was something of a pre-emptive strike,” Ginsparg says.

Furthermore, many chemists believe that chemistry’s culture differs so much from physics’ that preprints lacking rigorous expert review won’t, on balance, help their science.

For example, there are many more patenting opportunities in chemical research than in physics, and many chemists don’t want to release and share information that may be commercially valuable, say Bovenschulte and Robert J. Parker, general manager of journals and reviews for the Royal Society of Chemistry (RSC) in London.

Moreover, unlike experiments in high-energy physics, most chemistry experiments can be repeated in days, says Parker. That could enable others to claim they did the work first.

There’s also been a long history of paper preprints in the physics community, says Bovenschulte. Before the birth of the Los Alamos electronic server, physicists copied and mailed preprints of unreviewed papers to members of their research communities. In contrast, there has never been much of a preprint culture in chemistry, Bovenschulte says.

Submission numbers indicate that physicists moved faster than chemists to embrace new preprint servers. In the first 4 months after the start of their respective servers, physicists uploaded about twice as many preprints, even though access to the Internet was more difficult in 1991 than it is today and there are fewer physicists than chemists.

The difference between physicists’ love affair with electronic preprints and the chemistry community’s dispute over them runs deep, to a philosophical level, says Ginsparg. Physicists seem to be less reliant on authority than chemists are, he says, and they have never conceded much control to journals.

When an article appears in a peer-reviewed journal, that doesn’t mean that it’s correct, Ginsparg says; it means only that it’s not obviously wrong. He says that he critiques the research papers that he reads, regardless of any peer review they’ve already encountered.

The line between physics preprints and publications is blurring, Ginsparg says. Physicists have already taken a step toward replacing the hierarchy of journals with what he calls a “knowledge network.”

For the good of chemistry

Merino hadn’t been aware of the ACS editors’ congealing opinions when he submitted his paper to both ChemWeb and JOC last year. In the future, however, editors’ positions should be clear. Last month, the ACS journal editors released a formal statement saying that, in general, they won’t consider for publication any work that has appeared on an electronic preprint server.

The ACS editors, however, might distinguish between fully elaborated, submission-ready papers, which can’t be “re-published,” and unpolished, incomplete results similar to those that would be presented at a scientific meeting, says Bovenschulte. The final verdict on these decisions will be made by an individual journal editor on a case-by-case basis.

Bovenschulte says that money played no role in the editors’ decision. Their verdict “is for the good of their journals,” he says. “It’s for the good of chemistry. It’s for the good of the community.”

Yet, could a preprint server harm the ACS journals financially? If such a server becomes successful, then chemists may start to think that it doesn’t matter whether work is reviewed, and the publications could lose subscribers, admits Bovenschulte. On the other hand, he says, chemists might end up especially valuing reports vetted by journals among a sea of unreviewed preprints.

According to Bovenschulte, ACS is trying to walk a fine line between speeding up publication, which now can take several months, and maintaining the quality of research that makes it into print. Although the Chemistry Preprint Server permits readers to offer feedback, he says those commenters might not have the specialized expertise and commitment of reviewers that a journal selects. “It is haphazard peer review,” he says.

Chemist R. Stephen Berry of the University of Chicago heartily disagrees. Like Ginsparg, he says that peer review does not guarantee a paper’s validity. Berry argues that if an article, whether an electronic preprint or a journal publication, seems “important and surprising, then the scientific community will move in and decide if it really is.”

Chemistry journals

In practical terms, a chemistry preprint server can’t succeed if chemistry journals ban papers for which preprints have been posted, Berry says. In fact, the ACS editors’ position might be stunting the new server’s growth.

Not all journals, however, have a policy like that of the ACS editors. England’s RSC doesn’t ban papers that have appeared on a preprint server, says Parker. The society has, therefore, published articles on physical chemistry and chemical physics that authors also placed on the Los Alamos preprint server.

However, Parker says, the RSC considers the final, peer-reviewed article to be the most definitive information for dissemination either in print or electronically. Among members of the RSC editorial board and the journals committee, “there is no great enthusiasm for chemistry preprint servers at the moment,” Parker says.

“That could change if [a server] becomes popular,” he notes.

In the past few years, chemistry journals have begun to post submitted journal articles online very quickly after peer review. Both the ACS and the RSC have systems in place to offer the final version of an article online a month or more before traditional print editions reach a subscriber’s mailbox. One of the RSC’s journals can post electronic versions of peer-reviewed, author-revised papers as soon as 70 days after submission, notes Parker.

With these developments, “it is hard to see the value of the unrefereed preprint,” he says.

Berry argues that more rapid electronic publishing of peer-reviewed papers isn’t enough. It’s using the Internet only to do “a little more efficiently” what journals were already doing in print, he says. In contrast, “the Los Alamos server has just changed everybody’s thinking about what electronic communication can be,” Berry says.

Initial use

Two important questions remain: Do chemists want a preprint server, and will they use one?

Town says the ChemWeb server’s initial use by chemists is promising and may lead to the meteoric growth that the Los Alamos server experienced. As of mid-January, more than 1,700 unique visitors had viewed the most popular chemistry paper. As yet, however, the ChemWeb server remains unproven.

Bovenschulte, for one, doubts that many chemists want the technology right now. From talking to research chemists, he says, “I don’t run into very many who are enthusiastic about a preprint server.”

As for the opinions of chemists in general, many of whom don’t write research papers, Bovenschulte points to a survey mailed to 8,000 randomly selected members of the 160,000-strong ACS in the summer of 1999.

Among the 1,095 who returned surveys, respondents were nearly evenly split on whether they were in favor of the ACS starting a preprint server and also on whether they would contribute to it, provided that they could still publish in society journals.

More than 70 percent said that they would visit a free chemistry preprint server. But less than half said they would cite a paper from it in one of their own articles.

Currently, Town and his ChemWeb colleagues are promoting their preprint server at meetings and exhibitions, as well as in limited advertising. And news of it seems to be spreading by word of mouth, the ChemWeb team says. “The message needs to percolate out,” says Town.

Adventurous chemists will just have to wait to see if electronic servers such as ChemWeb’s will get the foothold in chemistry that they have in physics.

“I think that it will change in a generation,” says Berry. “And I hope it does.”