This note looks at Usenet, an internet-based newsgroup system.

It covers -

• introduction - what is usenet?
• structure and operation - how does it work?
• rise and fall - the trajectory from explosive growth to normalisation
• the utopian moment - were early expectations about 'Usenet = Democracy' realistic?
• statistics - figures on usenet traffic and readership
• academic and other studies
• landmarks - key points in the history of usenet

     introduction

Usenet is an internet-based newsgroup system that is independent of (and predates) the web. One enthusiast described it as

a fair, a cocktail party, a town meeting, the notes of a secret cabal, the chatter in the hallway at a conference, the sounds of a friday night fish fry, post-coital gossip, the conversations overhead in an airplane waiting lounge that launched a company, and a bunch of other things.

Another proclaimed that

In its simplest form, Usenet represents democracy, or the right to share almost everything that could be possibly shared from a computer. Most of the material in Usenet is contributed by the same people who actively read Usenet. Thus, the Usenet audience chooses the content and subject matter to be thought about, presented, and debated. In this way, Usenet is a worldwide uncensored forum for debate and informational exchange where many sides of an issue come into view. Rather than being force-fed by an uncontrollable power, the participants set the tone and emphasis of the different groups in Usenet. Without the time and effort put in by its users, Usenet would not be the democratic and resourceful informational forum it is today

It has been increasingly superseded by email mailing list/discussion group services and by blogs.

Usenet participants use newsreader software to read and post email-like messages (often characterised as 'articles) to a number of distributed newsgroups. Some groups are international; others have a more restricted coverage. They feature text and non-text content ('binaries') including images and music.

Downloading and posting involves access to a 'gateway' server, typically provided by an institution or internet service provider (ISP). That server communicates with other servers on a 'store and forward' basis. There is no single master usenet server: administration of usenet is instead distributed, with each usenet server administrator controlling what usenet groups are available to users of the particular server and what groups it propagates in communicating with other gateways.

As with email, another store and forward system, the administrator determines issues such as who can be a user and how long before messages 'expire'. An administrators newsgroup provides for cooperative coordination of the various administrators and their servers.

Usenet has been characterised as a space - if not the space - for "online community" (in particular the Netizen) - and as "the Internet Underground".

     structure and operation

Newsgroups are organised hierarchically under some seventy top-level categories, of which the major (in terms of traffic, users and subcategories) are -

• comp - oriented to ICT professionals and hobbyists, in particular computer science, hardware and software
• sci - relating to research in or application of the non-ICT sciences
• humanities - the arts and social sciences
• soc - 'social issues' and personal interaction.
• talk - debate-oriented, often without the 'hard information' that features in comp and sci
• news - concerned with the news network, group maintenance, and software
• rec - oriented toward hobbies, the arts and recreational activities
• misc - embracing themes outside the preceding categories or across the categories, including regulation
• alt - 'alternative' issues and content, including erotica (and sometimes dismissed as an acronym for "anarchists, lunatics, and terrorists"). It includes alt.activism, alt.alien.visitors, alt.cult-movies, alt.atheism, alt.fan.monty-python and
  alt.folklore.ghost-stories
• biz - business
• 'national' categories - often propagated only to servers within a particular nation - such as aus (including aus.jokes, aus.computers, aus.computers.ibm-pc, aus.religion, aus.general and aus.films).

Usenet groups can be 'unmoderated' (ie any participant one can post) or 'moderated' (posts are automatically directed to a moderator who edits or filters and then posts the results).

Some moderated groups (especially those that are actually gatewayed internet mailing lists) are distributed as 'digests', with groups of postings periodically collected into a single large posting with an index. Some newsgroups have parallel mailing lists for people without netnews access, with postings to the group automatically propagated to the list and vice versa.

Newsreader software (such as Xnews, Gnus and SLRN) allows users to post, reply, followup, find and save articles. They often communicate with the news server using the network news transfer protocol (NNTP) - RFC 977 - based on TCP/IP connections. Adoption of the web has seen many users shift to web-based interfaces for posting and reading. In December 2001 Google (which had acquired the Dejanews service) provided web access to an archive of much of the usenet news since 1981.

In contrast to email newsgroups, users do not subscribe to a mailing list and enjoy substantial anonymity from other users, although not from the administrator of their particular server. They instead access a usenet newsgroup server and select the relevant group from a particular hierarchy.

When a user posts an article that is initially only available on the user's news server. Ongoing communication of 'newsfeeds' between that news server and one or more of its peers results in copying of the article from server to server, in principle ultimately propagating the article across all usenet servers throughout the globe and into an archive such as Googlegroups.

     rise and fall

Adoption of Usenet followed the same trajectory as the early internet, with

• explosive growth from a handful of academic sites located in the US, Western Europe, Australia and other countries
• expansion from the 'hard' sciences - in particular those relating to ICT - and associated interests to a more diverse range of groups that encompassed economics, sociology, cooking, bedwetting and needlework
• normalisation of the Usenet population

It is generally considered to have begun in 1979 as a series of scripts written by University of North Carolina (UNC) grad student Steve Bellovin to facilitate Unix to Unix Copy Protocol (UUCP) communication between UNC and Duke University. These scripts - the basis of the Unix User Network connecting a small number of machines in public and private sector research organisations - were rewritten and extended in a program written in the C computer language by Steve Daniel and Tom Truscott - often referred to as the 'A' release of news.

Usenet started with two hierarchies: mod (all moderated groups) and net (all other groups). US high school student Matt Glickman and Berkeley grad student Mark Horton wrote the 'B' version of news in 1981, enhanced over the following three years and reflecting growth of usenet from an initial three sites in 1979 to 15 in 1980, 150 in 1981 and 400 in 1982. The 'B' version allowed any single group be moderated or open, the great renaming was undertaken. A Usenet backbone was created by Gene Spafford in 1983 to rationalise propagation of Usenet news.

The 'Great Renaming' of 1986 saw restructuring of Usenet, with establishment of several new main hierarchies (comp, misc, news, rec, sci, soc, talk) in addition to the existing net, mod and fa. Reorganization reflected the proliferation of groups (as with internet domain names there were administrative and navigational advantages in increased diversity at the top of the hierarchy) and movement of controversial groups to the talk" domain so that administrators could remove such groups from their newsfeed.

Demise of the original backbone in 1987, as Usenet traffic increasingly involved ARPANET connections, was accompanied by widespread replacement of UUCP by NNTP, pressure for 'democratisation' of the newsgroup creation procedure - notably attacks on "the Backbone Cabal" - and establishment of the alt hierarchy (kicking off with alt.sex, alt.rock-n-roll and alt.drugs under the auspices of figures such as John Gilmore).

Henry Spencer of the University of Toronto created the 'C' version of news in 1988-1989, followed by the proliferation of client newsreader programs. Usenet's coming of age - or irreversible decline, from the perspective of some digerati - was marked in 1993 when America Online offered access to its subscribers.

Acceptance of the web and of listserve programs (with messages being pushed to subscribers rather than to newsgroup peers) over the following decade saw what has arguably been a marginalisation of Usenet, increasingly perceived by some of the online population - and by regulators - as "the dark side of the net", "the home of dirty old men" or spammer's paradise.

One example was TIME magazine's 1995 feature on cyberporn, highlighting dubious claims by Marty Rimm that 83.5% of images on Usenet newsgroups were "pornographic". Another is recurrent comment that users are abandoning Usenet because "worthwhile messages are drowned" by spam.

     the utopian moment

Early writing about Usenet embodied many of the same values evident in advocacy for blogging or open source, with enthusiasts variously claiming that it

• was unprecedented
• offered an opportunity to reclaim truth from corporate media or government
• was necessarily (and properly) 'a technology of freedom'
• was communitarian and self-regulating
• embodied universal cultural values (often tied to a geek perception of the US constitution).

Ronda Hauben thus asserted that

the number of sites receiving Usenet is continually increasing, demonstrating its popularity. People are attracted to Usenet because of what it makes possible. People want to communicate and enjoy the thrill of finding others across the country (or across the world) who share a common interest or with whom to be in contact. Besides the common thrill, it is possible to form serious relationships online. Usenet makes this discovery possible because it is a public forum. People expose their ideas broadly, making it possible to find compatriots in thought. The same physical connections which carry Usenet often also transport private electronic mail. However, the interactions and discoveries are only made possible by the public aspect of Usenet. Mailing lists have as wide a range of discussion, but are available to much smaller groups. Being on Usenet can become tiresome at times, but it is rare that anyone leaves it permanently. Unless, of course, a person's life changes and this change means that time once spent online is no longer available. As more universities, schools, libraries, businesses, and individuals connect, the value of Usenet grows. Each new person can eventually add his or her unique opinion to the collection of thoughts and information that Usenet already has. Each new connection also increases the area where new connections can be made through cheap local phone calls. The potential for inexpensive expansion is limited only by the oceans, other natural barriers, or perhaps by mistaken government policies.

She continued that

The essence of Usenet means it will survive because of its users' determination. Usenet draws its strength from being a peer-to-peer network. People who use Usenet do so because they wish to communicate with others. This communal wish means that people on Usenet find it in their own and in the community's interest to be helpful. In this way, Usenet exists as a worldwide community of resources ready to be shared. Where else today is there so much knowledge that is freely available? Usenet represents a living library and is an important part of the worldwide computer network.

The very nature of Usenet promotes change. Usenet was born outside of established "networks" and transcends any one physical network. It exists of itself and through other networks. It makes possible the distribution of information that might otherwise not be heard through "official channels." This role makes Usenet a herald for social change. Because of the inherent will to communicate, people who do not have access to Usenet will want access when they become exposed to it, and people who currently have access will want Usenet to expand its reach so as to further even more communication. Usenet could grow to provide a forum through which people influence their governments, allowing for the discussion and debate of issues in a mode that facilitates mass participation. This discussion becomes a source of independent information. An independent source is helpful in the search for the truth

As we have suggested elsewhere on this site, those aspirations appear misplaced. Much of the traffic, for example, appears to have related to distribution of pictures of Asian teens, rather than earnest debates between philosopher kings. Governments have not fallen. States have not withered. The medium of enlightenment has not differentiated between chatter about alien conspiracies, Messiaen's Méditations sur le Mystère de la Sainte Trinité and Eminem's latest hit.

Hauben claimed that

Usenet is an uncensored forum for debate where many sides of an issue come into view. Instead of being force-fed by an uncontrollable source of information, the participants set the tone and emphasis on Usenet. People control what happens on Usenet.

The prevalence of flame wars, bullying and spam suggest that questions of control and consent are more subtle. Much of the "robust debate" featured on Usenet is attributable to

• absent or poor moderation (the famous town square in US online creation myths is a fine place to gather for a lynching)
• opportunities for pseudonymous or anonymous communication and use of false names
• differing expectations about netiquette, which in practice often privileges those who are aggressive, impatient with subtlety and different points of view or merely insane.

Henry Hardy commented that

flame wars are the most important means of social constraint on the Usenet system. In the absence of any central administration or much formal structure, flame wars provide a democratic way to air out differences. Even minor shifts in policy or procedure are likely to produce a flame war (as is just one person who had a bad day).

When a flame war begins, lurkers (people who read but never post) and newbies (new users) run for cover. Personal aspersions, outrageous exaggerations, and overheated rhetoric are the order of the day (or week, or month). A person's past transgressions (real or imagined), personal habits and proclivities (real or imagined) and unsupported claims of personal privilege or authority seem to rule the day, for a time. Eventually the source of irritation is removed, removes themselves, cooler heads prevail, or everyone just gets sick of it and moves on to another subject.

     statistics

Identifying and making sense of Usenet statistics is problematical, with most published research dating from the early 1990s and major uncertainty about demographics.

A 1991 analysis of what was characterised as "the top 1000 Usenet sites" showed about 58% US sites, 15% unknown, 8% Germany, 6% Canada, 2-3% each for the UK, Japan, and Australia, and the rest mostly scattered around Europe.

Brian Reid's January 1995 Usenet Readership Summary Report claimed that in July 1994 Usenet was received at around 190,000 sites, accessed regularly by an estimated 7.13 million people (from an overall online population of 23 million) and involved over 72,700 articles per day (some 189 megabytes).

At the same time it was claimed that the average number of netnews readers per site was 86, with articles being kept online for an average of nine days and an estimated 61 terabytes of disk space used globally.

As of 1996 it was estimated that the "alt.binaries" hierarchy comprised 2%-3% of all newsgroups but accounted for 65%-80% of all traffic

A number of commercial and not-for-profit services offer usenet statistics, for example NewsAdmin.

     studies

Much of the scholarly writing about usenet dates from the early 1990s, with attention subsequently moving to chat. The initial work is marked by a concentration on the nature of computer mediated communication (eg flame wars and pseudonymity) and a feel-good fuzziness about 'online community'.

For early views we recommend Tim North's 1994 thesis The Internet and Usenet Global Computer Networks and Henry Hardy's 1993 thesis. Other academic accounts include Richard MacKinnon's (1995) 'Searching for the Leviathan in Usenet' and Margaret McLaughlin & Kerry Osborne's 'Standards of Conduct on Usenet' in Cybersociety: Computer-Mediated Communication & Community (Thousand Oaks: Sage 1995) edited by Steven Jones, Nancy Baym's 'From Practice to Culture on Usenet' in The Cultures of Computing (Oxford: Blackwell 1995) edited by Susan Starr and Michele Tepper's 'Usenet communities and the cultural politics of information' in Internet Culture (London: Routledge 1997) edited by David Porter. Bruce Jones' study An Ethnography of the Usenet Computer Network and Ronda Hauben's 2001 Culture Clash paper offer insights into newsgroups.

Netizens: On the History & Impact of Usenet & the Internet (Los Alamitos: IEEE Press 1998) by Michael & Ronda Hauben is a curious mix of serious research, digital triumphalism and zany info-lib. We suggest that you read the initial chapters and skim the deliciously silly 'Proposed Declaration on the Rights of Netizens' or instead turn to Wendy Grossman's Net.Wars (New York: New York Uni Press 1997).

     landmarks

1976 Mike Lesk at AT&T Bell Labs creates Unix to Unix Copy Protocol (UUCP)

1979 Unix User Network (Usenet) created by Tom Truscott and Jim Ellis at University of North Carolina

1981 'B' version created by Matt Glickman and Mark Horton

1982 USENIX conference differentiates 'news' Usenet from overall UUCPNET network

1985 Network News Transfer Protocol (NNTP) introduced for distribution of Usenet articles over TCP/IP

1986 'Great Renaming' of the Usenet hierarchy

1988 Spencer's C version

1993 AOL offers Usenet access to its subscribers

1995 brouhaha over TIME's publication of claims by Marty Rimm

1995 Deja News establishes Usenet archive

2001 Google acquires Deja archive, offers 20 years of Usenet posts

2005 AOL announces end of its integrated Usenet service

::