Unix: Encyclopedia - Unix

Unix

Unix or UNIX is a computer operating system originally developed in the 1960s and 1970s by a group of AT&T Bell Labs employees including Ken Thompson, Dennis Ritchie, and Douglas McIlroy. Today's Unix systems are split into various branches, developed over time by AT&T, several other commercial vendors, as well as several non-profit organizations.

Unix was designed to be portable, multi-tasking and multi-user. The Unix systems are characterized by various concepts: plain text files, command line interpreter, hierarchical file system, treating devices and certain types of inter-process communication as files, etc. In software engineering, Unix is mainly noted for its use of the C programming language and for the Unix philosophy.

The present owner of the UNIX trademark is The Open Group, while the present claimants on the rights to the UNIX source code are The SCO Group and Novell. Only systems fully compliant with and certified to the Single UNIX Specification qualify as "UNIX" (others are called "UNIX system-like" or Unix-like).

During the late 1970s and early 1980s, Unix's influence in academic circles led to massive adoption (particularly of the BSD variant, originating from the University of California, Berkeley) of Unix by commercial startups, the most notable of which is Sun Microsystems.

Unix - History

Unix - 1960s and 1970s

In the 1960s, the Massachusetts Institute of Technology, AT&T Bell Labs, and General Electric worked on an experimental operating system called Multics (Multiplexed Information and Computing Service), which was designed to run on the GE-645 mainframe computer. The aim was the creation of an interactive operating system with many novel capabilities, including enhanced security. The project did develop production releases, but initially these releases turned out to have poor performance.

AT&T Bell Labs pulled out and deployed its resources elsewhere. One of the developers on the Bell Labs team, Ken Thompson, continued to develop for the GE-645 mainframe, and wrote a game for that computer called Space Travel. However, he found that the game was slow on the GE machine and was costly, apparently costing $75 per go in scarce computing time.

Thompson thus re-wrote the game in DEC PDP-7 Assembly language with help from Dennis Ritchie. This experience, combined with his work on the Multics project, led Thompson to start a new operating system for the DEC PDP-7. Thompson and Ritchie led a team of developers, including Rudd Canaday, at Bell Labs developing a file system as well as the new multi-tasking operating system itself. They included a command interpreter and some small utility programs as well. This project was called Unics, short for Uniplexed Information and Computing System, and could support two simultaneous users. The name has been attributed to Brian Kernighan, and was a hack on Multics. Following bad puns of Unics (homophone of eunuchs) being a castrated Multics, the name was later changed to Unix, and thus a legacy was born. The name is also a criticism of the overly general and bloated Multics system - Unix would do one thing, and do it well.

Up until this point there had been no financial support from Bell Labs, when the Computer Science Research Group wanted to use Unix on a much larger machine than the PDP-7. Thompson and Ritchie managed to trade the promise of adding text processing capabilities to Unix for a PDP-11/20 machine, and this itself led to some financial support from Bell. For the first time in 1970, the Unix Operating System was officially named and ran on the PDP-11/20. It added a text formatting program called roff and a text editor. All three were written in PDP-11/20 assembly language. This initial "text processing system", made up of Unix, roff, and the editor, was used by Bell Labs for text processing of patent applications at Bell. Runoff soon evolved into troff, the first electronic publishing program with a full typesetting capability. The UNIX Programmer's Manual was published on November 3, 1971.

In 1973, the decision was made to re-write Unix in the C programming language. The change meant that Unix could later easily be modified to work on other machines (thus becoming portable), and other variations could be created by other developers. The code was now more concise and compact, leading to an acceleration in the development of Unix. AT&T made Unix available to universities and commercial firms, as well as the United States government under licenses. The licenses included all source code except for the machine-dependent kernel, which was written in PDP-11 assembly code. However, bootleg copies of the annotated Unix machine-dependent kernel circulated widely in the late 1970's in the form of a much-copied book by John Lions of the University of New South Wales in Australia (the Lions' Commentary on UNIX 6th Edition, with Source Code), which led to considerable adoption of Unix as an educational operating system.

Development expanded, with Versions 4, 5 and 6 being released by 1975. These versions added pipes, leading to the development of a more modular code-base, increasing development speed still further. V5 and especially V6 led to a plethora of different Unix versions both inside and outside Bell Labs, including PWB/UNIX, IS/1 (the first commercial Unix), and the University of Wollongong's port to the Interdata 7/32 (the first non-PDP Unix).

In 1978, UNIX/32V, for the VAX, was released. By this time, over 600 machines were running Unix in some form. Version 7 Unix, the last version of Research Unix to be released widely, was released in 1979. Versions 8, 9 and 10 were developed through the 1980s but were only ever released to a few universities, though they did generate papers describing the new work. This research led to the development of Plan 9, a new portable distributed system.

Unix - 1980s

AT&T now developed UNIX System III, based on Version 7, as a commercial version and sold the product directly, the first version launching in 1982. However its subsidiary, Western Electric, continued to sell older Unix versions, based on the UNIX System (Versions 1 to 7). To end the confusion between all the differing versions, AT&T combined various versions developed at other universities and companies into UNIX System V Release 1. This introduced features such as the vi editor and curses from the Berkeley Software Distribution of Unix developed at the University of California, Berkeley (UCB). This also included support for the DEC VAX machine.

The new commercial Unix releases however no longer included the source code and so UCB continued to develop BSD Unix as an alternative to UNIX System III and V, originally on the PDP-11 architecture (the 2.xBSD releases, ending with 2.11BSD). Perhaps the most important aspect of the BSD development effort was the addition of TCP/IP network code to the mainstream Unix kernel. The BSD effort produced several significant releases that contained network code: 4.1cBSD, 4.2BSD, 4.3BSD, 4.3BSD-Tahoe ("Tahoe" being the nickname of the CCI Power 6/32 architecture that was the first non-DEC port of the BSD kernel), Net/1, 4.3BSD-Reno (to match the "Tahoe" naming, and that the release was something of a gamble), Net/2, 4.4BSD, and 4.4BSD-lite. The network code found in these releases is the ancestor of almost all TCP/IP network code in use today, including code that was later released in AT&T System V UNIX and Microsoft Windows. The accompanying Berkeley Sockets API is a de facto standard for networking APIs and has been copied on many platforms.

Other companies began to offer commercial versions of the UNIX System for their own mini-computers and workstations. Most of these new Unix flavors were developed from the System V base under a license from AT&T. Others chose BSD instead. One of the leading developers of BSD, Bill Joy, went on to co-found Sun Microsystems in 1982 and create SunOS (now Solaris) for their workstation computers. In 1980, Microsoft announced its first Unix for 16-bit microcomputers called Xenix, which the Santa Cruz Operation (SCO) ported to the Intel 8086 processor in 1983, and eventually branched Xenix into SCO UNIX in 1989.

In 1984, an industry group called X/Open was formed, with the aim of forming compatible open systems, that is, standardize the UNIX systems.

AT&T added various features into UNIX System V, such as file locking, system administration, job control (modelled on ITS), streams, the Remote File System and TLI. AT&T cooperated with Sun Microsystems and between 1987 and 1989 merged Xenix, BSD, SunOS, and System V into System V Release 4 (SVR4), independently of X/Open. This new release consolidated all the previous features into one package, and threatened the end of competing versions. It also greatly increased licensing fees.

Unix - 1990s

In 1990, the Open Software Foundation released OSF/1, their standard Unix implementation, based on Mach and BSD. The Foundation was started in 1988 and was funded by several Unix-related companies that wished to counteract the collaboration of AT&T and Sun on SVR4. Subsequently, AT&T and another group of licensees formed the group "UNIX International" in order to counteract OSF. This escalation of conflict between competing vendors gave rise to the phrase "Unix wars".

In 1991, a group of BSD developers (Donn Seeley, Mike Karels, Bill Jolitz, and Trent Hein) left the University of California to found Berkeley Software Design, Inc (BSDI). BSDI produced a fully-functional commercial version of BSD Unix for the inexpensive and ubiquitous Intel platform, which started a wave of interest in the use of inexpensive hardware for production computing. Shortly after it was founded, Bill Jolitz left BSDI to pursue distribution of 386BSD, the free software ancestor of FreeBSD, OpenBSD, and NetBSD.

By 1993 most of the commercial vendors of Unix had changed their commercial variants of Unix to be based upon SVR4, and many BSD features were added on top. In 1994, OSF stopped the development of OSF/1, as the only vendor using it was DEC, who rebranded it Digital UNIX.

Shortly after UNIX System V Release 4 was produced, AT&T sold all its rights to UNIX to Novell. Dennis Ritchie, one of the creators of Unix, likened this to the Biblical story of Esau selling his birthright for some lentils [1]. Novell developed its own version, UnixWare, merging its Netware with UNIX System V Release 4. Novell tried to use this to battle against Windows NT, but their core markets suffered considerably.

In 1994, Novell decided to split the bundle of UNIX-related assets and sell parts of them. The UNIX trademark and the certification rights were sold to the X/Open Consortium. In 1996, X/Open merged with OSF, creating the Open Group. Various standards by the Open Group now define what is and what is not a "UNIX" operating system, notably the post-1998 Single UNIX Specification.

In 1995, the business of administration and support of the existing UNIX licenses plus rights to further develop the System V code base were transferred to the Santa Cruz Operation. Whether Novell also sold the copyrights is currently the subject of litigation (see below).

Unix - 2000s

In 2000, the Santa Cruz Operation sold its entire UNIX business and assets to Caldera Systems, which later on changed its name to The SCO Group. This new player then started a huge legal campaign against various users and vendors of Linux. The SCO Group has offered various legal theories over the course of several cases. Some of these allege that Linux contains copyrighted Unix code now owned by The SCO Group. Others allege trade-secret violations by IBM, or contract violations by former Santa Cruz customers who since converted to Linux. The most far-reaching theory is that development work that IBM did for AIX is considered a derivative work and therefore also owned by SCO. If this is upheld it would affect all Unix licensees.

Under a program called SCOsource, the SCO Group is now offering licenses to all companies and individuals wishing to use operating systems with code based on UNIX System V Release 4 (and their own release, UNIX System V, Release 5).

However, Novell disputed the SCO group's claim to hold copyright on the UNIX source base. According to Novell, SCO (and hence the SCO group) are effectively franchise operators for Novell, which also retained the core copyrights, veto rights over future licensing activities of SCO, and 95% of the licensing revenue. The SCO Group disagreed with this, and the dispute had resulted in the SCO v. Novell lawsuit.

In 2005, Sun Microsystems also released an open source version of Solaris, called OpenSolaris. The OpenSolaris codebase is intended to take advantage of outside contributions to provide the next versions of Solaris; in addition, it has spawned at least one non-Sun distribution in the form of Jörg Schilling's SchilliX.

The dot-com crash has led to significant consolidation of Unix users as well. Of the many commercial flavors of Unix that were born in the 1980s, only Hewlett-Packard's HP-UX, IBM's AIX, NeXT's NEXTSTEP (later OPENSTEP, now Mac OS X) and Sun's Solaris operating systems are still doing relatively well in the market; players such as Digital Equipment Corporation, Data General, and the original Santa Cruz Operation (now known as Tarantella) have been bought out or gone out of business. The rise of Linux and the open-source BSD implementations as a dominating force in the Unix space has also dealt a damaging blow to commercial Unix development, as some companies opt for open source over closed.

Unix - Standards

Beginning in the late 1980s, an open operating system standardization effort known as POSIX provided a common baseline for all operating systems; IEEE based POSIX around the structure of the Unix system. At around the same time a separate but very similar standard, the Single UNIX Specification, was also produced by the Open Group. Starting in 1998 these two standards bodies began work on merging the two standards, and the latest revisions of both are in fact identical.

In an effort towards compatibility, several Unix system vendors agreed on SVR4's ELF format as standard for binary and object code files. The common format allows substantial binary compatibility among Unix systems operating on the same CPU architecture.

The directory layout of some systems, particularly on Linux, is defined by the Filesystem Hierarchy Standard. This type of standard however is controversial among many, and even within the Linux community adoption is far from universal.

Unix - List of Unixes

This is a list of Unixes (sing. Unix). Each version of the UNIX Time-Sharing System evolved from the version before, with version one evolving from the prototypal Unics. Not all variants and descendants are displayed.

Unix - Research Unix

• Unics (⇢Unix) (1969)
• UNIX Time-Sharing System v1 (1971)
• UNIX Time-Sharing System v2 (1972)
• UNIX Time-Sharing System v3 (1973)
• UNIX Time-Sharing System v4 (1973)
• UNIX Time-Sharing System v5 (1974)
  • UNSW 01 (1978)

• UNIX Time-Sharing System v6 (1974)
  • MINI-UNIX (1977)
  • PWB/UNIX 1.0 (1977)
    • USG 1.0
      • CB Unix 1
  • UCLA Secure UNIX (1979)

• UNIX Time-Sharing System v7 (1979)
  • Unix System III (1981)

• UNIX Time-Sharing System v8 (1985)
• UNIX Time-Sharing System v9 (1986)
• UNIX Time-Sharing System v10 (1989)

• Plan 9 from Bell Labs(1993)

Unix - AT&T UNIX Systems & descendants

Each of the systems in this list is evolved from the version before, with Unix System III evolving from both the UNIX Time-Sharing System v7 and the descendants of the UNIX Time-Sharing System v6.

• Unix System III (1981)
• Unix System IV (1982)
• Unix System V (1983)
• Unix System V Release 2 (1984)
• Unix System V Release 3.0 (1986)
• Unix System V Release 3.2 (1987)
• Unix System V Release 4 (1988)
• Unix System V Release 4.2 (1992)
• UnixWare 1.1 (1993)
• UnixWare 1.1.1 (1994)
• UnixWare 2.0 (1995)
• UnixWare 2.1 (1996)
• UnixWare 2.1.2 (1996)
• UnixWare 7 (1998)
• UnixWare 7.0.1 (1998)
• UnixWare 7.1 (1999)
• UnixWare 7.1.1 (1999)
• UnixWare NSC 7.1+IP (2000)
• UnixWare NSC 7.1+LKP (2000)
• UnixWare NSC 7.1DCFS (2000)
• Open Unix 8 (2001)
• Open Unix 8MP1 (2001)
• Open Unix 8MP2 (2001)
• Open Unix 8MP3 (2002)
• Open Unix 8MP4 (2002)
• SCO UnixWare 7.1.3 (2002)
• SCO UnixWare 7.1.3 Update Pack 1 (2003)

Unix - Free Unix-like operating systems

In 1983, Richard Stallman announced the GNU project, an ambitious effort to create a free software Unix-like system; "free" in that everyone who received a copy would be free to use, study, modify, and redistribute it. GNU's goal was achieved in 1992. Its own kernel development project, GNU Hurd, had not produced a working kernel, but a compatible kernel called Linux was released as free software in 1992 (under the GNU General Public License). These "GNU/Linux" systems are commonly referred to as just Linux. Work on GNU Hurd continues still, although very slowly.

In addition to their use in the GNU/Linux operating system, many GNU packages — such as the GNU Compiler Collection (and the rest of the GNU toolchain), the GNU C library and the GNU core utilities — have gone on to play central roles in other free Unix systems as well.

Distributions, comprising the GNU/Linux operating system plus large collections of compatible software have become popular both with hobbyists and in business. Popular distributions include Red Hat Linux, SuSE Linux, Mandriva Linux, Ubuntu Linux and Debian GNU/Linux.

Yet GNU/Linux is not alone. A free derivative of BSD Unix, 386BSD, was also released in 1992 and led to the NetBSD and FreeBSD projects. With the 1994 settlement of a lawsuit UNIX Systems Laboratories brought against the University of California and Berkeley Software Design Inc. (USL v. BSDi), it was clarified that Berkeley had the right to distribute BSD Unix — for free, if it so desired. Since then, BSD Unix has been developed in several different directions, including the OpenBSD and DragonFly BSD variants.

GNU/Linux and the BSD kin are now rapidly occupying the market traditionally occupied by proprietary UNIX operating systems, as well as expanding into new markets such as the consumer desktop and mobile and embedded devices. A measure of this success may be seen when Apple sought out a new foundation for its Macintosh operating system: it chose NEXTSTEP, an operating system developed by NeXT with a freely redistributable core operating system, renamed Darwin after Apple's acquisition. It was based on the BSD family and the Mach kernel. The deployment of Darwin BSD Unix in Mac OS X makes it one of the most widely-used Unix based systems on the market.

Unix - Impact

The Unix system had a great impact on the surrounding community. Some consider it the most influential operating system in changing other proprietary operating systems, leading Unix to be called "the most important operating system you may never use."

Following the lead of Multics, it was written in high level language as opposed to assembler (assembler was in vogue at the time).

It had a drastically simplified file model compared to many contemporary operating systems. The file system hierarchy contained machine services and devices (such as printers, terminals, or disk drives), providing a superficially uniform interface, but at the expense of requiring indirect mechanisms such as ioctl and mode flags to access features of the hardware that did not fit the simple "stream of bytes" model.

Unix also popularized the hierarchical file system with arbitrarily nested subdirectories, originally introduced by Multics. Other common operating systems of the era had ways to divide a storage device into multiple directories or sections, but they were a fixed number of levels and often only one level. The major proprietary operating systems all added recursive subdirectory capabilities also patterned after Multics. DEC's RSTS programmer/project hierarchy evolved into VMS directories, CP/M's volumes evolved into MS-DOS 2.0+ subdirectories, and HP's MPE group.account hierarchy and IBM's System 36 and OS/400 library systems were folded into broader POSIX file systems.

Making the command interpreter an ordinary user-level program, with additional commands provided as separate programs, was another Multics innovation popularized by Unix. The Unix shell used the same language for interactive commands as for scripting (shell scripts -- there was no separate job control language, like IBM's JCL for example). Since the shell and OS commands were "just another program", the user could choose (or even write) his/her own shell. Finally, new commands could be added without recompiling the shell. Unix's innovative command-line syntax for creating chains of producer-consumer processes (pipes) made a powerful programming technique (coroutines) widely available.

A fundamental simplifying assumption of Unix was its focus on ASCII text for 100% of its I/O package and the assumption that the machine word was a multiple of 8 bits in size. There were no "binary" editors in the original version of Unix - the entire system was configured using text shell commands and the least and greatest common denominator in the I/O system is the text byte - unlike "record-based" file systems in other computers. The focus on text for representing "everything" made Unix pipes useful. The focus on text and 8-bit bytes made the system far more scalable and portable than other systems. Over time text-based applications have also won in application areas, such as printing languages (PostScript - not Interpress - an earlier effort by the same people), and when feasible, at the application layer of the Internet Protocols, i.e. Telnet, FTP, SMTP, HTTP, SIP, XML, etc.

Unix popularised a syntax for regular expressions that found much wider use. The Unix programming interface became the basis for a standard operating system interface (POSIX, see above).

The C programming language, now ubiquitous in systems and applications programming, originated under Unix, and spread more quickly than Unix. The C language was the first agnostic language that did not attempt to force a coding style upon the programmer (e.g. support for 3 types of loops and all types of parameter passing.) The C language was the first programming languages to access a computer's full instruction set (e.g. masking, shifting, auto increment, auto decrement, jump tables, pointers.) However, the unsafeness of C leads to problems such as buffer overflows from C library functions such as gets() and scanf(), which are behind many notorious bugs, including one exploited by the Morris worm.

Early Unix developers were important in bringing the theory of software modularity and re-use into engineering practice.

Unix provided the TCP/IP networking protocol on relatively inexpensive computers, which later resulted in the Internet explosion of world-wide real-time connectivity. This quickly exposed several major security holes in the Unix architecture, kernel, and system utilities.

Over time, the leading developers of Unix (and programs that ran on it) developed a set of cultural norms for developing software, norms which became as important and influential as the technology of Unix itself. See Unix philosophy for more information.

Unix - Branding

In 1994, Novell, the company that owned the rights to the Unix System V source at the time, sold the trademarks of Unix to the X/Open Company (now The Open Group), and sold the related business operations to Santa Cruz Operation. Whether Novell also sold the copyrights to the actual software is currently the subject of litigation in SCO v. Novell.

By decree of The Open Group, the term "UNIX" refers more to a class of operating systems than to a specific implementation of an operating system; those operating systems which meet The Open Group's Single UNIX Specification should be able to bear the "UNIX" and UNIX98 trademarks today, after the operating system's vendor pays a fee to The Open Group. Systems licensed to use the UNIX® trademark include AIX, HP-UX, IRIX, Solaris, Tru64, A/UX and a part of z/OS.

In practice, the term, especially when written as "UN*X", "*NIX", or "*N?X" is applied to a number of other multiuser POSIX-based systems such as GNU/Linux, Mac OS X, FreeBSD, NetBSD, OpenBSD that do not seek UNIX branding because the royalties would be too expensive for a product marketed to consumers or freely available over the Internet; such systems claim that the term has now become a genericized trademark. To avoid this, The Open Group requests that "UNIX" is always used as an adjective followed by a generic term such as "system".

The term "Unix" is also used, and in fact was the original capitalisation, but the name UNIX stuck because, in the words of Dennis Ritchie "when presenting the original Unix paper to the third Operating Systems Symposium of the American Association for Computing Machinery, we had just acquired a new typesetter and were intoxicated by being able to produce small caps" (quoted from the Jargon File, version 4.3.3, 20 September 2002). Additionally, it should be noted that many of the operating system's predecessors and contemporaries used all-uppercase lettering, because many computer terminals of the time could not produce lower-case letters, so many people wrote the name in upper case due to force of habit.

Several plural forms of Unix are used to refer to multiple brands of Unix and Unix-like systems. Most common is the conventional "Unixes", but the culture that created Unix has a penchant for playful use of language, and "Unices" (treating Unix as Latin word) is also popular. The Anglo-Saxon plural form "Unixen" is not common, although occasionally seen.

Unix - Canonical Unix Commands

The most basic Unix commands and utilities are:

• Directory and file creation and navigation: ls cd pwd mkdir rm rmdir cp find
• File viewing and editing: touch more ed vi emacs ex
• Text processing: echo cat grep sort uniq sed awk tail tee head cut tr split printf
• File comparison: comm cmp diff patch
• Misc shell tools: yes test xargs
• System administration: chmod chown ps su w who
• Communication: mail telnet ftp finger ssh
• Shells: sh bash csh ksh tcsh

These are the 60 user commands from section 1 of the First Edition:

ar as b bas bcd boot cat chdir check chmod chown cmp cp date db dbppt dc df dsw dtf du ed find for form hup lbppt ld ln ls mail mesg mkdir mkfs mount mv nm od pr rew rkd rkf rkl rm rmdir roff sdate sh stat strip su sum tap tm tty type un wc who write

For a more complete and modern list, see the list of Unix programs.

Unix - Sources

• Salus, Peter H.: A Quarter Century of UNIX, Addison Wesley, June 1, 1994; ISBN 0201547775
• UNIX History. www.levenez.com. URL accessed on March 17, 2005.
• AIX, FreeBSD, HP-UX, Linux, Solaris, Tru64. UNIXguide.net. URL accessed on March 17, 2005.

See also

• Plan 9 - the successor of Unix developed at Bell Labs by Ken Thompson and others.
• BSD license
• Lions' Commentary on UNIX 6th Edition, with Source Code documents the 6th edition of Unix.
• Unix pipes
  • Filter (Unix)
  • Pipeline (Unix)
• Rare mode
• Single UNIX Specification
• UNIX-HATERS Handbook
• Unix devices
  • /dev/null - the 'bit bucket'
  • /dev/urandom and /dev/random - hardware random number generators
  • /dev/zero - a zero generator
• Unix-like
  • GNU
  • GNU Hurd
  • Cygwin
  • MinGW
  • Minix
  • further BSD descendants
• List of Unix daemons
• Unix manual
• Unix wars
• Computer term origins
• Comparison of file systems
• Open system
• Open standard
• Open format
• Opendocument
• Vendor lock-in
• Embrace, extend and extinguish
• Network effect

Adapted from the Wikipedia article "Unix", under the G.N U Free Docmentation License. Please also see http://en.wikipedia.org/wiki