What is a shell?Nearly every Unix and Linux manual has the standard diagram of a shell wrapping around the operating system, resembling some kind of candy bar. The shell really is nothing but an interface between the operating system, including the kernel, the file system and the various system calls and the user. For many years, it was the only interactive user interface before graphical user interfaces became common in the 1980s. Graphical user interfaces could also be considered a type of shell, as they serve many of the same functions: launching programs, configuring the system and managing files.
These humble text-based interfaces have a surprising amount of power. For one thing, they’re fully-fledged programming languages. Before the appearance of even more powerful scripting languages like Python, shell scripts were ideal for writing programs that didn’t necessarily need the power of C. They’re still useful for automating system tasks and for rapid prototyping.
They also have a number of features that make working with and finding files easier. One of the most widely used is "wildcarding" or "globbing." Almost all Unix and Linux users are familiar with the "*" wildcard to match any character. This is actually the shell’s job. Different shells have even more powerful options.
One of Unix’s distinctive features is the ability to redirect program input and output. The shell implements this functionality.
The shell is just another program, so it’s possible for any programmer with the right skills to create one. There have been several major shells that have emerged over the years.
History and a Roundup of ShellsAlthough there were several Unix shells in the early days of the operating system, the first one to get major recognition outside of Bell Labs was the Bourne Shell, named after Stephen R. Bourne. The shell's main innovation was that it supported features for structured programming, making it possible for the first time to use the shell as a real programming language. It’s so indispensable that all modern Unix and Linux versions still use it, though it’s usually one of the newer shells emulating the Bourne shell.
The next major shell was the C Shell, commonly abbreviated as "csh." This shell was developed at UC Berkeley, becoming a major component of the BSD flavor of Unix. As the name suggests, its syntax is designed to resemble the C programming language, but it was really designed for interactive use.
It included a history mechanism that allowed users to go back and repeat any commands they issued earlier without having to retype an entire line and improved job control, which made running multiple tasks easier. (Remember, this was a time when most people still used text-based terminals.)
The next major shell was the Korn Shell, which also came out of Bell Labs. The shell was named after David Korn, not the band, by the way. The Korn shell’s main innovation is the introduction of command-line editing, extending the history functionality even further. Users can go back and edit the commands they’ve typed using commands similar to either the vi or Emacs editors.
Of the major shells, the Bourne Again Shell, or bash, is the most popular since its introduction in the late '80s. This shell, developed as part of the the GNU project, incorporates the innovations of the C and Korn shells while maintaining compatibility with the Bourne shell, hence the name. It’s the "standard" shell on most Linux distributions.
The Z Shell (zsh), first released in 1990, is a command-line user’s dream. Not only does it have most of the other major features that the other shells have, it’s insanely customizable with lots of powerful features. One of the most powerful is recursive globbing, which allows users to match filenames in subdirectories when issuing commands rather than files in the current working directory. Really advanced users can also customize the completion options, matching files without having to type them out completely. And for the fat-fingered typists, it can also correct your spelling. This shell is so advanced, its manual page has been split into several very long sections.
ScriptingAs previously mentioned, shells are not just command line interfaces, but powerful programming languages. The beauty of shell scripting is that you can use the same language in both regular interactive use as well as in scripts, which makes the learning curve much flatter. Modern shells include all the usual programming language features, including flow control, functions and variables. A few of them even have advanced data structures like associative arrays.
Despite their power, programming in shells has a few pitfalls. The biggest problem is that it’s too easy to write scripts that depend on some program that may not be on another system, or that depends on a particular flavor of Unix or Linux. That’s why shell scripts are best suited for programs you know are only going to be run on one system. If you’re trying to build something portable and don’t want to write a C program, your best bet is to write in another scripting language like Perl or Python.