Home · RSS · E-Mail · GitHub · GitLab · Mastodon · Twitter · LinkedIn

Writing a simple shell in Go

first published:

Updated 2018-07-03: Based on the reddit comments, I’m now using the correct output devices.

Updated 2019-03-21: Here you can find my Rust port of the shell.

» Introduction

In this post, we will write a minimalistic shell for UNIX(-like) operating systems in the Go programming language and it only takes about 60 lines of code. You should be a little bit familiar with Go (e.g. how to build a simple project) and the basic usage of a UNIX shell.

UNIX is very simple, it just needs a genius to understand its simplicity. - Dennis Ritchie

Of course, I’m not a genius, and I’m not even sure if Dennis Ritchie meant to include the userspace tools. Furthermore, a shell is only one small part (and compared to the kernel, it’s really an easy part) of a fully functional operating system, but I hope at the end of this post, you are just as astonished as I was, how simple it is to write a shell once you understood the concepts behind it.

» What is a shell?

Definitions are always difficult. I would define a shell as the basic user interface to your operating system. You can input commands to the shell and receive the corresponding output. When you need more information or a better definition, you can look up the Wikipedia article.

Some examples of shells are:

The graphical user interfaces of Gnome and Windows are shells but most IT related people (or at least I) will refer to a text-based one when talking about shells, e.g. the first two in this list. Of course, this example will describe a simple and non-graphical shell.

In fact, the functionality is explained as: give an input command and receive the output of this command. An example? Run the program ls to list the content of a directory.

Input:

1

ls

Output:

1
2
3

Applications		etc
Library				home
...

That’s it, super simple. Let’s start!

» The input loop

To execute a command, we have to accept inputs. These inputs are done by us, humans, using keyboards ;-)

The keyboard is our standard input device (os.Stdin) and we can create a reader to access it. Each time, we press the enter key, a new line is created. The new line is denoted by \n. While pressing the enter key, everything written is stored in the variable input.

1
2

reader := bufio.NewReader(os.Stdin)
input, err := reader.ReadString('\n')

Let’s put this in a main function of our main.go file, and by adding a for loop around the ReadString function, we can input commands continuously. When an error occurs while reading the input, we will print it to the standard error device (os.Stderr). If we would just use fmt.Println without specifying the output device, the error would be directed to the standard output device (os.Stdout). This would not change the functionality of the shell itself, but separate devices allow easy filtering of the output for further processing.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10

func main() {
    reader := bufio.NewReader(os.Stdin)
    for {
        // Read the keyboad input.
        input, err := reader.ReadString('\n')
        if err != nil {
            fmt.Fprintln(os.Stderr, err)
        }
    }
}

» Executing Commands

Now, we want to execute the entered command. Let’s create a new function execInput for this, which takes a string as an argument. First, we have to remove the newline control character \n at the end of the input.
Next, we prepare the command with exec.Command(input) and assign the corresponding output and error device for this command. Finally, the prepared command is processed with cmd.Run().

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14

func execInput(input string) error {
    // Remove the newline character.
    input = strings.TrimSuffix(input, "\n")

    // Prepare the command to execute.
    cmd := exec.Command(input)

    // Set the correct output device.
    cmd.Stderr = os.Stderr
    cmd.Stdout = os.Stdout

    // Execute the command and return the error.
    return cmd.Run()
}

» First Prototype

We complete our main function by adding a fancy input indicator (>) at the top of the loop, and by adding the new execInput function at the bottom of the loop.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16

func main() {
    reader := bufio.NewReader(os.Stdin)
    for {
        fmt.Print("> ")
        // Read the keyboad input.
        input, err := reader.ReadString('\n')
        if err != nil {
            fmt.Fprintln(os.Stderr, err)
        }

        // Handle the execution of the input.
        if err = execInput(input); err != nil {
            fmt.Fprintln(os.Stderr, err)
        }
    }
}

It’s time for a first test run. Build and run the shell with go run main.go. You should see the input indicator > and be able to write something. For example, we could run the ls command.

1
2
3
4

> ls
LICENSE
main.go
main_test.go

Wow, it works! The program ls was executed and gave us the content of the current directory. You can exit the shell just like most other programs with the key combination CTRL-C.

» Arguments

Let’s get the list in long format with ls -l.

1
2

> ls -l
exec: "ls -l": executable file not found in $PATH

It’s not working anymore. This is because our shell tries to run the program ls -l, which is not found. The program is just ls and -l is a so-called argument, which is parsed by the program itself. Currently, we don’t distinguish between the command and the arguments. To fix this, we have to modify the execLine function and split the input on each space.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11

func execInput(input string) error {
    // Remove the newline character.
    input = strings.TrimSuffix(input, "\n")

    // Split the input to separate the command and the arguments.
    args := strings.Split(input, " ")

    // Pass the program and the arguments separately.
    cmd := exec.Command(args[0], args[1:]...)
    ...
}

The program name is now stored in args[0] and the arguments in the subsequent indices. Running ls -l now works as expected.

1
2
3
4
5

> ls -l
total 24
-rw-r--r--  1 simon  staff  1076 30 Jun 09:49 LICENSE
-rw-r--r--  1 simon  staff  1058 30 Jun 10:10 main.go
-rw-r--r--  1 simon  staff   897 30 Jun 09:49 main_test.go

» Change Directory (cd)

Now we are able to run commands with an arbitrary number of arguments. To have a set of functionality which is necessary for a minimal usability, there is only one thing left (at least according to my opinion). You might already come across this while playing with the shell: you can’t change the directory with the cd command.

1
2
3
4
5

> cd /
> ls
LICENSE
main.go
main_test.go

No, this is definitely not the content of my root directory. Why does the cd command not work? When you know, it’s easy: there is no real cd program, the functionality is a built-in command of the shell.

Again, we have to modify the execInput function. Just after the Split function, we add a switch statement on the first argument (the command to execute) which is stored in args[0]. When the command is cd, we check if there are subsequent arguments, otherwise, we can not change to a not given directory (in most other shells, you would then change to your home directory). When there is a subsequent argument in args[1] (which stores the path), we change the directory with os.Chdir(args[1]). At the end of case block, we return the execInput function to stop further processing of this built-in command.
Because it is so simple, we will just add a built-in exit command right below the cd block, which stops our shell (an alternative to using CTRL-C).

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16

// Split the input to separate the command and the arguments.
args := strings.Split(input, " ")

// Check for built-in commands.
switch args[0] {
case "cd":
    // 'cd' to home dir with empty path not yet supported.
    if len(args) < 2 {
        return  errors.New("path required")
    }
    // Change the directory and return the error.
    return os.Chdir(args[1])
case "exit":
    os.Exit(0)
}
...

Yes, the following output looks more like my root directory.

1
2
3
4
5
6
7

> cd /
> ls
Applications
Library
Network
System
...

That’s it. We have written a simple shell :-)

» Considered improvements

When you are not already bored by this, you can try to improve your shell. Here is some inspiration:

» Conclusion

We reached the end of this post and I hope you enjoyed it. I think, when you understand the concepts behind it, it’s quite simple.

Go is also one of the more simple programming languages, which helped us to get to the results faster. We didn’t have to do any low-level stuff as managing the memory ourselves. Rob Pike and Ken Thompson, who created Go together with Robert Griesemer, also worked on the creation of UNIX, so I think writing a shell in Go is a nice combination.

As I’m always learning too, please just contact me whenever you find something which should be improved.

» Full Source-Code

Below is the full source-code. You can also check the repository, but the code there might already have diverged from the code presented in this post.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61

package main

import (
    "bufio"
    "errors"
    "fmt"
    "os"
    "os/exec"
    "strings"
)

func main() {
    reader := bufio.NewReader(os.Stdin)
    for {
        fmt.Print("> ")
        // Read the keyboad input.
        input, err := reader.ReadString('\n')
        if err != nil {
            fmt.Fprintln(os.Stderr, err)
        }

        // Handle the execution of the input.
        if err = execInput(input); err != nil {
            fmt.Fprintln(os.Stderr, err)
        }
    }
}

// ErrNoPath is returned when 'cd' was called without a second argument.
var ErrNoPath = errors.New("path required")

func execInput(input string) error {
    // Remove the newline character.
    input = strings.TrimSuffix(input, "\n")

    // Split the input separate the command and the arguments.
    args := strings.Split(input, " ")

    // Check for built-in commands.
    switch args[0] {
    case "cd":
        // 'cd' to home with empty path not yet supported.
        if len(args) < 2 {
            return ErrNoPath
        }
        // Change the directory and return the error.
        return os.Chdir(args[1])
    case "exit":
        os.Exit(0)
    }

    // Prepare the command to execute.
    cmd := exec.Command(args[0], args[1:]...)

    // Set the correct output device.
    cmd.Stderr = os.Stderr
    cmd.Stdout = os.Stdout

    // Execute the command and return the error.
    return cmd.Run()
}

Home · RSS · E-Mail · GitHub · GitLab · Mastodon · Twitter · LinkedIn