32: od

I believe I'd never used od until I wrote this exercise. That's one of the great things about these projects. You'll learn about tools you may never consider while you're learning about the Go standard library.

The od tool is a way to view the contents of a binary file as a sequence of octal values. Think of it as "cat for hackers." It's not the greatest way to view a binary file, and most people can't even read its output, but it is fun to implement.

The Challenge

You'll be required to implement the arguments -w, -x, and -o for the command line tool od. The instructions for od are:

• https://www.gnu.org/software/coreutils/manual/html_node/od-invocation.html

You should try to use each option to understand what they do.

Requirements

While working on od I used the following libraries. You don't have to use these, but if you aren't sure where to start, these libraries are most likely a good place:

Spoilers

If you are totally stuck and can't figure it out, then you can view my code. If you're really, really stuck then take the time to do a copy of my code.

package main

import (
  "fmt"
  "os"
  "flag"
  "log"
  "bufio"
)

type Opts struct {
  Width int
  Filenames []string
  Hex bool
  Octal bool
  Format string
}

func ParseOpts() (Opts) {
  var opts Opts

  flag.IntVar(&opts.Width, "w", 16, "Width of output grid")
  flag.BoolVar(&opts.Hex, "x", false, "Output hex bytes")
  flag.BoolVar(&opts.Octal, "o", false, "Output octal bytes")

  flag.Parse()

  if flag.NArg() == 0 {
    log.Fatal("USAGE: od [files]")
  }

  if opts.Hex {
    opts.Format = "%0.2x "
  } else {
    opts.Format = "%0.3o "
  }

  opts.Filenames = flag.Args()
  return opts
}

func main() {
  opts := ParseOpts()

  for _, filename := range opts.Filenames {
    reader, err := os.Open(filename)
    defer reader.Close()
    if err != nil { log.Fatalf("can't open: %s: %v", filename, err) }

    buf := bufio.NewReader(reader)
    count := buf.Size()

    fmt.Printf("%0.8o ", 0);

    for index := 0; index < count; index++ {
      data, err := buf.ReadByte()
      if err != nil { break }
      fmt.Printf(opts.Format, data);

      if (index + 1) % opts.Width == 0 {
        fmt.Print("\n")
        fmt.Printf("%0.8o ", index);
      }
    }
  }
}

If you do a copy, then follow the Remake It process. Get your copy working, take notes on how it worked, then delete it and do a version based on your notes.

Testing It

In this test we first modify the ParseOpts() to take a new argument:

View Source file go-coreutils/tested/od/main.go Only

package main

import (
  "fmt"
  "os"
  "flag"
  "log"
  "bufio"
)

type Opts struct {
  Width int
  Filenames []string
  Hex bool
  Octal bool
  Format string
}

func ParseOpts(args []string) (Opts) {
  var opts Opts
  myflags := flag.NewFlagSet("od", flag.ContinueOnError)

  myflags.IntVar(&opts.Width, "w", 16, "Width of output grid")
  myflags.BoolVar(&opts.Hex, "x", false, "Output hex bytes")
  myflags.BoolVar(&opts.Octal, "o", false, "Output octal bytes")

  myflags.Parse(args[1:])

  if myflags.NArg() == 0 {
    log.Fatal("USAGE: od [files]")
  }

  if opts.Hex {
    opts.Format = "%0.2x "
  } else {
    opts.Format = "%0.3o "
  }

  opts.Filenames = myflags.Args()
  return opts
}

func main() {
  opts := ParseOpts(os.Args)

  for _, filename := range opts.Filenames {
    reader, err := os.Open(filename)
    defer reader.Close()
    if err != nil { log.Fatalf("can't open: %s: %v", filename, err) }

    buf := bufio.NewReader(reader)
    count := buf.Size()

    fmt.Printf("%0.8o ", 0);

    for index := 0; index < count; index++ {
      data, err := buf.ReadByte()
      if err != nil { break }
      fmt.Printf(opts.Format, data);

      if (index + 1) % opts.Width == 0 {
        fmt.Print("\n")
        fmt.Printf("%0.8o ", index);
      }
    }
  }
}

Then we write the test that simply calls ParseOpts(args) with different options. Here's a simple way to do the test now:

View Source file go-coreutils/tested/od/od_test.go Only

package main

import (
  "testing"
  "github.com/stretchr/testify/assert"
)

func TestParseOpts(t *testing.T) {
  args := []string{"od.exe", "-x", "main.go"}
  opts := ParseOpts(args)
  assert.Equal(t, opts.Hex, true, "-x didn't work")

  // now we can test as much as we want
  args = []string{"cat.exe", "-o", "main.go"}
  opts = ParseOpts(args)
  assert.Equal(t, opts.Octal, true, "-o didn't work")
}

You'll also notice I added the "github.com/stretchr/testify/assert" package to get nicer assert.Equal() functions. We are automating aren't we? Why write 17 lines of test when you could automate it with one then?

To add this module you only need to put "github.com/stretchr/testify/assert" in your import and run:

go mod tidy

Then your test will work when you run it:

$ go test
PASS
ok      lcthw.dev/go/go-coreutils/od    0.004s

To Augment or Not

There's two schools of thought when it comes this kind of testing:

1. You should not change the code you're trying to test so that it's as close as possible to what you actually intend to run in production. Or, to avoid having alternative avenues for others to attack your software.
2. It's better to allow for instrumentation of your code--which is what your tests use--because you'll have to add this instrumentation in production anyway.

To explain both viewpoints we can look at video games. You know how most video games have "god mode" that lets you do what you want? Your tests are kind of using a god mode on your code, and someone else can usually also access this "god mode." Just like with games, if you don't want people to get this kind of access then either don't include these special "escape hatches", or just make sure nobody can access them.

Or, just like with "god mode" in a game, sometimes these instrumented APIs are insanely useful for people who need to use your APIs. That's why the second perspective exists: You'll very often need to hook into these instrumented APIs in production servers to monitor what they're doing or alter how they run. If you sell an API someone uses they may also need this to debug their own servers.

Which to Choose

It's not really a binary decision. There's times when you need tests that thrash the target externally, and there's times when you'll need instrumentation. I recommend learning how to do both since they're both useful strategies depending on the situation.

My approach is to first attempt to test without instrumentation--usually called "black box" or "integration testing"--and then add instrumentation when I can't, or if I know I'll need to have instrumentation in production.