44: curl or wget

I give you a choice here since both tools are really good for accessing network resources from the command line. I chose to implement curl but you could stay in the "GNU family" and implement wget instead. Both will teach the same concepts, and you could even implement both to see if there's differences.

The Challenge

You know the drill, choose the tool you want to implement and make it in Go. When I implemented curl I implemented these options:

• -O -- save the file to disk.
• -I -- inspect the headers

You should implement these too, but if you do wget then you're free to implement whatever you like.

Here's the documentation for both:

• https://curl.se/
• https://www.gnu.org/software/wget/manual/html_node/index.html

Requirements

My implementation of curl only used one new package:

Spoilers

What I find interesting about this project is that you quickly realize accessing a web page is easier than reading a file in Go. Sort of tells you that a company that crawls the web wrote it.

package main

import (
  "fmt"
  "net/http"
  "flag"
  "log"
  "os"
  "io"
  "path/filepath"
)

func main() {
  var save bool
  var headers bool

  flag.BoolVar(&save, "O", false, "output file to local disk")
  flag.BoolVar(&headers, "I", false, "inspect headers")
  flag.Parse()

  if flag.NArg() == 0 { log.Fatal("USAGE: curl [-O] [-I] URL") }

  target_url := flag.Args()[0]

  resp, err := http.Get(target_url)
  if err != nil { log.Fatal(err) }

  if headers {
    for key, values := range resp.Header {
      for _, val := range values {
        fmt.Printf("%s: %s\n", key, val)
      }
    }
  } else if save {
    output := filepath.Base(resp.Request.URL.Path)

    out, err := os.Create(output)
    if err != nil { log.Fatal(err) }

    io.Copy(out, resp.Body)
  } else {
    io.Copy(os.Stdout, resp.Body)
  }
}

Testing It

If you want to push your learning further then you can try to implement an automated test for this. I actually need to learn how to test utilities like this with Go, so for now just consider this an extra challenge for later until I learn how to teach it.