How to print JSON output from your CLI

JSON Output Mode

C01t has been hard at work building the command line app for project Feijoa. He has taken great care to include all the tips from Naomi and Sam into the app. First of all, C01t chose Go as the programming language. He is planning to release the CLI app for all 3 major desktop operating system: Windows, Linux and OS X. In addition, he picked the spf13/cobra library to speed up development. C01t also decided to support human readable and JSON output from the start for every command.

Finally, C01t has been writing docs and READMEs to capture his coding choices along the way. So today he is documenting the command output pattern.

Command output

Most command line apps focus on producing human readable output. But having machine readable output, such as JSON output, is just as important. Machine readable output unlocks additional use cases, eg. scripting of the CLI, and improves testability. Furthermore, we aim to build helpers for rendering machine readable output from every command.

Output mode flag

To control how the CLI renders output, we define an “output mode” flag. Furthermore, we aim to implement the flag consistently across all commands.


const (
	// OutputModeFlag is the name for the flag that lets user specify the
	// desired output mode.
	OutputModeFlag = "output"

	// OutputModeFlagShorthand is the shorthand value for the OutputModeFlag.
	OutputModeFlagShorthand = "o"
)

In addition, we define the set of values for the flag. We start with human readable output, “standard”, and JSON output. In the future, we can extend the values list easily to support new output modes.


// Valid values for OutputModeFlag.
const (
	OutputModeStandard = "standard"
	OutputModeJSON = "json"
)

Finally, we add the output mode flag as a persistent flag to the root command. As a result, the flag can be set on every command.


rootCmd.PersistentFlags().StringP(
	OutputModeFlag,
	OutputModeFlagShorthand,
	OutputModeStandard,
	fmt.Sprintf(
		"Command output mode. One of: %s, %s", OutputModeStandard, OutputModeJSON),
)

The action function

First of all, each command must collect the data to be rendered in machine readable format. So action functions return both a result struct and error. The signature of an action function could look something like:


func MyAction(cmd *cobra.Command, args []string) (ActionResult, error)

The result struct serves as the collector of data produced as the action is executed. In addition, we use struct tags control the serialization of the results.


type ActionResult struct {
	Message string `json:"msg,omitempty"`
}

Finally, we can emit human readable output inside the action function body. The only limitation is that we must write all output to the writer attached to the cmd param:


fmt.Fprint(cmd.OutOrStdout(), "Hello world")

Since, the app displays information as a command is running, it fells more responsive and interactive. However, we can suppress this output by setting the writer on the cmd param to ioutil.Discard. Doing so keeps human readable output from interfering with the machine readable output.

The helpers

So far we have we have seen the base building blocks for implementing JSON output. Therefore it is now time to see how we tie it all together.

First we create a functions to map the action function we defined previously to the action function signatures expected for the spf13/cobra framework. The code sample below implements a function to map to the RunE action function:


type (
	// RunEAction defines the function signature for the action to be wrapped by
	// RunE.
	RunEAction func(cmd *cobra.Command, args []string) (interface{}, error)
)

func RunE(
	action RunEAction,
) (func(cmd *cobra.Command, args []string) error) {
	return func(cmd *cobra.Command, args []string) error {
		return runE(action, cmd, args)
	}
}

Most noteworthy in the code above is the runE function. The function determines the active output mode. Then it invokes helpers to execute the action and produce the output.


func runE(action RunEAction, cmd *cobra.Command, args []string) error {
	mode, err := cmd.Flags().GetString(OutputModeFlag)
	if err != nil {
		return errors.Wrap(err, "failed to determine output mode")
	}

	switch mode {
	case OutputModeStandard:
		err = runEStandard(action, cmd, args)
	case OutputModeJSON:
		err = runEJSON(action, cmd, args)
	default:
		err = errors.Errorf("unsupported output mode: '%s'", mode)
	}
	return err
}

runEStandard just calls the action function. runEJSON sets the output writer on cmd to ioutil.Discard. Then executes the action function. And renders the result as JSON.

Finally, we declare a command as follows:


cmd := &cobra.Command{
	Use: "my-cmd",
	RunE: RunE(MyAction),
}

Coda

The Green GUAVA cookiecutter-go template has a full implementation of the helpers described. Use the template to build CLI apps with a variety of output modes.


Image by: unsplash-logoJustin Peralta

Exploring the basic anatomy of a CLI

Explore the basic anatomy of a CLI

Our friend C01t is riding the elevator to the 15th floor for a meeting with Naomi, one of the system admins in charge of Guava Orchard’s IT infrastructure. After settling in as a member of project Feijoa, C01t has been tasked with building a command line app. Admins will use the tool to configure, maintain and troubleshoot the Feijoa service. He has never built a CLI before. Furthermore, no one on the team has a clear idea of what the requirements for the tool are. Therefore, Kaya suggested he meet with Naomi to get her input on the requirements.

“Why don’t I show you the Tamarillo command line app”, Naomi proposes once they are settled in the conference room. “Sam and I think its easily one of the best CLIs we use”, she continues nodding at the system admin sitting across from her. “I run Linux. Sam runs windows. Yet both of us can use the Tamarillo CLI from our desktops.” Then Naomi demos the intuitive command structure and contextual help feature of the app.

Next Sam opens one of the bash scripts they wrote to provision new users. To his surprise, C01t notices that the script invokes the same app that Naomi just demoed. “Scripting this tool was easy and straight forward. That capability alone simplifies our jobs a ton”, explains Sam.

“How long have you been using this app?”, asks C01t. “When did we deploy Tamarillo? Maybe 15 months ago”, replies Naomi. “That sounds about right”, confirms Sam. “And you know what has been truly amazing? In these 15 months they have released 22 updates to the command line app. That’s roughly a new version every 3 weeks. And not once did they break or regress existing functionality”, Naomi explains. “That is amazing”, agrees C01t. “They must have a huge army of testers.” Naomi gives him a knowing smile. “As a matter of fact 2 developers spend 20% of their time developing the app. That’s it. There is not army of testers. Automated tests validate all functionality”, she explains still smiling.

The anatomy of a CLI

Naomi and Sam introduced C10t to a well written CLI. The features they highlighted provide an excellent road-map for teasing out the characteristics of a good command line app.

Deliver portability

Our primary goal when developing a CLI is to make it available on all platforms that our customers use. Furthermore, we aim to make installation of the CLI as simple as possible. Therefore, go is our preferred language for the task.

With go you can write the code once and run it everywhere. Additionally, all go programs compile into binary executables. Since version 1.5 go has built-in support for cross-compiling. Check out The Go Cookbook for instructions on how to cross compile your code. In addition, you can find the list of supported compile targets in the official docs.

spf13/cobra and urfave/cli are 2 widely used libraries for building command line apps in the vein of the git command. Both provide a declarative way to define the command and sub-commands. Furthermore, both will handle the parsing and extraction of flags and command arguments. And both implement a contextual help system that enable you to build self documenting apps. Overall the libs are very similar. So you can confidently choose either one.

Design for usability

We aim to make the command line app easy to use. Therefore, we need to be mindful of how commands are organized and how complex the user input is.

Command structure

At the core of every CLI app there are 2 concepts: actions and entities those actions apply to. By organizing our command structure around these 2 concepts, we can build an intuitive self documenting application.

Consequently we can chose from 2 different approaches. The first approach is to build an action centric command structure. In this model the primary commands represent actions and the secondary commands are the entities. To illustrate this approach consider what command to start a service for the Tamarillo app would look like:


> tamarillo start service [args ...]

Another option is to build an entity centric command structure. In this case the primary commands all represent entities and the secondary commands are the actions. Now the same command to start a service would look like:


> tamarillo service start [args ...]

So how do we decide which structure to apply? First we consider the symmetry of the actions. Do all or a plurality of the actions apply to a plurality of the entities? Next we consider the cardinality of the 2 concepts. Does our domain have a large number of actions but only a handful of entities? Or vice versa? If the actions are symmetric we make the primary command the actions regardless of cardinality. Otherwise we use the concept with the lower cardinality as the primary command.

The key is consistency. A consistent structure makes it easier for a new customer to explore the commands. Therefore, we must apply the structure we choose consistently to all commands.

Input

To improve ease of use we avoid requiring the user to provide UUIDs as parameters to commands. Because UUIDs are the preferred entity identifiers in most distributed systems, satisfying this principle is more complicated than it seems.

We recommend using config files to load and save the UUID(s) of the “active” entity(s). If the app is similar to git, where commands are executed in the context of a working folder, then we use a config file per working folder. However, if the commands don’t have a folder execution context we can store any config files in the user’s home folder. To save the UUIDs we can implement explicit app commands for setting the active entity.


> tamarillo service set -id a0744c52-36fe-11e8-b467-0ed5f89f718b

However, our preferred way is to implicitly set the “active” entity to the last entity that was explicitly specified. Then all subsequent commands can omit specifying the UUID explicitly.


# Start the service
> tamarillo service start -id a0744c52-36fe-11e8-b467-0ed5f89f718b

# Query for the status of the serice
> tamarillo service status

# Restart the service
> tamarillo service restart

Another option is to accept entity names instead of unique identifiers. However, for this to work the service has to support entity names.

Output

There is no output harder to read than rows of data where the columns are misaligned. So we align tabular output produced by the app. We use the tablewriter library to format tabular output.

Another priciple is to make information stand out by leveraging colorful text. However, we must be judicious with the use of color. Too much color can actually impair the readability of your output. In a number of our applications we have used the color library to generate colorized output.

Build for versatility

Probably the most consequential omission during the initial design of a CLI is scriptability. Following a few simple patterns we can implement a command line app that is capable of producing JSON output in addition to the human readable one. And as a result one can now easily parse the output using jq in any shell script.

Here are some simple rules of thumb to follow:

  • Output a single JSON object. Even displaying multiple JSON objects with clear delimiters, e.g. “\n”, makes the output considerably harder to parse.
  • Suppress all other output. If we do not any additional output will interfere with parsing of the JSON output.
  • Display errors as valid JSON objects when the command fails.

Structure for testability

Finally, you want to write suite of automated tests to guard against regressions. To accomplish this goal we write both integration tests and unit tests.

We write the integration tests to exercise the app against real services/environments. Here we leverage all the work we did to make the CLI scriptable. Because we can emit JSON output from all commands, we can build robust test cases that validate app behavior. Furthermore, we are able to reliably implement complex test scenarios that requires us to pass parameters between commands.

On the other hand, we use unit-tests to ensure all execution paths are covered, especially failure paths. However, to write unit tests we have to carefully structure the app. First of all, we use client abstractions to interact with external services. During unit-test execution we replace the clients with mocks. Then we make the command action function easy to unit-test by following some simple rules such as: do not access global variables; return detailed results that can be asserted on; propagate errors through return values – never use os.Exit or panic; and pass in an io.Writer to receive all output.

How to build a ruby development environment

How to build a ruby development environment

The beta release date for project Feijoa was moved up 4 weeks. So that gives Kaya and the project Feijoa team only 6 more weeks to complete the beta. C01t, Nick, and Vijay, from the JAM team will pitch to help meet the new deadline. The programming language for project Feijoa is ruby. Since C01t has no experience with ruby, Kaya is his buddy for the first week on the project. “I’ll show you how the project is organized and how the ruby development environment works”, Kaya tells C01t. “Then you can take the rest of the day to setup and poke around the code.”

“Like all other teams here at Guava Orchards, we have fully Docker-ized our build and development environment”, explains Kaya. ruby projects use rake as the build utility instead of make. However, since rake requires ruby to run, project Feijoa has a Makefile wrapper to execute rake tasks inside the devbox container. C01t decides to try out this magical wrapper. So he types make -- -T in the terminal. As a result, the rake -T command executes inside the container and displays all tasks define in the Rakefile. This is awesome. C01t can run unit-tests and lint tools on his changes without having to install ruby natively.

“We use RSpec to write all our tests, simplecov to collect code coverage numbers, and RuboCop for linting and coding style enforcement”, Kaya informs C01t. First C01t gets a quick overview of the tools by reading the “Getting started guide” for project Feijoa. Then he checks out the configuration files at root of the project. C01t notices that .simplecov sets the minimum coverage threshold to 95%. In addition, he spots that .rspec randomizes specs execution order. Finally, he discovers that .rubocop.yml disables the BlockLenght cop for spec files.

“Another important tidbit of information for you to know is, that we use unbuilt gems to organize the codebase of project Feijoa into loosely coupled components”, Kaya tells him. Each top level folder is a component. Therefore, the file and folder structure inside a top level folder looks like a typical gem folder. There is a *.gemspec file and a Gemfile. There is a lib folder, a spec, and sometimes a bin folder.


x/
   .rspec
   .rubocop.yml
   .simplecov
   Gemfile
   Gemfile.lock
   Makefile
   Rakefile
   chairman/
      lib/
      spec/
      chairman.gemspec
      Gemfile
   dev/
      ...
   housekeeper/
      bin/
      lib/
      spec/
      housekeeper.gemspec
      Gemfile
   ...

The ruby development environment

Let us explore how create a ruby development environment for a project using un-built gems. The code samples are representative of what the files would look like for project Feijoa.

Dockerize

Above all we want to enable development of a ruby project like Guava Orchards project Feijoa without requiring a native installation of ruby. Therefore, we aim execute all specs, lint-ing tools and the service or app inside Docker containers. To create the scripts that build and run the containers we will follow the steps in our post: The case of the missing development environment. Therefore, we create the same structure under the dev folder.


dev
  docker
     docker-compose.yml
     devbox
       Dockerfile
       ...

However, this time we use ruby:latest as the base image instead of python:3.6.

rake via make

Because, every ruby development environment should employ rake as its build utility, we have to provide an easy way to execute rake tasks. Running rake requires ruby. Hence all tasks must run in the container. On the other hand, running make only needs the binary to be in a folder on your path. Therefore, we built a simple Makefile executes rake tasks inside of the devbox container instead of its own targets.


DEVBOX_CONTAINER=x_devbox

ifdef NO_DOCKER
  CMD = $(1)
else
  CMD = docker exec -t $(DEVBOX_CONTAINER) bash -c "cd /src && $(1)" 
endif

.DEFAULT:
  $(call CMD, bundle)
  $(call CMD, rake $@)

.PHONY: bundle
bundle:
  $(call CMD, bundle)

The value assigned to DEVBOX_CONTAINER is the name of the container started by the docker-compose.yaml file you created in the dev folder. Furthermore, the devbox container must mount the root of the project folder under /src.

So now if you have a rake task called unit-test, you can execute it via the command make unit-test. (If you copy your make binary and rename it to rake you can run the command as rake unit-test.) Furthermore, you can pass flags to rake by inserting -- before the flags you want passed on. So to see all rake task you would run make -- -T.

Finally, you define rake tasks as you normally would. You can implement them directly in the Rakefile or in *.rake files.

Un-built gems

To effectively work with the un-built gems we create a Gemfile in the root of the project and add all components as dependent gems. We also need to add the test only dependencies of the components explicitly into the root Gemfile. Therefore, the Gemfile in the root folder of project Feijoa would look something like:


source "http://rubygems.org"

group :development, :test do
  path "." do
    gem "chairman"
    gem "housekeeper"
  end

  gem "rake"
  gem "rspec"
  ...
end

Next we define a couple of rake tasks to run specs and lint tools from the root folder. For RuboCop we can use the task defined in the gem:


require "rubocop/rake_task"

desc "Default RuboCop task"
RuboCop::RakeTask.new(:rubocop)

Since all specs are implemented in folders nested inside the component folders, running those specs from the root is a tad more tricky. However, we can still make use of the task implementation from the RSpec gem:


require "rspec/core/rake_task"

components = %i[chairman housekeeper]
namespace :unittest do
  components.each do |component|
    desc "RSpec task for #{component}"
    RSpec::Core::RakeTask.new(component) do |t|
      test_dir = Rake.application.original_dir
      t.rspec_opts = [
        "-I#{test_dir}/#{component}/spec",
        "-I#{test_dir}/#{component}/spec/support",
        "-I#{test_dir}/#{component}/lib",
        "{test_dir}/#{component}",
      ]
    end
  end
  
  # other tasks in namespace unittest
end

Finally, with the tasks in place configuring RuboCop, RSpec, and simplecov is straight forward. We can just drop the .rubocop.yml, .rspec, and .simplecov files into the root folder.

Congratulations. You should now have a functioning ruby development environment.

Keep documentation up to date with a doc generator

Keep documentation up to date with a doc generator

A very important new feature for project JAM, is to provide better visibility into the expiration time of a Jar entity. It also happens to be C01t’s very first end-to-end feature. He is responsible for designing, implementing, and testing in. Therefore, C01t is determined to do a great job.

As a first step, C01t decides try and improve the Jar access APIs. He schedules a brain storming session with Aaron and JP. After a bit of debate they agree to start with enhancing the GetJar API.

For GetJar, C01t will add support for the caller to specify the desired format for datetime fields in the response. First, C01t adds a new optional “options” parameter to the GetJar method. While modifying the method signature, he also adjusts the docstring.


  def GetJar(id, options = None):
    """Retrieve meta-data of a Jar entity.
    
    Args:
      id (str): The unique identifier for the Jar entity.
      options (GetJarOptions): The config values used to customized API behavior.

    Returns:
      Jar: Entity meta-data.
    """
    ...

Furthermore, he defines the value object GetJarOptions. Since he wants a smother code review experience, he follows the style of the module and adds a docsting for the new class.


from typing import NamedTuple

class GetJarOptions(NamedTuple):
  """ Value object used to pass options to the GetJar API.

  Attributes:
    date_time_format (str): The format to use for returning datetime fields.  
  """

  date_time_format: str

And to verify his changes he runs make. In the command output C01t notices that a tool called sphinx was run. So he googles “sphinx python” and finds out that the tool is popular doc generator for python. C01t explores the files generated by sphinx. He is pleasantly surprised to discover that the API changes he made are reflected in the doc thanks to the docstrings he wrote.

Generate documentation from code comments

C01t amended the documentation for project JAM without even knowing he was doing so. The documentation generator picked up his docstrings changes and refreshed the docs. Because the docstrings are stored in the code files, C01t updated them when he modified the code. As a result, code and docs changed in lockstep.

Pretty much all popular programming languages today have a doc generator. Following is a list of recommended doc generators.

python

The doc generator for python is sphinx. Sphinx uses reStructuredText as its markup language. Furthermore, you can use the autodoc extension of sphinx to include docstrings into the generated docs. In addition, python has a style guide for writing docstrings, PEP 257. pydocstyle validates that your docstrings follow PEP 257 conventions.

Once you write your docs using sphinx, you can host them publicly on Read the Docs.

sphinx: website, project, docs, pypi package
pydocstyle: project, docs, pypi package

ruby

One of the most popular doc generators for ruby is YARD. In keeping with the tradition of ruby, you can easily customize and extend YARD. In addition, the tool is fully compatible with other ruby documentation formats. Most notably, you can use it to process RDoc formatted documents.

And if you are looking for service to host documentation for your ruby projects, you can do so at YARD powered RubyDoc.info.

yard: website, project, docs, gem, getting started guide

go

Go, in the form of godoc, has the most tightly integrated doc generator. Not only does godoc parse comments, but it also parses the code itself. Furthermore, the comments read by godoc do not have to follow a special syntax. Instead the comments are governed by a small set of conventions very well explained in the godoctricks tutorial.

godoc.org is the official documentation hosting solution for Go packages. Adding a package to godoc.org is as simple as searching for the package by import path.

godoc: project, docs, godoc-tricks tutorial

java

JavaDoc, the doc generator for Java is the original doc generator. The tool comes bundled with all JDKs and SDKs. So you don’t need to download it separately. To learn how to write documentation comments for JavaDoc follow the official guide.

For your open source Java projects you can host JavaDocs on javadoc.io.

JavaDoc: website, docs

javascript

To document your javascript projects a good choice of generator is ESDoc. The tool processes your source files and parses documentation tags from comment blocks. It supports tags very similar to those used by JSDoc. Due to its core plugin architecture, you can add missing functionality to ESDoc. You can improve the quality of your source code documentation using the “doc coverage” metrics and linting feature of the tool. In addition, you can augment the docs further by leveraging ESDoc‘s ability to import description strings from test code and integrate content from Markdown files into the generated docs.

You can host docs for your javascript libraries on the ESDoc Hosting Service.

ESDoc: website, project, package

php

phpDocumentor is the the de-facto standard doc generator for php. Using phpDocumentor you can generate docs directly from comments in your source files. Like with most other generators you can customize the look and feel of the generated docs via a flexible templating system. In addition, phpDocumentor can perform static analysis of your code and produce graphs and reports.

phpDocumentor: website, project, docscompose package

c#

The c# compiler has built in support for generating documentation from xml documentation comments. The compiler generated file then needs to be processed with a tool such as Sandcastle to generate the final docs. Another useful tool for writing xml documentation comments is automineer. automineer generates and updates xml documentation blocks for your code.

Sandcastle: website, project, docs
automineer: website, docs


Image courtesy of: Bank Phrom

Let me tell you why I love RSpec

Let me tell you why I love RSpec

Carefully balancing his tray so as not to spill the cup of soup, C01t walks slowly over to his teams lunch table. As he sets the tray down on the table he hears Kaya say: “Ian, let me tell you why I love rspec and why I look for similar test frameworks in other programming languages!” C01t wonders what this is all about. Leaning over so as to not disturb the ongoing conversation he asks Nick: “What is rspec?”. “rspec is the BDD test framework for Ruby“, replies Nick. C01t settles into his seat, as Kaya starts to lay out her argument. This is one debate he does not want to miss.

Nested test groups

rspec‘s DSL is a powerful tool for organizing test cases, aka examples. You can declare test groups with the methods describe and context. Use test groups to associate tests that verify related functionality or share the same execution context. Furthermore, you can nest test groups. Nested groups are essentially sub classes of the outer groups and provide the expected inheritance semantics. There is no limit on the depth of the nesting.

To illustrate how to best use the describe and context, let’s consider the following class:


class Froyo
  def add_toppings(toppings)
    ...
  end

  def price(coupon)
    ...
  end
end

First you define a describe with the Froyo as the parameter to identify the class under test. Then, for each method of Froyo you add a nested describe with a string containing the method name as the parameter. (Instance method names should be preceded by "#". Class method names should be preceded by ".".) Finally, we define a nested context for each relevant scenario.

You can implement tests inside any of the describe or context blocks. As a result, when reading this spec you can quickly identify what functionality and scenario is being tested. Additionally, when you need to add additional tests for any of the methods it is obvious where to insert them.

The complete spec file for the Froyo class would look something like:


describe Froyo do
  describe "#add_toppings" do
    # add scenarios and test cases
  end

  describe "#price" do
    it "returns a number >= 0" do
      # some test code
    end

    context "with no toppings" do
      it "costs $4.5" do
        # some test code
      end
    end

    context "with 2 toppings" do
      it "costs $5.0" do
        # some test code
      end

      it "cost $4.5 with free toppings coupon" do
        # some test code
      end
    end

    # more contexts
  end
end

Hierarchical before and after

Like most other test frameworks, rspec provides before and after hooks for performing setup and tear down. And similarly, you can scope the hooks to either a single test case (before(:each) and after(:each)), a group of test cases (before(:all) and after(:all)), or the entire run (before(:suite) and after(:suite)).

But what distinguishes rspec from most other test frameworks, is that you can define hooks inside any test group as well as in a rspec configure (a global configuration section). And for a given test, rspec will find and execute all applicable setup and tear down methods. Therefore, you can decompose test setup and tear down cleanly between the nested test groups.

Finally, consider the following example spec:


describe Froyo do
  before(:all) { puts "Froyo one-time setup" }
  after(:all) { puts "Froyo one-time tear down" }

  before(:each) { puts "Froyo setup" }
  after(:each) { puts "Froyo tear down" }

  describe "#price" do
    before(:all) { puts "price one-time setup" }
    after(:all) { puts "price one-time tear down" }

    before(:each) { puts "price setup" }
    after(:each) { puts "price tear down" }

    it "costs at least $0" do
      puts "costs at least $0"
    end
   
    it "validates the type of the coupon parameter" do
      puts "validates the type of the coupon parameter"
    end 

    # more scenarios and tests
  end
end

If you run the above spec file, you will get the following output:

Froyo one-time setup
price one-time setup

Froyo setup
price setup
costs at least $0
price teardown
Froyo tear down

Froyo setup
price setup
validates the type of the coupon parameter
price teardown
Froyo tear down

price one-time teardown
Froyo tear down

rspec invoked all relevant setup and tear down blocks, without blocks nested in the same group as the tests explicitly referring to blocks in the parent groups.

Memoized subject and let helpers

You can use subject and let declarations inside test groups to replace local test variables with methods whose return values are memoized. The values returned by the subject and let declarations are allocated on first use. Since the return values are memoized, the methods can be used repeatedly within a test.


describe Froyo do
  subject(:froyo) { described_class.new }
  let(:toppings) { %w[sprinkles 'gummy bears'] }

  it "costs $4.50 without toppings" do 
    expect(froyo.price).to eq(4.50)
  end

  it "costs $5.00 with 2 toppings" do
    froyo.add_toppings toppings
    expect(froyo.price).to eq(5.00)
  end
end

So the first test above does not incur the penalty of allocating the toppings array. Furthermore, the second test calls froyo twice and received the same object.

Additionally, both declarations play well with nested test groups, and before(:each) and after(:each) hooks. When executing a test or a before/after hook referencing a method declared via subject or let, rspec searches the test group hierarchy and invokes the method closest to the test.


describe Froyo do
  subject(:froyo) { described_class.new }
  let(:toppings) { [] }

  before { froyo.add_toppings toppings }

  it "costs $4.50 without toppings" do 
    expect(froyo.price).to eq(4.50)
  end

  context "with 2 toppings" do
    let(:toppings) { %w[sprinkles 'gummy bears'] }

    it "costs $5.00 with 2 toppings" do
      expect(froyo.price).to eq(5.00)
    end
  end

  # more scenarios and tests
end

First, you used froyo and toppings inside a before(:each) hook. Then you overwrote let(:toppings) inside the context "with 2 toppings". And during execution, for the test inside context "with 2 toppings" when the before hook in the parent group was executed the overridden value of toppings was used.

Finally, putting it all together you can DRY test code using subject and let:


describe Froyo do
  subject(:froyo) { described_class.new }
  let(:toppings) { [] }

  before { froyo.add_toppings toppings }

  describe "#price" do
    let(:coupon) { "FREE_TOPPINGS" }

    context "with no toppings" do
      it "costs $4.50" do
        expect(froyo.price).to eq(4.50)
      end

      it "costs $4.50 with a free toppings coupon" do
        expect(froyo.price(coupon)).to eq(4.50)
      end
    end

    context "with 2 toppings" do
      let(:toppings) { %w[sprinkles 'gummy bears'] }

      it "costs $5" do
        expect(froyo.price).to eq(5.00)
      end

      it "costs $4.50 with a free toppings coupon" do
        expect(froyo.price(coupon)).to eq(4.50)
      end
    end

    # more scenarios and tests 
  end
end

Test case reuse with shared_examples

Finally, you can use the method shared_examples to define test groups that can be nested into multiple other test groups. The shared test groups are scoped based of where they are defined. Therefore, they are available to for inclusion in the group they were defined in or child groups, but not in sibling or parent groups.

You include a shared test group to be evaluated in the context of another test group using the it_behaves_like method.

Consequently, you can use shared groups to execute a common set of tests for each scenario of the price method of the Froyo class:


describe Froyo do
  subject(:froyo) { described_class.new }
  let(:toppings) { [] }

  before { froyo.add_toppings toppings }

  describe "#price" do
    shared_examples "real price" do
      it "costs at least $0" do
        expect(froyo.price).to be >= 0.0
      end
    end

    context "with no toppings" do
      it_behaves_like "real price"
    end

    context "with 2 toppings" do
      let(:toppings) { %w[sprinkles 'gummy bears'] }

      it_behaves_like "real price"
    end

    # more scenarios and tests 
  end
end

Additional Resources

Probably the two most useful resources when using rspec are:
the official documentation and better specs. Most of all, you should review better specs before writing any tests.

In addition all the code in this post is available in our lab-ruby repository.

Avoid the wrath of the human code style checker

Avoid the wrath of the human code style checker

After 3 days of hard work, C01t has finished implementing his first feature for project JAM. He has even written a couple of unit-tests. Proud of his accomplishment, he submits the change for code review (CR) and heads out for lunch. He expects to be able to commit the change by the end of the day after addressing the few code review comments he is sure to receive.

Back from lunch C01t checks in on his CR. It looks like Ian, the Technical Lead for project JAM, has some comments. Ian’s review reads: “Looks like you are using tabs instead of spaces. I also noticed that some of your lines are longer than 80 characters. Please fix and resubmit CR.” Easy enough to fix, thinks C10t. He quickly fixes the tab vs spaces issue using the editor. Then he starts hunting down the lines longer than 80 characters.

However, just before resubmitting the CR he notices additional comments from Ian. “All class definitions and public methods need a doc-string. Also, I pointed out issues with white space, blank lines, and method and variable names in blueberry.py. Take a look at those comments and fix similar issues everywhere else.”, reads the summary of Ian’s review. So, C01t opens up blueberry.py. “Add 2 blank lines before the class definition”, reads the first comment. “Remove the blank space after the opening brace ‘(‘”, reads the second comment. “Use lowercase with words separated by underscore instead of camelCase for variable and function names”, reads the third comment.

“What on earth is going on? Fixing all this stuff is going to take me at least a day of work. Furthermore, I am bound to miss some instances. None of this stuff impacts correctness. So why does it matter.” Visibly frustrated C01t marches over to Ian’s desk to hash this out in person. 30 minutes later C01t is back at his desk, working on addressing all the comments and muttering to himself. Ian was not receptive to any of his arguments.

After addressing all coding style issues and 2 minor correctness issues, C01t finally receives sign-off on his CR 2 days later. He commits the changes relieved to be done with this feature. However, as he starts working on his next item, he is already dreading the next round of CRs.

Run a code style checker tool

C10t’s first feature commit would have gone much smoother had project JAM made use of a code style checker. A code style checker precisely identifies every violation. Therefore, CR owners can quickly fix issues and reviewers can focus on correctness. Additionally, people don’t feel the need to argue with a tool. They might try and convince their team member to change the tool configuration. But, by and large they are more likely to fix the violations and get on with their day.

Pretty much every popular programming language today has a code style checker. Following is a list of recommended checkers by programming language.

python

The style guide for python is PEP 8. And, the pycodestyle tools checks your python code against the PEP 8 conventions. In addition, python has a style guide for writing docstrings, PEP 257. pydocstyle validates that your docstrings follow PEP 257 conventions.

pycodestyle: project, docs, pypi package
pydocstyle: project, docs, pypi package

ruby

The most popular style guide for ruby is the The Ruby Style Guide and its companion Ruby on Rails Style Guide. Same as python, ruby too has a tool to check your ruby code against the guide. The tool is RuboCop. Furthermore, RuboCop has official documentation where you can learn more about configuration options and how to run the tool.

rubocop: website, project, docs, gem

javascript

JavaScript Standard Style is a widely used coding style for javascript. The standard NPM package implements the tool you can use to check your code against the guide. While standard comes with a lot of snazzy features it does not allow you to customize any of the style rules. To customize rules run ESLint directly with rule modifications layered on top of standard rules.

standard: website, project, docs, npm package

php

The coding style guide used by many php projects is PSR-2. And PHP_CodeSniffer is the tool to use to check your code against the style guide. Furthermore, PHP_CodeSniffer can also automatically correct any coding standard violations.

PHP_CodeSnniffere: project, docs, composer package

java

checksyle is the tool used to enforce coding style guides for java projects. Furthermore, the tool ships with configurations conforming to two popular code style guides: Sun Code Conventions and the Google Java Style Guide.

checkstyle: project, docs

go

The closest go has to a coding style guide is the Effective Go document. Two tools, golint and gofmt, help you keep your code compliant with the coding style outlined in Effective Go.

golint: project, docs
gofmt: docs

c#

StyleCop is the widely used tool to enforce a common coding style in C# projects. Peruse the project documentation for information on how to use this tool.

StyleCop: project, docs

Additional resources

GitHub’s Clean Code Linters showcase provides a handy list of code style checker tools and linters for various languages.

The case of the missing development environment

The case of the missing development environment.

Today is C01t’s first day on project JAM. JAM is developed in python. Since this is his first python project, C01t does not have a python development environment configured. So C01t sets out to setup a python environment on his Microsoft Surface Book.

building ‘twisted.test.raiser’ extension
error: Microsoft Visual C++ 14.0 is required. Get it with “Microsoft Visual C++ Build Tools”: http://landinghub.visualstudio.com/visual-cpp-build-tools

Nick runs Linux. So he is not going to be of any help. The rest of the team run on OS X. So they too have no helpful suggestions. Google it is then. A quick search reveals that there are official python installers for Windows. So C01t downloads and installs the latest version, python 3.6.2. Eager to test his setup he runs make inside the root folder of project JAM. Seems like things are working. pip is installing packages. And then progress comes to a crashing halt. The twisted package failed to install because of missing Microsoft Visual C++.

raiser.c
c:\users\xxx\appdata\local\programs\python\python36\include\pyconfig.h(222): fatal error C1083: Cannot open include file: ‘basetsd.h’: No such file or directory

Maybe, installing the suggested Microsoft Visual C++ Build Tools will help. C01t downloads the install package from >Microsoft’s website and installs the tools. A few minutes later C01t runs the make command again. This time the error is about a missing header file.

ModuleNotFoundError: No module named ‘win32api’

Google to the rescue! http://www.lfd.uci.edu/~gohlke/pythonlibs/ has a twisted 17.5.0 wheel package for python 3.6 built for windows. After downloading and manually installing the twisted package, pip install completes successfully. However, running the scraper fails with a long call stack. The error is a missing win32api module.

overlapped.c
c:\users\xxx\appdata\local\programs\python\python36\include\pyconfig.h(222): fatal error C1083: Cannot open include file: ‘basetsd.h’: No such file or directory

The internet once again has the solution! After running the command pip install pypiwin32, C01t can successfully run make. As a result having JAM running on his machine, the very next day C01t is ready starting to work on his first item. He pulls the latest code from the repository and runs make again. The command fails. JAM now has a dependency on the trollius package. And attempting to install the package fails with a compilation error due to a missing header file error.

Exasperated C10t wonders: “Will I ever get to write any code!”.

Building a development environment

All of C01t’s troubles could have been avoided if project JAM had a standardized development environment. We use Docker and Docker Compose to create multi-platform containerized development environment. First of all, it is critical to create a “devbox” environment for compiling/running, unit-testing and debugging the code.

We use the following folder structure to store the configuration files for the development environment in the projects repository:

dev
  docker
     docker-compose.yml
     devbox
       Dockerfile
       ...

Building the container

We are building a “devbox” environment for project JAM. To build the “devbox” container, we create a Dockerfile in the dev/docker/devbox folder. The starting point for the new containers is one of the official python containers.

FROM python:3.6

Then we customize this container further by installing some additional useful tools such as sudo, vim, wget, iputils-ping etc. We can also update some package already installed, such as pip for python.

RUN apt-get update && apt-get -y install curl wget sudo vim iputils-ping && apt-get clean && rm -rf /var/lib/apt/lists/*
RUN pip install --upgrade pip

Finally we make the startup command ping. This allows us to start the container with compose and attach an interactive shell to it later.

CMD ["/bin/ping", "-i 360", "localhost"]

The complete Dockerfile looks like:

FROM python:3.6

RUN apt-get update && apt-get -y install curl wget sudo vim iputils-ping && apt-get clean && rm -rf /var/lib/apt/lists/*
RUN pip install --upgrade pip

CMD ["/bin/ping", "-i 360", "localhost"]

Running the container

Now we are ready to add a devbox service to the docker-compose.yml file in the dev/docker folder. The new service in the docker-compose.yml file is our “devbox” environment.

We want to be able to run “devbox” environments from multiple projects on the same machine. Therefore, we assign unique names to all artifacts associated with the service. We pick a unique service name, image name and container name. The easiest way to do this is to pre-pend the project name or project acronym. (“cc_wp” stands for Cookiecutter WordPress)

  cc_wp_devbox:
    image: ccwp:devbox
    container_name: cc_wp_devbox

Next we specify the source for the service. The source is the container built using the Dockerfile authored before.

    build: ./devbox

Additionally, we load the root folder of the project as a volume mounted under /src inside the container. Consequently, we are able to edit the source code on the host using our code editor of choice. All changes made to source files on the host are instantaneously available inside the container.

    volumes: 
      - ./../../:/src

Finally, we set an environment variable to mark container as a “devbox” environment. We’ll use the environment variable when we author the Makefile to drive the environment.

    environment:
      DEVBOX: 1

The complete docker-compose.yml file looks like:

version: '2'
services:
  cc_wp_devbox:
    image: ccwp:devbox
    container_name: cc_wp_devbox
    build: ./devbox
    volumes: 
      - ./../../:/src
    environment:
      DEVBOX: 1

Creating the Makefile

The last piece of the puzzle is a Makefile. Above all we want to make the Makefile runable from outside and inside the container. Therefore, we generate a wrapper for all commands based on the DEVBOX environment variable. Remember? We set that when we started the container.

ifdef DEVBOX
	CMD = $(1)
else
	CMD = docker exec -t cc_wp_devbox bash -c "cd /src && $(1)"
endif

Then we author commands for targets and rules to be executed within this wrapper. For example, a target to run pip against a requirements.txt looks like:

.PHONY: install
install:
	$(call CMD, pip install -r requirements_dev.txt)

Finally, a sample Makefile for project JAM looks like:

.PHONY: dev
dev: test

.PHONY: install_dev
install_dev:
	$(call CMD, pip install -r requirements_dev.txt)

# test targets
test: unittest

.PHONY: unittest
unittest: install_dev
	$(call CMD, pytest --verbose tests)

# variables used to determine the command to run
ifdef DEVBOX
	CMD = $(1)
else
	CMD = docker exec -t cc_wp_devbox bash -c "cd /src && $(1)"
endif

A land without templates

Templates to the rescue

Meet C01t, the newest “rock star developer” for Guava Orchards. C01t just had this great idea for a new service. Eager to get started he says to himself: “I’m going to crank out this prototype as fast as I can. If it proves out I’ll go back and fix formatting, add some comments, write a full suite of automated tests… you know turn all this into production ready code”.

The prototype generates all sorts of excitement. A deluge of new feature ideas comes pouring in. C01t bolts more and more code onto the prototype. Everyone who tries the service loves it. Out of nowhere there is talk about launching a beta by the end of the month. “That should be no problem right? After all the services is only missing 2 small features”, they say. That’s when C01t experiences his first moment of panic. This was only supposed to be a prototype!

C01t explains that the service needs a complete rewrite. He has to implement a fully automated test suite. Heck, right now he can only build and run the service on his box. “I need at least 3-4 months to get the service production ready”, C01t declares confidently. “We have to ship within the next 4 weeks. Tell you what! We’ll assign Nick to this project full time. He can help with testing and automating the deployment”, they say.

Nick and C01t set out to make the impossible possible. C01t implements the missing features. He writes some tests. Nick builds a fully automated build/test/deployment pipeline. He sets up monitoring and alerts. He even manages to cobble together a basic getting stared guide. Finally “launch day” arrives. The pipeline has only had one successful run in the last 2 weeks. However, they go head and launch anyways.

Almost from the get go, issues start cropping up. At first the requests take too long to process. Then the error rate spikes. C01t digs in and finds the bug. He codes up the fix. Then Nick deploys the fix to production after fighting the pipeline for a week and a half. A moment of respite. But then the alert for authentication failures goes off. The fix broke the main login flow.

And that’s when C01t decides: “No more prototype code ever again”!

Templates to the rescue

Does C01t’s story sound familiar? So, how can you avoid a similar debacle in the future? Write production ready code from day 1. Use a Green GUAVA template and the cookiecutter command-line utility to generate your project.

Every one of our templates configures your project according to the following principles:

  • Build/development environment: Create a fully isolated and consistent build environment with Docker containers. Furthermore, make builds repeatable by employing a package management tool to control dependencies.
  • Static code analysis: Enforce “programming style” with an automated tool. Style checkers keep the entire code base consistent. In addition they eliminate “religious arguments” during code reviews. Finally, utilize linters to eradicate all common sources of bugs from your code base.
  • Testing: A comprehensive test suite allows you to iterate quickly without regressing existing functionality. Therefore, take advantage of capable test frameworks to facilitate BDD and TDD. Also, set “code coverage” thresholds to prevent functionality from going untested.
  • Documentation: Generate documentation from code comments. Keep documentation close to the code, and update it whenever the implementation changes. You can enable this entire process with “doc generators”.
  • Production ready: Log relevant debug information and collect metrics as you build your features. Use a structured logger to greatly simplify log post-processing.

Kick start your project with a Green GUAVA template.