The Section Filter

The section filter can be used to extract indented sections of text based on lines that match a regular expression. By default, the matching lines and any indented sections under the matching lines are printed. Using options, the section start can be changed to be at an earlier line than the pattern match. Selected sections are printed to the standard output. Input is either read from files given as command line arguments, or from standard input.

Documentation

The section filter is documented in the manual page section.1 included in the source code archive. You can read an HTML version of the section manual page online:

HTML version of the section manual page section.1

The section filter prints brief usage information when invoked with either the -h or --help option.

Download

The section filter is GPL (version 3 or later) licensed free software written in the Go programming language. It uses only the Go standard library, no external dependencies. The source code is available for download as a Gzip compressed Tar archive. (I do not provide ready to use binaries.)

Source Code

section-0.9.1.tar.gz

Development Repository

A mirror of the development repository for section is available on GitHub.


Here be Dragons

I have successfully tested compiling section, and using the resulting binary, with both the free software GCC and the open source go.dev (formerly golang.org) Go compilers.

Using the Go Tool to build (go build) or install (go install) software may have privacy and security implications:

Setting some bespoke environment variables, at least one per problematic behavior, to specific values before executing the Go Tool is supposed to deactivate the default active problematic functionality.

By default, section uses make and the GCC Go compiler instead of the Go Tool to avoid these issues. But if the GCC Go compiler is not available, but the Go Tool is, the make build system will automatically fall back to use the Go Tool.


Use Cases

Indentation is often used to indicate hierarchy. This is used in input for programs, e.g., for make. This is also used in output of programs; one example would be iproute2. This is also often used in programming languages, either as a convention how to format source code for good readability, or as part of the language specification (e.g., in Python):

The section filter can be helpful when looking at textual information where indentation is used to denote hierarchy. Thus the section filter can be used when working with any of the above. It takes advantage of the already existing indentation based hierarchy.

Availability of the section filter also allows to forego implementing some output modifier options in other tools. E.g., restricting ip address show output to information pertaining to a single interface (e.g., ip address show dev ens224 could be replaced by ip address show | section ens224). It also allows to extend the existing output modification functionality, e.g., to display the IP address information of more than one, but not all, interfaces with iproute2 (e.g., ip address show | section 'vlan.*@ens224').


Examples


Known Bugs and Limitations

As is the case with most software, section has bugs and limitations.

Line Length

The Go implementation of the section filter uses Go's bufio.Scanner to split input data into lines. This imposes an arbitrary limit for the line length. Instead of the default value of 64KiB, a value of 512MiB is used.

Regular Expressions

The Go implementation of the section filter uses Go's regexp package, and thus inherits its strengths, weaknesses, and peculiarities.

Options

The Go implementation of the section filter uses Go's flag package, and thus inherits its strengths, weaknesses, and peculiarities. For example, single character options cannot be combined, options longer than a single character cannot be abbreviated, and all options can be started with either a single (-) or double (--) hyphen. Additionally, all options must come before the first non-option argument (i.e., the PATTERN).

YAML

While section supports a --yaml-seq-indent option, it does not parse the YAML format. This option rather changes the interpretation of what indentation means to accept YAML block sequence indicators, i.e., dash and space (- ), as a trailing part of indentation. The structure created by these sequence indicators is ignored by section. The section filter always uses indentation depth to determine sections.


Alternative Implementations

Before starting this section implementation in the Go programming language, I wrote three simpler implementations (without any options) in the programming languages AWK (with a shell wrapper), Perl (version 5), and Python (compatible with both versions 2 and 3, but optimized for neither).

In contrast to the Go implementation, they treat Tabulator (tab) characters that are part of leading white space (i.e., indentation) as identical to 8 space characters. Otherwise they implement the same algorithm as the Go implementation of section when invoked without options.

Treating a Tabulator character the same as 8 space characters is generally incorrect, but it works for the default indentation settings of the Vim editor.

My three simpler section implementations are available under the GNU All-Permissive License and can be found in my Single File Tools (SFT) collection:


Variations on the Section Theme

The basic idea behind the section filter is the interpretation of indentation as representing hierarchical structure. The output of the section filter can be changed from the default behavior while still following this idea.

Section Start on a Line Preceeding the Pattern Match

The section implementations mentioned so far always start a section by matching a pattern. It can be useful to also support different section start definitions:

  1. A section could always be started by a line with top level indentation, e.g., no indentation, or the minimum indentation level seen so far. When a line in the input matches the pattern, the whole top level section is selected.

    I have written a simple AWK and Shell implementation of this variant called top-level-section as part of my Single File Tools (SFT) collection.

    The Go implementation of section supports this variant via the --top-level option since version 0.4.0.

  2. A section could be started by the last line before the line matching a pattern that has less indentation than the matched line. This can be described as selecting the section that encloses the matched line, in other words, the enclosing section. If no such line exists, the section would start with the matched line.

    I have written a simple AWK and Shell implementation of this variant called enclosing-section as part of my Single File Tools (SFT) collection.

    The Go implementation of section supports this variant via the --enclosing option since version 0.5.0.

Selecting Additional Lines

For any section that does not start at the top level, it can be useful to also include the starting lines (which could be called section headers) of all enclosing sections up to the top level. This retains the structure of the selected sections in the output of the section filter.

The Go implementation of section supports this variant via the --headers option since version 0.9.0.


Inspiration for the Section Filter

I did not develop the idea for a section filter from nothing. Quite to the contrary, I first encountered a so called section output modifier in the CLI of a proprietary router running Cisco IOS, and used this as inspiration and example. (Such an output modifier has some similarities to a Unix filter, but is much more limited.)

This Cisco IOS section output modifier started a section on the matched line. This works quite well in Cisco IOS, because its configuration display strictly adheres to using indentation for hierarchy representation. This usually also works well for the output of show commands in the Cisco IOS CLI.

Later, I observed different section output modifiers in additional proprietary router (respectively switch) CLIs. On Arista switches, it would select the top level section containing the matched pattern. On Huawei routers, there were three different related output modifiers: section include, section exclude, and section begin. The first two versions selected the section enclosing the matched pattern, to either print or suppress it. The third printed everything starting with the top level section containing the first pattern match.

The Go implementation of the section filter from this web site provides a superset of the functionality of those proprietary output modifiers. The following table shows options that can be used to select behavior similar to those output modifiers.

Router OS Output Modifier Section Filter Invocation
Cisco IOS section section
Arista EOS section section --top-level
Huawei VRP section include section --enclosing
Huawei VRP section exclude section --omit --enclosing
Huawei VRP section begin section --begin --top-level

Related Software

The section filter prints indentation based sections of text by matching a pattern. This complements existing tools that also print text based on matching a pattern, but use different strategies to decide which parts of the text to print.

Grep for Printing Lines

The grep filter prints the individual lines matching a pattern. Some grep implementations, e.g., GNU grep, allow to also print a fixed number of lines preceding and/or following a matched line. (Adding some kind of paragraph output support to GNU grep has been asked for several times through the years, but has not yet been added.)

AWK for Printing Paragraphs

The AWK pattern scanning and processing language allows to print a paragraph containing a pattern match. In this context, a paragraph is a section of text delimited by empty lines. By setting AWK's record separator variable RS to the empty string, providing the pattern enclosed in slash characters, and omitting the action to use AWK's default action of printing the matched record, AWK prints the matched paragraphs:

awk -v RS='' /PATTERN/ [FILE...]

(If you want to do more than just print a complete paragraph, and want AWK to treat each line of a paragraph as one field, you need to set the field separator variable FS to a newline, e.g., by adding the option -F '\n'.)


back to my homepage.