PowerShell - Lesson 2

Overview

In this lesson, you will learn about the PowerShell pipeline, some design patterns, and useful parameters.  To follow along, you will need to have PowerShell 7.0 installed and an understanding of variables and cmdlets.  If you need help with these pre-requisites, check out Lesson 1.

Pipeline

In a traditional programming language, output from one step is stored in a variable and then used as input for the next step.  This can be inefficient because data has to be held in memory at each step.  To work around this, scripting languages have used something called the pipeline.  The pipeline is like connecting different sprinklers together.  The water comes out of the spigot through one hose to the first sprinkler then another hose to another sprinkler and on down the line.  At each step, some work is done and some water passes to the next step.  In a script pipeline, data is generated at the first step and passed as input to the next step.  That output is passed along as input to the next step and so on.  In PowerShell, the commands in a pipeline are connected by the pipe (|) character.  When the pipe character is used, the output of the command on the left is sent as a special parameter to the command on the right.

Here we get take a list of names, select all the ones where the first letter is further along in the alphabet than "C" and the sort the results in descending alphabetical order.  You can see where the 5 names initially entered the pipeline and were piped (|) to the next command.  Where-Object then took each name and evaluated the first letter to return only the names starting with letters greater than "C".  That output was passed along the pipeline to Sort-Object.  Sort-Object then sorted the remaining names in Descending order.  At this point, the pipeline is finished and the results are written to the console.

In PowerShell, elements of the input are passed as individual objects through the pipeline.  Each step of the pipeline is executed on each input object.  This is more efficient, because resources are only used to store one object at a time.  For large input sets, this can be much more efficient provided that cmdlets that don't need all the data at once are used.

Not all cmdlets accept pipeline input.  To determine if a particular cmdlet supports it, run Get-Help -Full against the cmdlet and look for a parameter that accepts pipeline input

To recap, the pipeline is a series of commands with the output of one command linked to the input of another command.  Every object of the input is processed through the pipeline in an efficient manner.

Design Patterns

In programming, there are certain standard ways to solve problems.  We call these design patterns.  There are literally books written to teach these to programmers. There are several useful examples that can be used as Design Patterns in PowerShell.

Input - Process - Output

The Input - Process - Output pattern provides for three steps.  The Input step gathers data to process.  It can be a static list like the names above, an import from a file, or a query against a directory service.  The Process step is normally a Foreach-Object cmdlet.  This step specifies what will be done with the input.  It can be a transform of some type, a step that adds data to the input, or a conversion that gets entirely new data based on the input.  The Output step takes the work that was done in the Process step and dumps it out.  It can either display it to the console, a file, or something else (like Out-GridView).

In the example above, we took a list of names, ran a query against Active Directory to get the Description, and then wrote the values to the console.

1. Input - the list of names
2. Process - query the Description value from Active Directory
3. Output - Output the description value in a list

This is a very useful pattern.  There are a number of examples where you have a list of account names and need to gather data.  Or you have a query to run against a directory and then transform the data to be output to a file.

Input - Process - Select/Sort - Output

A minor variation of the Input - Process - Output pattern is the Input - Process - Select/Sort - Output pattern.  In this pattern, we add an additional step that Selects or Sorts certain records.

The Select/Sort step can be done with a combination of the Where-Object, the Select-Object, and the Sort-Object cmdlets.

• Where-Object - The Where-Object cmdlet can be used to filter the out unneeded values before returning them as output.
• Select-Object - The Select-Object cmdlet can be used to filter the returned object's properties to the desired set before the output is generated. This includes generating new values based on properties returned in the Input and Process stages. It can also be used to eliminate duplicate values.
• Sort-Object - The Sort-Object cmdlet is used to sort the output on one or more properties either Ascending or Descending.

Input - Process

The Input - Process pattern is for scenarios where no output is needed.  It works best when you have a list of input that needs some action performed.  It's similar to the above patterns but produces no output.  In this scenario, the ForEach-Object cmdlet is run on a set of input and performs some action on each object.

In the example above, three accounts are enabled in Active Directory based on a list of input.

When you are given a task or find you self starting a problem, ask if it can be solved by applying one of these patterns.  Most of the time the answer is yes.  Having a plan out of the starting gate can speed you along and reduce the amount of time it takes to complete the task.

Useful Cmdlet Parameters

There are three cmdlets that are useful in these design patterns and I want to review some of their features.

Where-Object (where, ?)

Most people know Where-Object for it's ability to compare a value or property to another value and return only matching values.  We have looked at a couple of examples so far.  

One interesting tidbit is that the -FilterScript parameter, which handles filters specified inside curly braces ({,}), is actually executing a script block.  The normal comparison operators used inside the curly braces are a valid script.

This means we can actually put ANYTHING that is a valid script and Where-Object will execute it on every object passed in.  Any objects where the script returns True are passed and any that return False are not.

The example above searches a file for any matches and returns items where a match is found.

Select-Object

Select-Object has a few interesting and useful switches.  It can be used to get a small sample of data for testing a pipeline with the -First or -Last parameters.

This is very useful for working with large datasets and creating batches.  You can select the first 100 objects to use as a first batch.  You can then select subsequent batches using the -Skip parameter.

The last line is an example of a batch of 100 users after completing 500 previous users.

Normally, when you run Select-Object you get back a PowerShell object with the properties you selected.  This happens even if you only select one property.

If you want to get just back one property, you can use the ExpandProperty parameter.  The ExpandProperty parameter takes one field as input and returns just that value for each object.

This is very useful if you want to do further processing in the pipeline on these values or to output a list of names or values to a file.

Another useful parameter, is the Unique parameter.  It allows you to eliminate duplicate values from the input.  Unique will either work on the input as is or you can specify values to select as well.

As you can see, the Unique parameter eliminated the duplicate values from my array of strings.  It also eliminated the duplicate values from the properties of the AD search of active accounts.  The second command shows with the Unique parameter and the last one shows the same command without the Unique parameter.

The last feature of Select-Object I'm going cover is advanced parameter selecting.  This allows you to build a value and assign it a label.  This is useful if you need to select a sub-property of an object or call a method to get a value.  The format is

@{label="<label>";expression={<expression>}}

you can also use a shorthand version like this

@{l="<label>";e={<expression>}}

The label is just a string used as a name for the returned property.  The expression is a scriptblock containing the command to run on each item to generate the value.  As with Where-Object, the object passed in the pipeline is represented by $_.

In the example above, the advanced parameter selection is used to make two values from an input of 25 random numbers (1 to 99).  One value is the number itself.  I called this one Value and the expression was just $_.  The second one was the number rounded down to the nearest 10.  I called this one Bucket and used some math to round down.  The last three lines of the command just group the output by the Bucket value, sort it, and format it as an Auto-sized table for better readability.  You can see the raw object values that were the output from the Select-Object cmdlet in the Group column.  Each one has a Value and Bucket listed.

Summary

In this lesson, we learned about the PowerShell pipeline, some basic design patterns, and a few useful cmdlets.  The pipeline is a very useful construct that allows stringing multiple cmdlets together to perform a task.  The Design Patterns provide a template that can be used solve common problems.  Use them as a reference to accelerate your tasks.  Finally, there are some very useful and commonly used features and switches of the Where-Object and Select-Object cmdlets.  Learn these features and recognize how and when to apply them.