Wording my graphing

in Programming / Powershell / Static-analysis on Powershell, Static-analysis, Call-graphs

I bet you know the feeling of running Install-Module -Name SomeInterestingModule, waiting, then, with your fingers on the keys, so ready to go play with SomeInterestingModule just downloaded. You stop. Pause. Thinking to yourself: where do I start with this module?

Okay, you could run Get-Command -Module SomeInterestingModule and then Get-Help Get-CommandInSomeInterestingModule. However, the module would still be a blackbox. This is where my function Out-PSModuleCallGraph comes into play. It helps you dive into the inner workings of SomeInterestingModule. The commands the public commands of the module are calling, what those commands are calling and so forth. In other words. A way for you to get a look behind the scenes. And thereby an idea into which commands to go-to in specific situations.

Here is some goodies on how to use Out-PSModuleCallGraph and the experience I had developing it.

Use it!

  1. Run Save-Script -Name Out-PSModuleCallGraph -Path "PATH_WHERE_YOU_WANT_TO_STORE_THE_FILE" (you could also use Install-Script. But with Save-Script you have more control and can look through the code (for security reasons))
    1. Pre-requisites:
      • A connection to the Internet.
      • The PSGallery should be registered for the PowerShellGet module to use (Guide to the PowerShellGallery).
      • PowerShell 5.1+ on your box. (PSCore if on Linux or MacOS).
  2. Dot-source the script. E.g. . "PATH_WHERE_YOU_STORED_THE_FILE". When you want to use it.
    • Here is an example of dot-sourcing > . ./usr/local/share/powershell/Scripts/Out-PSModuleCallGraph.ps1.
  3. After you have dot-sourced the Out-PSModuleCallGraph script, the Out-PSModuleCallGraph command is available in your PowerShell session. So now execute Out-PSModuleCallGraph. E.g. Out-PSModuleCallGraph -ModuleName Pester -ShowGraph. That cmdline will generate a call graph on the Pester module and show you the Graph after it has been generated.

Learning outcomes of developing Out-PSModuleCallGraph.

The best thing about programming is that you learn so much and it gets you deeper into the labyrinth of how stuff works in the world of IT. Not the least, it strongly requires your talent for being creative. It’s just b e a utiful…..Allllllrighty then!!! Let’s go. In the below I will share learnings and tips that I got out of developing Out-PSModuleCallGraph.

Attacking the problem.

In this case it was all about identifying the info I needed to derive from a PowerShell module being analyzed in order to be able to generate a call-graph on the module. I came up with the following list.

  • Identify the public functions.
  • Identify the private functions.
  • Somehow, parse these functions in order to infer what commands/functions they use.
  • Find a way to tag the commands each function uses. So that it can be identified what the scope of the command is. Meaning. If it is a public command in the same module. A private command in the same module or an external command, so not a part of the module.
  • Register the chronological order of the commands called in the functions.

Let’s concentrate on the tough challenges > Identifying the private functions and how to tell the boundaries of a function via code.

How to tell where a function ends

Yeah how exactly do you do that? Especially when you can’t glance at the code with those preying eyeballs. The solution I came up with, was to count brackets (these guys > {}). Funny how something you learned when walking your baby programmer shoes comes in handy later on. So the story goes…..When I started out programming and was so foolish not to use a proper IDE. It was before the days of VS Code, Atom and the like. You could somewhat easily end-up “forgetting” to end a function declaration or so. You would then get a run-time error along the lines of ...missing closing bracket }..... So now you had to figure out where this ending bracket was missing. You then had to count from the starting bracket. Add to your brain memory counter every time you encountered a starting bracket in the code. Detract when a closing bracket was encountered. You then knew you had the closing bracket when the calculation was at a nice round 0. So I basically programmed a function to use with Out-PSModuleCallGraph that calculates the boundaries of a function by counting brackets. Lovely!

Identifying private functions

Identifying the public functions is quite easy. It’s just = Import the module being analyzed » then use the Get-Command Cmdlet to get the public functions of the module.

The private functions though, that is quite different. A private function is … yeah … private. So there is nothing else to do than parse the code of every file in a PowerShell module » determine the declared functions in the files » look into a list of the public functions » if the function is not in the list » it is private. It takes some time and requires juggling around with several collections. But that’s how I did it. You might have asked yourself. What does this dude mean by “…parse the code….” in the above. Let me explain. What is really going on there is e.g.:

# Tokenize the content of the file
$ast = [System.Management.Automation.PSParser]::Tokenize( (Get-Content -Path $PS1File.FullName), [ref]$null)

# Identify the declared functions in the file. Goes from top to bottom. So the first declared function will be at idx 0
[Array]$DeclaredFunctions = $ast.where( { $_.Type -eq "Keyword" -and $_.Content -eq "function" } )

The code above is from the code section in Out-PSModuleCallGraph where private functions is being processed. A PS1 file is read » the internal .NET class *.Automation.PSParser is being used to divide the code in the file into AST tokens. AST stands for abstract syntax tree and is something you can read more about here. It is really powerful so give the link a friendly click. I then, throughout the code use the .Where() method of the AST object to filter on the AST. In order to e.g. get me the declared functions, the AST tokens containing brackets and so forth. What a joy / the AST is my toy.

Tips and tricks

Tips on PowerShell parameter validation

I don’t often see people validate collections. It is the nice thing to do :smile: and it is easy. E.g.:

[Parameter(Mandatory, ParameterSetName="LinesToExclude")]
[ValidateScript({$_.Count -ge 1})]
[System.Collections.ArrayList]$LinesToExclude

validates that the collection of type ArrayList actually contains members. I would say that it is good coding style to fail early. Especially in regards to parameter validation. Whenever you can do that early, please do! It helps the end-user and makes your code cleaner, as some conditionals can be avoided in the code.

Another one is to be specific on the type of the parameter. PowerShell is not a strongly typed language. However, as for many other programming languages it helps to declare the type when declaring variables. The IDE you use can infer much more on the code you are writing. And you will be helping the end-user by declaring the type of a parameter. As PowerShell will be able to tell the end-user that the value they are trying to feed into a parameter is wrong and state what the type needs to be. As you can see by the below example you can get pretty crazy when specifying the type. So be as specific as needs be.

[Parameter(Mandatory)]
[ValidateScript({$_.Count -ge 1})] # Validate that it is not an empty collection.
[System.Collections.ObjectModel.Collection`1[System.Management.Automation.PSToken]]$AST

The type declaration is the [System.Collections.ObjectModel.Collection1[System.Management.Automation.PSToken]]$AST part.

Let me use Out-PSModuleCallGraph to exemplify.

  1. I followed the guide under Use it!
  2. Ran > Out-PSModuleCallGraph -ModuleRoot ./FranzJaeger/FranzJaegerClient/ -OutputFormat jpg -ShowGraph. Which gave me:

Wonderful. What can we reason from that picture.

  • We can quickly get an overview over the number of public functions in the module.
  • The names of every function in the module.
  • What calls what.
  • Which external commands are used.
  • The chronological order by which a function calls the commands it uses.

Kudos

  • Thank you Brian Bunke. Your publishing-scripts blog post helped me publish the script to the PSGallery.
  • Thank you Kevin Marquette. For developing PSGraph. Out-PSModuleCallgraph uses PSGraph to generate the graph after analyzing a PowerShell module.
  • Thank you Mikkel Rømer. For discussing this idea with me and for giving me some tips on how to possibly attack the challenge.

Postludium

Thank you for reading along. I hope you learned something and that you can use the Out-PSModuleCallGraph script/function.

Nota bene. If you bump into a bug or have a feature request feel free to report them to the issues section on this GitHub repository PowerShellTooling-Github.

Over and out :dash:

© 2022. All rights reserved.

Powered by Hydejack v7.5.0