FPS / Shortcuts

Shortcuts – CICD

By j.dickey | Published September 25, 2024 | No Comments

The FPS Aloha CICD (Continuous Implementation, Continuous Development) Shortcut (SC-CICD.ps1) will be used in two different ways.  First will be interactively through the designed shortcut and secondly will be at a particular interval through a scheduled job Poller. The CICD code that runs within the Poller is the same as what runs interactively and will introduce multiple features that are inherent to most encapsulations that you will see in FPS Aloha code. 

[Requirement b5.1] In prior releases of Foundational Ops, locations and the build process were consistently turning up as the root cause for most issues. These issues have been solved within the Aloha release by utilizing an “Initialization” region in each Shortcut/CmdLet that isolates the setting of locational variables and for loading all known dependencies. Using these methods greatly enhances the speed and efficiency of developing and resolving issues when setting locations and loading dependencies.  Also, by loading dependencies using this method allows for the removal of a number of complex code processes employed by legacy build. [Key Insight b5.2] To test the values generated from initializing locations or loading dependencies, set $testCmdLet to the name of the shortcut being tested and the locations/dependencies will be printed as they are set. Similiarly, use $overrideTestSnippet when testing a snippet.  [Key Insight b5.3] When utilizing the initialization region within a Snippet, set $overrideSnippet to the value of the origination snippet before calling the location/dependency snippet.  [Key Insight b5.4] Each module will have a Location Initializer (file format: Initialize-[module]-Locations.ps1) and a Dependency loader (format: Load-[module]-Dependencies.ps1) located in the script location of the Make Module.  For the CICD shortcut, the Initialization region design is as follows:

#region Initialization

$testCmdLet, $overrideTestSnippet, $overrideSnippet = “”, “”, “” # Initialize override variables
. (Get-ActiveFileNm -PS1 Initialize-Make-Locations)
. (Get-ActiveFileNm -PS1 Load-Make-Dependencies)

#endregion

When the Initialization region is used, it can be expanded upon, but the base definition illustrated above must always be present.  For the locations designed, all of the variables created will begin with the acronym used for the module.  The prefix acronyms are: Common Module – $cmn, DataResources Module – $drc, $Make Module – $mak, and Services Module – $svc. The following initialization files should be used within the Initialization region:

To initialize locations: Initialize-Common-Locations.ps1, Initialize-DataResources-Locations.ps1, Initialize-Make-Locations.ps1, and Initialize-Services-Locations.ps1

To load dependencies: Load-Common-Dependencies.ps1, Load-DataResources-Dependencies.ps1, Load-Make-Dependencies.ps1, and Load-Services-Dependencies.ps1

After initialization, the Shortcut proceeds to process directories it the runtime drive is different than the source drive.  If they are the same, then no action is necessary.  Also, the $global:pro_CI.CanConnectToSource variable must be equal to “True” otherwise the source is deemed inaccessible (a vault situation or otherwise isolated domain) and the process will end without action. There two dependencies for CICD: Sync-FPS and the Shortcut Poller.  Sync-FPS does all the syncing of a source directory and has a lengthy dependency model of its own.  The Poller is an always-evolving Shortcut that is run every 15 minutes by a scheduled job to perform redundant time based activities.  Both will be introduced and reviewed within this blog.

Module: Make

Public CmdLet: Sync-FPS.ps1

Dependencies: Get-Folder.ps1, Update-Log.ps1, Remove-DirTree.ps1, Move-SingleFile.ps1

Sync-FPS performs a sync operation from a source location to a target location.  Used in FPS in a CICD perspective, it replaces and obsoletes all of the legacy build processes used in prior releases of Foundational Ops. The CmdLet has a -Sync switch that must be present if an actual sync of the source path to the target path will occur. Otherwise, just a logging of what would changes would be made would occur.  The logs of activity that occur during a sync run are located at [runtime drive]\FPS\Media\Logs\Make\[YYYY]-Sync-FPS\[MM]\[DD]-Log.txt.  [Key Insight b5.5] If either the source or target paths are missing, Get-Answer will be used to enter the paths interactively.  NOTE: If Sync-FPS is used in a batch job it is required to include valid source and target paths.

[Requirement b5.2] Not only does the Sync-FPS sync what exists in the source with the target, it also removes directories and files in the target that are not in the source. Legacy CICD performed only adds and updates forgoing destructive actions best left to the build process.  Since the build process is now obsolete, a better rounded CICD process with destructive actions was necessary.  Now during a CICD process the runtime and the source will always match when the process completes.

The following are the dependencies for Sync-FPS:

Module: Common

Private CmdLet: Update-Log.ps1

Dependencies: SN-SetDateFormats.ps1

The method used for log insertion is to copy the existing log to a scratch location, add the data line, then copy it back to the transcript location.  When the scratch location is in existence, the log is locked, and the CmdLet will wait up to 20 seconds (in two second intervals) in order to claim the scratch area via its own copy.  It will return true if the log was successful and false if not.

Module: Common

Snippet: SN-SetDateFormats.ps1

Dependencies: ConvertTo-ZeroFill.ps1

This script sets the date values for calling Cmdlets. If there is a need for a specific date format for a CmdLet, it should be added to the list in this snippet so it can propagate across the code base.

Module: Common

Public CmdLet: ConvertTo-ZeroFill.ps1

Dependencies: None

This CmdLet left-Zero-pads the string sent for the number of Zeros to make a consistent spacing sequence.  It’s a simple re-write of a Legacy CmdLet and is used a good bit throughout FPS.  If the fill number <= the current length of the str, the original str will be returned.

Module: Common

Public CmdLet: Get-Folder.ps1

Dependencies: None

Uses Robocopy to list all files under the filesystem directory regardless of character limitation on path depth. This file was originally pulled from the internet and has been used as-is since.  Unfortunately, the author and other credits were lost when downloaded. 

Module: Common

Public CmdLet: Remove-DirTree.ps1

Dependencies: None

When deleting files recursively using a simple Remove-Item “folder” -Recurse I sometimes see an intermittent error : [folder] cannot be removed because it is not empty.  This method attempts to prevent the error by individually deleting the files.

Module: Common

Public CmdLet: Move-SingleFile.ps1

Dependencies: None

The CmdLet will either move or copy one file to another location.  If a file exists in the destination directory, -KeepTarget can be used to rename that file to a numeric sub (file_1.txt).  It will not bother any files in the destination directory unless they are in the original directory.  If additional directories are present that are not in the original directory tree, those directories will remain.  If the directory or any of its subs that are in the original directory are not present on the destination directory, those additional directories will be created in the destination.

Closely associated shortcuts:

Module: Make

Public CmdLet: SC-Changes.ps1

Dependencies: None

Identifies what has changed in either the Runtime or Source (-Source param, Runtime is default) for a period of time (-Delta).  The shortcut currently displays files from the {FPS:Containers}, {FPS:Media:Web}, and {FPS:Metadata} directories.  This is useful when doing backfreshes from the Runtime to the Source for syncing offline updates. The output is designed for a UI and should not been tested for use programmatically.

Referenced Source Files

Latest posts