Artifact Migration via Pipeline
This guide explains how to create and run an Azure DevOps YAML pipeline to migrate ProcessFactorial artifacts (Flows, Forms, Integration Orchestrations) in bulk from one installation to another.
Migration has 2 phases:
- Export artifacts from source.
- Import exported artifacts into target.
Prerequisites
- Azure DevOps permissions to:
- Create/edit YAML pipelines.
- Create/read variable groups in Library.
- Queue pipeline runs.
- Repository access containing:
validateMigrationParams.ps1exportArtifacts.ps1importArtifacts.ps1.azure-pipeline/migrate-artifacts.yml
- Source ProcessFactorial identifiers:
sourceCustomerId: Guid of the customer created in the source environment. Can be easily copied from the URL on the PF portal.sourceProjectId: Guid of the project within the customer created in the target environment.
- Target ProcessFactorial identifiers:
destCustomerId: Guid of the customer created in the target environment.destProjectId: Guid of the project within the customer in the target environment.destEnvironmentId: Guid of the logical environment within the target project.
- Source authentication values:
<SOURCE>_BASE_URL(for examplehttps://<sourceportalbackend>.azurewebsites.net)<SOURCE>_CLIENT_ID(AAD app client id)<SOURCE>_CLIENT_SECRET(AAD app secret)<SOURCE>_TENANT_ID(AAD tenant id)
- Target authentication values:
<TARGET>_BASE_URL(for examplehttps://<targetportalbackend>.azurewebsites.net)<TARGET>_CLIENT_ID(AAD app client id)<TARGET>_CLIENT_SECRET(AAD app secret)<TARGET>_TENANT_ID(AAD tenant id)
-
Optional for Custom mode:
artifactSpecs.jsonfile in the repo path referenced by theartifactSpecsparameter.- Sample file format given below:
{ "forms": [ { "id": "9f7a2a43-bcac-49ca-96e2-2a1ccbac91c0", "version": "1.2.0.0", "export": true, "import": true } ], "flows": [ { "id": "c701bc60-e926-493b-a4bf-edd0d5701354", "version": "1.2.0.0", "export": false, "import": false } ], "integrationOrchestrations": [ { "id": "e8531af9-8413-4e8e-863e-56c45dc060d2", "version": "1.4.0.0", "export": true, "import": true } ] }
Variable Group Setup (Credentials and Endpoints)
Create a variable group in ADO (Library) and add the following variables.
Source variables:
Note: Replace SOURCE in the variable names, with an appropriate prefix for the environment. For instance, if you want to export from the 'DEV' environment, rename the variable for storing the base url as DEV_BASE_URL and likewise for the other variables
<SOURCE>_BASE_URL<SOURCE>_CLIENT_ID<SOURCE>_CLIENT_SECRET(secret)<SOURCE>_TENANT_ID
Target variables:
Note: Replace TARGET in the variable names, with an appropriate prefix for the environment. For instance, if you want to export from the 'SIT' environment, rename the variable for storing the base url as SIT_BASE_URL and likewise for the other variables.
<TARGET>_BASE_URL<TARGET>_CLIENT_ID<TARGET>_CLIENT_SECRET(secret)<TARGET>_TENANT_ID
Use the same variable group name referenced in YAML (migrationenvs) or update the YAML accordingly.
Finally, you should have something that resembles this:

How to Create the Pipeline in ADO
- Go to
Pipelines->New pipeline. - Select the repository containing this YAML.
- Choose
Existing Azure Pipelines YAML file. - Select
.azure-pipeline/migrate-artifacts.yml. - Save and run.
How to Run the Pipeline
At run time, provide these parameters:

artifactType: one or more ofFlows,Forms,IntegrationOrchestrations.artifactSpecMode:All: pipeline discovers IDs and exports latest published versions.Custom: provideartifactSpecsJSON path and export those IDs/versions directly.
- Source IDs:
sourceCustomerIdsourceProjectId
- Target IDs:
destCustomerIddestProjectIddestEnvironmentId
- Optional (Custom mode):
artifactSpecs(default./artifactSpecs.json)
High-level diagram
flowchart TD
%% =================================================
%% ENTRY (MANUAL)
%% =================================================
Start((Manual Run))
%% =================================================
%% PARAMETERS
%% =================================================
Params["Migration Parameters<br/>
• Artifact Type(s)<br/>
• Spec Mode (All / Custom)<br/>
• Source IDs<br/>
• Destination IDs"]
Start --> Params
%% =================================================
%% STAGE 1 – VALIDATE PARAMETERS
%% =================================================
subgraph VALIDATE["Stage 1 - Validate Parameters"]
direction TB
ParseParams["Run validateMigrationParams.ps1"]
Decision{"artifactSpecMode = Custom?"}
ParseParams --> Decision
end
Params --> VALIDATE
Decision -->|Yes| CustomArgs["Include artifactSpecs.json"]
Decision -->|No| AllMode["Migrate All Selected Types"]
%% =================================================
%% STAGE 2 – EXPORT
%% =================================================
subgraph EXPORT["Stage 2 - Export Artifacts (Source)"]
direction TB
SourceEnv[(Source Environment)]
ExportScript["Run exportArtifacts.ps1"]
ExportedFiles[(Exported Artifacts ZIP Files)]
Publish["Publish Build Artifact exported-artifacts"]
SourceEnv --> ExportScript --> ExportedFiles --> Publish
end
CustomArgs --> ExportScript
AllMode --> ExportScript
%% =================================================
%% STAGE 3 – IMPORT
%% =================================================
subgraph IMPORT["Stage 3 - Import Artifacts (Destination)"]
direction TB
Download["Download exported-artifacts"]
DestEnv[(Destination Environment)]
ImportScript["Run importArtifacts.ps1"]
Download --> ImportScript --> DestEnv
end
Publish --> Download
%% =================================================
%% STYLES
%% =================================================
style VALIDATE fill:#ecf0f1,stroke:#7f8c8d,stroke-width:2px
style EXPORT fill:#eafaf1,stroke:#27ae60,stroke-width:2px
style IMPORT fill:#ebf5fb,stroke:#3498db,stroke-width:2px
YAML Used by the Pipeline
trigger:
branches:
include:
- none
name: 'migration-$(Date:yyyyMMdd)$(Rev:.r)'
pool:
vmImage: 'windows-latest'
parameters:
- name: migrationMode
displayName: 'Select migration mode'
type: string
default: 'Both'
values:
- ExportOnly
- ImportOnly
- Both
- name: extractExportedJson
displayName: 'Extract exported artifacts zips'
type: boolean
default: true
- name: importExpectZippedInput
displayName: 'Import PreZipped files (Enable only if zips are already created in the artifacts folder)'
type: boolean
default: false
- name: artifactType
displayName: 'Select the type of artifact to be migrated'
type: stringList
values:
- Flows
- Forms
- IntegrationOrchestrations
default:
- Flows
- Forms
- IntegrationOrchestrations
- name: artifactSpecMode
displayName: 'Provide artifact specifications for migration; either All artifacts or custom set (needs specification json)'
type: string
default: 'All'
values:
- All
- Custom
- name: sourceCustomerId
displayName: 'Provide Source Customer Id'
type: string
default: '6f7a2a43-bcac-49ca-96e2-2a1ccbac91cf'
- name: sourceProjectId
displayName: 'Provide Source Project Id'
type: string
default: '280ad80c-de2a-4430-b9e6-294d16421250'
- name: destCustomerId
displayName: 'Provide Destination Customer Id'
type: string
default: '6f7a2a43-bcac-49ca-96e2-2a1ccbac91cf'
- name: destProjectId
displayName: 'Provide Destination Project Id'
type: string
default: '280ad80c-de2a-4430-b9e6-294d16421250'
- name: destEnvironmentId
displayName: 'Provide Destination Environment Id'
type: string
default: 'eac720f7-308e-4129-a559-c2a58aa5ae2f'
- name: artifactSpecs
displayName: 'Provide artifact spec JSON (with id/version/export/import per entry) when Custom is selected'
type: string
default: './ArtifactSpecsFEATDEV.json'
- name: verboseLogging
displayName: 'Verbose Logging'
type: boolean
default: false
- name: sourceEnvironment
displayName: 'Source environment variable prefix'
type: string
default: 'FEATDEV'
- name: destinationEnvironment
displayName: 'Destination environment variable prefix'
type: string
default: 'FEATDEV'
variables:
- group: 'migrationenvs'
- name: zipOutputFile
value: '$(Build.ArtifactStagingDirectory)\package.zip'
- name: sourceBaseUrl
value: $[ variables['${{ parameters.sourceEnvironment }}_BASE_URL'] ]
- name: sourceClientId
value: $[ variables['${{ parameters.sourceEnvironment }}_CLIENT_ID'] ]
- name: sourceClientSecret
value: $[ variables['${{ parameters.sourceEnvironment }}_CLIENT_SECRET'] ]
- name: sourceTenantId
value: $[ variables['${{ parameters.sourceEnvironment }}_TENANT_ID'] ]
- name: destinationBaseUrl
value: $[ variables['${{ parameters.destinationEnvironment }}_BASE_URL'] ]
- name: destinationClientId
value: $[ variables['${{ parameters.destinationEnvironment }}_CLIENT_ID'] ]
- name: destinationClientSecret
value: $[ variables['${{ parameters.destinationEnvironment }}_CLIENT_SECRET'] ]
- name: destinationTenantId
value: $[ variables['${{ parameters.destinationEnvironment }}_TENANT_ID'] ]
- name: EXPORT_ENDPOINT
value: '/api/customer/${{parameters.sourceCustomerId}}/tenant/${{parameters.sourceProjectId}}/process/{{processid}}/export/{{version}}'
- name: IMPORT_ENDPOINT
value: '/api/customer/${{parameters.destCustomerId}}/tenant/${{parameters.destProjectId}}/environment/${{parameters.destEnvironmentId}}/process/import'
- name: extractExportedJson
value: ${{ parameters.extractExportedJson }}
stages:
# --------------------------------------------
# Stage 1: Validate the parameters
# --------------------------------------------
- stage: ValidateParameters
displayName: 'Validate Parameters'
jobs:
- job: Prep
displayName: 'Validate and prepare variables'
pool:
vmImage: 'windows-latest'
steps:
- task: PowerShell@2
name: ParseParams
displayName: 'Parse and validate parameters'
inputs:
targetType: 'filePath'
pwsh: true
filePath: '$(Build.SourcesDirectory)/validateMigrationParams.ps1'
${{ if eq(parameters.artifactSpecMode, 'Custom') }}:
arguments: >
-artifactSpecsFile '$(Build.SourcesDirectory)/${{ parameters.artifactSpecs }}'
-sourceCustomerId '${{ parameters.sourceCustomerId }}'
-sourceProjectId '${{ parameters.sourceProjectId }}'
-destCustomerId '${{ parameters.destCustomerId }}'
-destProjectId '${{ parameters.destProjectId }}'
-destEnvironmentId '${{ parameters.destEnvironmentId }}'
${{ if ne(parameters.artifactSpecMode, 'Custom') }}:
arguments: >
-sourceCustomerId '${{ parameters.sourceCustomerId }}'
-sourceProjectId '${{ parameters.sourceProjectId }}'
-destCustomerId '${{ parameters.destCustomerId }}'
-destProjectId '${{ parameters.destProjectId }}'
-destEnvironmentId '${{ parameters.destEnvironmentId }}'
# --------------------------------------------
# Stage 2: Export Artifacts
# --------------------------------------------
- stage: ExportArtifacs
displayName: 'Export Artifacts'
dependsOn: ValidateParameters
condition: and(succeeded(), or(eq('${{ parameters.migrationMode }}', 'ExportOnly'), eq('${{ parameters.migrationMode }}', 'Both')))
jobs:
- job: Export
displayName: 'Export artifacts from source'
pool:
vmImage: 'windows-latest'
steps:
- checkout: self
persistCredentials: true
- task: PowerShell@2
displayName: 'Export artifacts and publish as build artifact'
inputs:
targetType: 'filePath'
pwsh: true
filePath: '$(Build.SourcesDirectory)/exportArtifacts.ps1'
${{ if eq(parameters.artifactSpecMode, 'Custom') }}:
arguments: >
-baseUrl '$(sourceBaseUrl)'
-clientId '$(sourceClientId)'
-clientSecret '$(sourceClientSecret)'
-tenantId '$(sourceTenantId)'
-sourceCustomerId '${{ parameters.sourceCustomerId }}'
-sourceProjectId '${{ parameters.sourceProjectId }}'
-artifactType '${{ join(parameters.artifactType, ',') }}'
-artifactSpecsFile '$(Build.SourcesDirectory)/${{ parameters.artifactSpecs }}'
-outputFolder '$(Build.ArtifactStagingDirectory)\exports'
-verboseLogging '${{ parameters.verboseLogging }}'
-extractExportedJson '${{ parameters.extractExportedJson }}'
-repoArtifactsPath '$(Build.SourcesDirectory)\artifacts'
${{ if ne(parameters.artifactSpecMode, 'Custom') }}:
arguments: >
-baseUrl '$(sourceBaseUrl)'
-clientId '$(sourceClientId)'
-clientSecret '$(sourceClientSecret)'
-tenantId '$(sourceTenantId)'
-sourceCustomerId '${{ parameters.sourceCustomerId }}'
-sourceProjectId '${{ parameters.sourceProjectId }}'
-artifactType '${{ join(parameters.artifactType, ',') }}'
-outputFolder '$(Build.ArtifactStagingDirectory)\exports'
-verboseLogging '${{ parameters.verboseLogging }}'
-extractExportedJson '${{ parameters.extractExportedJson }}'
-repoArtifactsPath '$(Build.SourcesDirectory)\artifacts'
- task: PowerShell@2
displayName: 'Commit extracted artifact json files to repo'
condition: eq(variables['extractExportedJson'], 'true')
inputs:
targetType: 'inline'
pwsh: true
script: |
git config user.email "ado-pipeline@local"
git config user.name "ADO Pipeline"
git add --all -- artifacts
$changes = git status --porcelain -- artifacts
if ([string]::IsNullOrWhiteSpace($changes)) {
Write-Host 'No changes detected under artifacts. Skipping commit.'
exit 0
}
git commit -m "chore: store exported artifact jsons [skip ci]"
git push origin "HEAD:$(Build.SourceBranchName)"
Write-Host 'Committed and pushed extracted artifact json files.'
- task: PublishBuildArtifacts@1
displayName: 'Publish exported artifacts'
condition: eq(variables['extractExportedJson'], 'false')
inputs:
PathtoPublish: '$(Build.ArtifactStagingDirectory)\exports'
ArtifactName: 'exported-artifacts'
publishLocation: 'Container'
# --------------------------------------------
# Stage 3: Import Artifacts
# --------------------------------------------
- stage: ImportArtifacts
displayName: 'Import Artifacts'
dependsOn:
- ValidateParameters
- ExportArtifacs
condition: and(succeeded('ValidateParameters'), or(eq('${{ parameters.migrationMode }}', 'ImportOnly'), and(eq('${{ parameters.migrationMode }}', 'Both'), succeeded('ExportArtifacs'))))
jobs:
- job: Import
displayName: 'Download exported artifacts and import into destination'
pool:
vmImage: 'windows-latest'
steps:
- checkout: self
persistCredentials: true
- task: PowerShell@2
displayName: 'Refresh branch for latest committed artifact jsons'
condition: eq('${{ parameters.migrationMode }}', 'Both')
inputs:
targetType: 'inline'
pwsh: true
script: |
git fetch origin "$(Build.SourceBranchName)"
git checkout "$(Build.SourceBranchName)"
git pull --ff-only origin "$(Build.SourceBranchName)"
- task: PowerShell@2
displayName: 'Import artifacts into destination'
inputs:
targetType: 'filePath'
pwsh: true
filePath: '$(Build.SourcesDirectory)/importArtifacts.ps1'
${{ if eq(parameters.artifactSpecMode, 'Custom') }}:
arguments: >
-baseUrl '$(destinationBaseUrl)'
-clientId '$(destinationClientId)'
-clientSecret '$(destinationClientSecret)'
-tenantId '$(destinationTenantId)'
-destCustomerId '${{ parameters.destCustomerId }}'
-destProjectId '${{ parameters.destProjectId }}'
-destEnvironmentId '${{ parameters.destEnvironmentId }}'
-artifactType '${{ join(parameters.artifactType, ',') }}'
-verboseLogging '${{ parameters.verboseLogging }}'
-expectZippedInput '${{ parameters.importExpectZippedInput }}'
-artifactSpecsFile '$(Build.SourcesDirectory)/${{ parameters.artifactSpecs }}'
-artifactsPath '$(Build.SourcesDirectory)\artifacts'
${{ else }}:
arguments: >
-baseUrl '$(destinationBaseUrl)'
-clientId '$(destinationClientId)'
-clientSecret '$(destinationClientSecret)'
-tenantId '$(destinationTenantId)'
-destCustomerId '${{ parameters.destCustomerId }}'
-destProjectId '${{ parameters.destProjectId }}'
-destEnvironmentId '${{ parameters.destEnvironmentId }}'
-artifactType '${{ join(parameters.artifactType, ',') }}'
-verboseLogging '${{ parameters.verboseLogging }}'
-expectZippedInput '${{ parameters.importExpectZippedInput }}'
-artifactsPath '$(Build.SourcesDirectory)\artifacts'
Parameter validation
Validation of input prameters is done by the dedicated validateMigrationParams.ps1 script.
Validation Script (Current)
param (
[Parameter(Mandatory = $false)][string]
$artifactSpecsFile, # Path to JSON file grouping artifacts by type (forms, flows, integrationOrchestrations) with id/version/import/export entries
[Parameter(Mandatory = $true)][string]
$sourceCustomerId,
[Parameter(Mandatory = $true)][string]
$sourceProjectId,
[Parameter(Mandatory = $true)][string]
$destCustomerId,
[Parameter(Mandatory = $true)][string]
$destProjectId,
[Parameter(Mandatory = $true)][string]
$destEnvironmentId
)
function Test-IsGuid {
param([string]$Value)
if ([string]::IsNullOrWhiteSpace($Value)) { return $false }
$g = [guid]::Empty
return [System.Guid]::TryParse($Value, [ref]$g)
}
function Get-ArtifactSpecEntries {
param(
[pscustomobject]$JsonObject,
[string]$SpecsFilePath
)
if (-not $JsonObject) {
return @()
}
$typePropertyMap = @{
Flows = 'flows'
Forms = 'forms'
IntegrationOrchestrations = 'integrationOrchestrations'
}
$entries = @()
foreach ($typeName in $typePropertyMap.Keys) {
$propertyName = $typePropertyMap[$typeName]
if (-not $JsonObject.PSObject.Properties.Match($propertyName)) {
continue
}
$items = @($JsonObject.$propertyName)
foreach ($item in $items) {
if (-not $item) {
continue
}
$artifactId = "$($item.id)".Trim()
if ([string]::IsNullOrWhiteSpace($artifactId)) {
throw "artifactSpecsFile '$SpecsFilePath' contains an entry under '$propertyName' without an id."
}
$versionValue = $null
if ($item.PSObject.Properties.Match('version')) {
$versionValue = "$($item.version)".Trim()
if ([string]::IsNullOrWhiteSpace($versionValue)) {
$versionValue = $null
}
}
$exportValue = $true
if ($item.PSObject.Properties.Match('export')) {
if ($item.export -isnot [bool]) {
throw "artifactSpecsFile '$SpecsFilePath' entry for id '$artifactId' has non-boolean 'export' value '$($item.export)'. Expected true or false."
}
$exportValue = [bool]$item.export
}
$importValue = $true
if ($item.PSObject.Properties.Match('import')) {
if ($item.import -isnot [bool]) {
throw "artifactSpecsFile '$SpecsFilePath' entry for id '$artifactId' has non-boolean 'import' value '$($item.import)'. Expected true or false."
}
$importValue = [bool]$item.import
}
$entries += [pscustomobject]@{
Type = $typeName
Id = $artifactId
Version = $versionValue
Export = $exportValue
Import = $importValue
}
}
}
return @($entries)
}
# Validate single GUID parameters
$guidParams = @{
sourceCustomerId = $sourceCustomerId
sourceProjectId = $sourceProjectId
destCustomerId = $destCustomerId
destProjectId = $destProjectId
destEnvironmentId = $destEnvironmentId
}
foreach ($key in $guidParams.Keys) {
$val = $guidParams[$key]
if (-not (Test-IsGuid $val)) {
throw "Parameter '$key' must be a valid GUID. Value: '$val'"
}
else {
Write-Host "Parameter '$key' is a valid GUID."
}
}
if ( -not $artifactSpecsFile) {
Write-Host "No artifactSpecsFile provided. Skipping artifact spec validation."
return
}
if ([string]::IsNullOrWhiteSpace($artifactSpecsFile)) {
throw "artifactSpecsFile must be provided"
}
if (-not (Test-Path -LiteralPath $artifactSpecsFile)) {
throw "artifactSpecsFile does not exist: '$artifactSpecsFile'"
}
$versionPattern = '^[0-9]+\.[0-9]+\.[0-9]+\.[0-9]+$'
try {
$rawArtifactSpecs = Get-Content -LiteralPath $artifactSpecsFile -Raw -ErrorAction Stop
}
catch {
throw "Unable to read artifactSpecsFile '$artifactSpecsFile'. Error: $($_.Exception.Message)"
}
if ([string]::IsNullOrWhiteSpace($rawArtifactSpecs)) {
throw "artifactSpecsFile '$artifactSpecsFile' is empty"
}
try {
$artifactSpecsRoot = $rawArtifactSpecs | ConvertFrom-Json -ErrorAction Stop
}
catch {
throw "artifactSpecsFile '$artifactSpecsFile' is not valid JSON. Error: $($_.Exception.Message)"
}
if ($artifactSpecsRoot -is [System.Array]) {
throw "artifactSpecsFile '$artifactSpecsFile' must be a JSON object describing artifacts, not a JSON array."
}
if ($artifactSpecsRoot -isnot [pscustomobject] -and $artifactSpecsRoot -isnot [hashtable]) {
throw "artifactSpecsFile '$artifactSpecsFile' must contain a JSON object describing artifacts."
}
$artifactSpecsObject = [pscustomobject]$artifactSpecsRoot
$artifactEntries = Get-ArtifactSpecEntries -JsonObject $artifactSpecsObject -SpecsFilePath $artifactSpecsFile
if (-not $artifactEntries -or $artifactEntries.Count -eq 0) {
throw "artifactSpecsFile '$artifactSpecsFile' contains no artifact specs."
}
foreach ($entry in $artifactEntries) {
if (-not (Test-IsGuid $entry.Id)) {
throw "Artifact id '$($entry.Id)' in artifactSpecsFile '$artifactSpecsFile' is not a valid GUID."
}
if ($null -ne $entry.Version) {
$isLatest = $entry.Version.Equals('latest', [System.StringComparison]::OrdinalIgnoreCase)
if (-not $isLatest -and -not ($entry.Version -match $versionPattern)) {
throw "Artifact version '$($entry.Version)' for artifact id '$($entry.Id)' is not in the expected format 'x.x.x.x' or 'latest'."
}
}
}
$specCount = $artifactEntries.Count
$entryLabel = if ($specCount -eq 1) { 'entry' } else { 'entries' }
Write-Host "Validated $specCount artifact spec $entryLabel from '$artifactSpecsFile'."
Export
Export is handled by exportArtifacts.ps1.
Export Behavior
- If
artifactSpecsFileis provided:- Script skips discovery API calls.
- Script directly attempts export using supplied
artifactId:versionpairs. - Export endpoint errors are treated as source of truth.
- If
artifactSpecsFileis not provided:- Script fetches artifact IDs from type list endpoints.
- Script fetches published versions for each artifact ID using versions endpoint.
- Script exports latest published version per artifact.
Export Endpoints
- List IDs by type:
GET /api/customer/{customerId}/tenant/{tenantId}/processesGET /api/customer/{customerId}/tenant/{tenantId}/demformsGET /api/customer/{customerId}/tenant/{tenantId}/integrationorchestrations
- Published versions per artifact:
GET /api/customer/{customerId}/tenant/{tenantId}/environment/{environmentId}/process/{processId}/versionsGET /api/customer/{customerId}/tenant/{tenantId}/demform/{demformId}/versionsGET /api/customer/{customerId}/tenant/{tenantId}/integrationorchestration/{integrationorchestrationId}/versions
- Export:
POST /api/customer/{customerId}/tenant/{tenantId}/process/{processId}/export/{version}POST /api/customer/{customerId}/tenant/{tenantId}/demform/{demformId}/export/{version}POST /api/customer/{customerId}/tenant/{tenantId}/integrationorchestration/{integrationorchestrationId}/export/{version}
Export Script (Current)
param (
[Parameter(Mandatory = $true)]
[string]$baseUrl,
[Parameter(Mandatory = $true)]
[string]$clientId,
[Parameter(Mandatory = $true)]
[string]$clientSecret,
[Parameter(Mandatory = $true)]
[string]$tenantId,
[Parameter(Mandatory = $true)]
[string]$sourceCustomerId,
[Parameter(Mandatory = $true)]
[string]$sourceProjectId,
[Parameter(Mandatory = $false)]
[Alias("artifactType")]
[object]$artifactTypes,
[Parameter(Mandatory = $false)]
[string]$artifactSpecsFile,
[Parameter(Mandatory = $false)]
[string]$outputFolder,
[Parameter(Mandatory = $false)]
[object]$verboseLogging = $false,
[int]$maxRetries = 3,
[int]$retryDelaySeconds = 5
)
$ErrorActionPreference = "Stop"
function Test-IsTruthy {
param([object]$Value)
if ($Value -is [bool]) { return $Value }
$normalized = "$Value".Trim().ToLowerInvariant()
return $normalized -in @('true', '1', 'yes', 'y')
}
function Convert-HeaderMapToLogString {
param([object]$Headers)
if ($null -eq $Headers) { return "<none>" }
$pairs = @()
foreach ($key in $Headers.Keys) {
$value = "$($Headers[$key])"
if ($key -ieq 'Authorization') {
if ($value -match '^Bearer\s+(.+)$') {
$token = $Matches[1]
$suffixLength = [Math]::Min(8, $token.Length)
$suffix = $token.Substring($token.Length - $suffixLength, $suffixLength)
$value = "Bearer ***$suffix"
}
else {
$value = '***REDACTED***'
}
}
$pairs += "${key} - $value"
}
if (-not $pairs -or $pairs.Count -eq 0) { return "<none>" }
return ($pairs -join '; ')
}
function Write-VerboseHttpRequest {
param(
[string]$Method,
[string]$Url,
[object]$Headers,
[object]$Body
)
Write-Verbose "HTTP Request => $Method $Url"
Write-Verbose "HTTP Request Headers => $(Convert-HeaderMapToLogString -Headers $Headers)"
if ($null -ne $Body) {
$bodyText = $Body
if ($Body -isnot [string]) {
try { $bodyText = ($Body | ConvertTo-Json -Depth 8 -Compress) } catch { $bodyText = "$Body" }
}
Write-Verbose "HTTP Request Body => $bodyText"
}
else {
Write-Verbose "HTTP Request Body => <none>"
}
}
function Write-VerboseHttpResponse {
param(
[string]$Method,
[string]$Url,
[object]$StatusCode,
[object]$Headers,
[string]$Body
)
Write-Verbose "HTTP Response <= $Method $Url"
if ($null -ne $StatusCode) {
Write-Verbose "HTTP Response Status => $StatusCode"
}
Write-Verbose "HTTP Response Headers => $(Convert-HeaderMapToLogString -Headers $Headers)"
if ([string]::IsNullOrWhiteSpace($Body)) {
Write-Verbose "HTTP Response Body => <empty>"
}
else {
Write-Verbose "HTTP Response Body => $Body"
}
}
function Get-JwtPayloadClaims {
param([string]$Token)
if ([string]::IsNullOrWhiteSpace($Token)) { return $null }
$parts = $Token.Split('.')
if ($parts.Count -lt 2) { return $null }
$payload = $parts[1].Replace('-', '+').Replace('_', '/')
switch ($payload.Length % 4) {
2 { $payload += '==' }
3 { $payload += '=' }
0 { }
default { return $null }
}
try {
$bytes = [System.Convert]::FromBase64String($payload)
$json = [System.Text.Encoding]::UTF8.GetString($bytes)
return ($json | ConvertFrom-Json)
}
catch {
return $null
}
}
if (Test-IsTruthy $verboseLogging) {
$VerbosePreference = "Continue"
}
else {
$VerbosePreference = "SilentlyContinue"
}
$typeMap = @{
Flows = 'process'
Forms = 'demform'
IntegrationOrchestrations = 'integrationorchestration'
}
$listEndpointMap = @{
Flows = 'processes'
Forms = 'demforms'
IntegrationOrchestrations = 'integrationorchestrations'
}
# Function to Get authentication headers using client credentials
function Get-AuthHeaders {
param(
[string]$ClientId = $clientId,
[string]$ClientSecret = $clientSecret,
[string]$TenantId = $tenantId
)
$tokenUrl = "https://login.microsoftonline.com/$TenantId/oauth2/v2.0/token"
Write-Host "Attempting to acquire access token from Azure AD for $tokenUrl"
$tokenRequestBody = @{
grant_type = "client_credentials"
client_id = $ClientId
client_secret = $ClientSecret
scope = "$ClientId/.default"
}
$tokenRequestLogBody = @{
grant_type = "client_credentials"
client_id = $ClientId
client_secret = "***REDACTED***"
scope = "$ClientId/.default"
}
try{
Write-VerboseHttpRequest -Method "POST" -Url $tokenUrl -Headers @{ "Content-Type" = "application/x-www-form-urlencoded" } -Body $tokenRequestLogBody
$tokenResponse = Invoke-WebRequest -Method Post `
-Uri $tokenUrl `
-ContentType "application/x-www-form-urlencoded" `
-Body $tokenRequestBody `
-UseBasicParsing `
-ErrorAction Stop
Write-VerboseHttpResponse -Method "POST" -Url $tokenUrl -StatusCode $tokenResponse.StatusCode -Headers $tokenResponse.Headers -Body $tokenResponse.Content
$token = ($tokenResponse.Content | ConvertFrom-Json).access_token
if ($token) {
Write-Host "Successfully acquired access token from Azure AD."
$claims = Get-JwtPayloadClaims -Token $token
if ($null -ne $claims) {
Write-Verbose "Access token claims => aud=$($claims.aud); tid=$($claims.tid); appid=$($claims.appid); azp=$($claims.azp); iss=$($claims.iss); exp=$($claims.exp)"
}
$authHeaders = @{ Authorization = "Bearer $token" }
Write-Verbose "Auth headers prepared => $(Convert-HeaderMapToLogString -Headers $authHeaders)"
return $authHeaders
}
}
catch {
Write-Warning "Failed to acquire access token from Azure AD: $($_.Exception.Message)"
return $null
}
throw "No authentication mechanism available. Ensure SYSTEM_ACCESSTOKEN is passed to the task env, or pass -pat."
}
function Write-VerboseExceptionDetails {
param(
[System.Management.Automation.ErrorRecord]$ErrorRecord,
[string]$Context
)
if (-not $ErrorRecord) { return }
Write-Verbose "$Context Exception: $($ErrorRecord.Exception.Message)"
$response = $null
if ($ErrorRecord.Exception.PSObject.Properties['Response']) {
$response = $ErrorRecord.Exception.Response
}
elseif ($ErrorRecord.Exception.InnerException -and $ErrorRecord.Exception.InnerException.PSObject.Properties['Response']) {
$response = $ErrorRecord.Exception.InnerException.Response
}
if ($null -eq $response) { return }
try {
if ($response.PSObject.Properties['StatusCode']) {
Write-Verbose "$Context StatusCode: $([int]$response.StatusCode)"
}
if ($response.PSObject.Properties['Headers']) {
Write-Verbose "$Context Headers: $($response.Headers | Out-String)"
}
$body = $null
$isHttpResponseMessage = $false
if ($null -ne $response -and $response.PSObject -and $response.PSObject.TypeNames) {
$isHttpResponseMessage = $response.PSObject.TypeNames -contains 'System.Net.Http.HttpResponseMessage'
}
if ($isHttpResponseMessage) {
if ($null -ne $response.Content) {
$body = $response.Content.ReadAsStringAsync().GetAwaiter().GetResult()
}
}
elseif ($response.PSObject.Methods.Name -contains 'GetResponseStream') {
$stream = $response.GetResponseStream()
if ($null -ne $stream) {
$reader = New-Object System.IO.StreamReader($stream)
$body = $reader.ReadToEnd()
$reader.Dispose()
$stream.Dispose()
}
}
if (-not [string]::IsNullOrWhiteSpace($body)) {
Write-Verbose "$Context Body: $body"
}
}
catch {
Write-Verbose "$Context Response body could not be read: $($_.Exception.Message)"
}
}
function Get-HttpErrorDetails {
param(
[System.Management.Automation.ErrorRecord]$ErrorRecord
)
$details = @{
StatusCode = $null
Headers = $null
Body = $null
}
if (-not $ErrorRecord) { return $details }
$response = $null
if ($ErrorRecord.Exception.PSObject.Properties['Response']) {
$response = $ErrorRecord.Exception.Response
}
elseif ($ErrorRecord.Exception.InnerException -and $ErrorRecord.Exception.InnerException.PSObject.Properties['Response']) {
$response = $ErrorRecord.Exception.InnerException.Response
}
if ($null -eq $response) { return $details }
try {
if ($response.PSObject.Properties['StatusCode']) {
$details.StatusCode = [int]$response.StatusCode
}
if ($response.PSObject.Properties['Headers']) {
$details.Headers = $response.Headers
}
$isHttpResponseMessage = $false
if ($null -ne $response -and $response.PSObject -and $response.PSObject.TypeNames) {
$isHttpResponseMessage = $response.PSObject.TypeNames -contains 'System.Net.Http.HttpResponseMessage'
}
if ($isHttpResponseMessage) {
if ($null -ne $response.Content) {
$details.Body = $response.Content.ReadAsStringAsync().GetAwaiter().GetResult()
}
}
elseif ($response.PSObject.Methods.Name -contains 'GetResponseStream') {
$stream = $response.GetResponseStream()
if ($null -ne $stream) {
$reader = New-Object System.IO.StreamReader($stream)
$details.Body = $reader.ReadToEnd()
$reader.Dispose()
$stream.Dispose()
}
}
}
catch {
# Keep details best-effort only.
}
return $details
}
function Write-ConsoleHttpErrorDetails {
param(
[System.Management.Automation.ErrorRecord]$ErrorRecord,
[string]$Context
)
$httpError = Get-HttpErrorDetails -ErrorRecord $ErrorRecord
if ($null -ne $httpError.StatusCode) {
Write-Warning "$Context HTTP StatusCode: $($httpError.StatusCode)"
}
if ($null -ne $httpError.Headers) {
if ($httpError.Headers['WWW-Authenticate']) {
Write-Warning "$Context WWW-Authenticate: $($httpError.Headers['WWW-Authenticate'])"
}
if ($httpError.Headers['x-ms-request-id']) {
Write-Warning "$Context x-ms-request-id: $($httpError.Headers['x-ms-request-id'])"
}
if ($httpError.Headers['x-correlation-id']) {
Write-Warning "$Context x-correlation-id: $($httpError.Headers['x-correlation-id'])"
}
}
if (-not [string]::IsNullOrWhiteSpace($httpError.Body)) {
Write-Warning "$Context Response body: $($httpError.Body)"
}
if ($VerbosePreference -eq 'Continue') {
Write-Verbose "$Context Full response headers: $(Convert-HeaderMapToLogString -Headers $httpError.Headers)"
if ([string]::IsNullOrWhiteSpace($httpError.Body)) {
Write-Verbose "$Context Full response body: <empty>"
}
else {
Write-Verbose "$Context Full response body: $($httpError.Body)"
}
}
}
# Function to normalize artifact types input into an array of valid type names
function NormalizeArtifactTypes {
param([object]$RawArtifactTypes)
if ($null -eq $RawArtifactTypes) {
return @()
}
$values = @()
if ($RawArtifactTypes -is [System.Array]) {
$values = $RawArtifactTypes
}
else {
$raw = "$RawArtifactTypes".Trim()
if ([string]::IsNullOrWhiteSpace($raw)) {
return @()
}
if ($raw.Contains(',') -or $raw.Contains(';')) {
$values = $raw -split '[,;]'
}
elseif ($raw.Contains(' ')) {
$values = $raw -split '\s+'
}
else {
$values = @($raw)
}
}
$normalized = @()
foreach ($v in $values) {
$name = "$v".Trim()
if ([string]::IsNullOrWhiteSpace($name)) { continue }
$normalized += $name
}
return @($normalized | Select-Object -Unique)
}
# Function to get artifact Ids for a given artifact type from a customer/project
function Get-ArtifactIdsByType {
param(
[string]$BaseUrl,
[string]$CustomerId,
[string]$ProjectId,
[string]$ArtifactType,
[hashtable]$Headers
)
$endpointSuffix = $listEndpointMap[$ArtifactType]
if (-not $endpointSuffix) {
throw "Unsupported artifact type '$ArtifactType'."
}
$url = "$BaseUrl/api/customer/$CustomerId/tenant/$ProjectId/$endpointSuffix"
Write-Host "Fetching artifact ids for $ArtifactType from $url"
try {
Write-VerboseHttpRequest -Method "GET" -Url $url -Headers $Headers -Body $null
$response = Invoke-WebRequest -Method Get -Uri $url -Headers $Headers -UseBasicParsing -ErrorAction Stop
Write-VerboseHttpResponse -Method "GET" -Url $url -StatusCode $response.StatusCode -Headers $response.Headers -Body $response.Content
$responseBody = $response.Content | ConvertFrom-Json
}
catch {
Write-Warning "Failed to fetch artifact ids for type '$ArtifactType': $($_.Exception.Message)"
Write-ConsoleHttpErrorDetails -ErrorRecord $_ -Context "Artifact ID request for type '$ArtifactType'"
Write-VerboseExceptionDetails -ErrorRecord $_ -Context "Artifact ID request for type '$ArtifactType'"
throw
}
$items = @($responseBody.content)
Write-Verbose "Fetched $($items.Count) raw items for type '$ArtifactType'."
$artifactIds = @{}
foreach ($item in $items) {
if (-not $item.id) { continue }
$artifactIds["$($item.id)"] = $true
}
return @($artifactIds.Keys)
}
# Function to get published versions for a given artifact
function Get-PublishedVersionsForArtifact {
param(
[string]$BaseUrl,
[string]$CustomerId,
[string]$ProjectId,
[string]$TypeSegment,
[string]$ArtifactId,
[hashtable]$Headers,
[switch]$IncludeUnitTestProcesses
)
$url = "$BaseUrl/api/customer/$CustomerId/tenant/$ProjectId/$TypeSegment/$ArtifactId/versions"
Write-Host "Fetching published versions for artifact '$ArtifactId' from $url"
try {
Write-VerboseHttpRequest -Method "GET" -Url $url -Headers $Headers -Body $null
$response = Invoke-WebRequest -Method Get -Uri $url -Headers $Headers -UseBasicParsing -ErrorAction Stop
Write-VerboseHttpResponse -Method "GET" -Url $url -StatusCode $response.StatusCode -Headers $response.Headers -Body $response.Content
$responseBody = $response.Content | ConvertFrom-Json
}
catch {
Write-Warning "Failed to fetch published versions for artifact '$ArtifactId': $($_.Exception.Message)"
Write-ConsoleHttpErrorDetails -ErrorRecord $_ -Context "Published versions request for artifact '$ArtifactId'"
throw
}
$items = @($responseBody.content)
Write-Verbose "Fetched $($items.Count) published version records for artifact '$ArtifactId'."
$versions = @{}
foreach ($item in $items) {
if ($null -eq $item) { continue }
if ($TypeSegment -eq 'process' -and $null -ne $item.PSObject.Properties['isUnitTestProcess']) {
if ((-not $IncludeUnitTestProcesses) -and [bool]$item.isUnitTestProcess) { continue }
}
if ($null -ne $item.PSObject.Properties['version'] -and -not [string]::IsNullOrWhiteSpace("$($item.version)")) {
$versions["$($item.version)"] = $true
}
}
return @($versions.Keys)
}
function Get-PublishedVersionRecordsForArtifact {
param(
[string]$BaseUrl,
[string]$CustomerId,
[string]$ProjectId,
[string]$TypeSegment,
[string]$ArtifactId,
[hashtable]$Headers,
[switch]$IncludeUnitTestProcesses
)
$url = "$BaseUrl/api/customer/$CustomerId/tenant/$ProjectId/$TypeSegment/$ArtifactId/versions"
Write-Host "Fetching published version records for artifact '$ArtifactId' from $url"
try {
Write-VerboseHttpRequest -Method "GET" -Url $url -Headers $Headers -Body $null
$response = Invoke-WebRequest -Method Get -Uri $url -Headers $Headers -UseBasicParsing -ErrorAction Stop
Write-VerboseHttpResponse -Method "GET" -Url $url -StatusCode $response.StatusCode -Headers $response.Headers -Body $response.Content
$responseBody = $response.Content | ConvertFrom-Json
}
catch {
Write-Warning "Failed to fetch published version records for artifact '$ArtifactId': $($_.Exception.Message)"
Write-ConsoleHttpErrorDetails -ErrorRecord $_ -Context "Published version records request for artifact '$ArtifactId'"
throw
}
$items = @($responseBody.content)
$records = @()
foreach ($item in $items) {
if ($null -eq $item) { continue }
if ($TypeSegment -eq 'process' -and $null -ne $item.PSObject.Properties['isUnitTestProcess']) {
if ((-not $IncludeUnitTestProcesses) -and [bool]$item.isUnitTestProcess) { continue }
}
if ($null -eq $item.PSObject.Properties['version']) { continue }
$versionText = "$($item.version)".Trim()
if ([string]::IsNullOrWhiteSpace($versionText)) { continue }
$publishedDateText = $null
foreach ($candidateField in @('publishedDate', 'publishDate', 'publishedOn', 'modifiedDate', 'modifiedOn', 'createdDate', 'createdOn')) {
if ($null -ne $item.PSObject.Properties[$candidateField]) {
$publishedDateText = "$($item.$candidateField)"
if (-not [string]::IsNullOrWhiteSpace($publishedDateText)) { break }
}
}
$publishedDate = $null
if (-not [string]::IsNullOrWhiteSpace($publishedDateText)) {
try { $publishedDate = [datetime]$publishedDateText } catch { $publishedDate = $null }
}
$records += [pscustomobject]@{
Version = $versionText
PublishedDate = $publishedDate
IsUnitTest = ($null -ne $item.PSObject.Properties['isUnitTestProcess'] -and [bool]$item.isUnitTestProcess)
RawItem = $item
}
}
return @($records)
}
function Get-LatestPublishedRecord {
param([object[]]$Records)
if (-not $Records -or $Records.Count -eq 0) { return $null }
return @($Records | Sort-Object -Property @{ Expression = { if ($null -eq $_.PublishedDate) { [datetime]::MinValue } else { $_.PublishedDate } } }, @{ Expression = { $_.Version } } -Descending)[0]
}
# Function to get the latest version from a list of versions
function Get-LatestVersion {
param([string[]]$Versions)
if (-not $Versions -or $Versions.Count -eq 0) { return $null }
return @($Versions | Sort-Object { [version]$_ } -Descending)[0]
}
function Get-HighestVersionNumberFromRecords {
param([object[]]$Records)
if (-not $Records -or $Records.Count -eq 0) { return $null }
$sortable = @()
foreach ($record in $Records) {
if ($null -eq $record) { continue }
$versionText = "$($record.Version)".Trim()
if ([string]::IsNullOrWhiteSpace($versionText)) { continue }
try {
$sortable += [pscustomobject]@{
VersionText = $versionText
VersionObj = [version]$versionText
}
}
catch {
Write-Verbose "Version '$versionText' is not parseable as [version]; ignoring for numeric highest-version comparison."
}
}
if (-not $sortable -or $sortable.Count -eq 0) { return $null }
return @($sortable | Sort-Object -Property VersionObj -Descending)[0].VersionText
}
function Resolve-VersionForSpec {
param(
[string]$RequestedVersion,
[string]$ArtifactId,
[string]$TypeName,
[string]$TypeSegment,
[string]$BaseUrl,
[string]$CustomerId,
[string]$ProjectId,
[hashtable]$Headers
)
$requested = "$RequestedVersion".Trim()
$mustResolveLatest = [string]::IsNullOrWhiteSpace($requested) -or $requested.Equals('Latest', [System.StringComparison]::OrdinalIgnoreCase)
if (-not $mustResolveLatest) {
if ($TypeSegment -eq 'process') {
$recordsWithUnitTests = Get-PublishedVersionRecordsForArtifact `
-BaseUrl $BaseUrl `
-CustomerId $CustomerId `
-ProjectId $ProjectId `
-TypeSegment $TypeSegment `
-ArtifactId $ArtifactId `
-Headers $Headers `
-IncludeUnitTestProcesses
$matchingRecord = @($recordsWithUnitTests | Where-Object { $_.Version -eq $requested })
if ($matchingRecord.Count -gt 0 -and $matchingRecord[0].IsUnitTest) {
Write-Warning "Artifact '$ArtifactId' of type '$TypeName' requested version '$requested' is a unit-test flow version. Skipping export."
return $null
}
}
return $requested
}
$records = Get-PublishedVersionRecordsForArtifact `
-BaseUrl $BaseUrl `
-CustomerId $CustomerId `
-ProjectId $ProjectId `
-TypeSegment $TypeSegment `
-ArtifactId $ArtifactId `
-Headers $Headers
if (-not $records -or $records.Count -eq 0) {
throw "No published versions found for artifact '$ArtifactId' of type '$TypeName' while resolving Latest."
}
$latestByPublishedDate = Get-LatestPublishedRecord -Records $records
$resolvedVersion = $latestByPublishedDate.Version
$highestVersion = Get-HighestVersionNumberFromRecords -Records $records
if (-not [string]::IsNullOrWhiteSpace($highestVersion) -and $resolvedVersion -ne $highestVersion) {
$publishedDateText = if ($latestByPublishedDate.PublishedDate) { $latestByPublishedDate.PublishedDate.ToString('o') } else { '<unknown>' }
Write-Warning "Artifact '$ArtifactId' of type '$TypeName' resolved Latest by published date to version '$resolvedVersion' (publishedDate $publishedDateText), but highest numeric version is '$highestVersion'."
}
Write-Verbose "Resolved artifact '$ArtifactId' ($TypeName) version to '$resolvedVersion' using published date ordering."
return $resolvedVersion
}
# Function to read and validate artifact specs from a JSON file
function Read-ArtifactSpecs {
param([string]$SpecsFilePath)
if ([string]::IsNullOrWhiteSpace($SpecsFilePath)) {
return $null
}
$raw = Get-Content -LiteralPath $SpecsFilePath -Raw
$json = $raw | ConvertFrom-Json
$typePropertyMap = @{
Forms = 'forms'
Flows = 'flows'
IntegrationOrchestrations = 'integrationOrchestrations'
}
$specs = @()
foreach ($typeName in $typePropertyMap.Keys) {
$propertyName = $typePropertyMap[$typeName]
$entries = @($json.$propertyName)
foreach ($entry in $entries) {
if ($null -eq $entry) { continue }
$artifactId = "$($entry.id)".Trim()
if ([string]::IsNullOrWhiteSpace($artifactId)) { continue }
$versionValue = $null
if ($null -ne $entry.PSObject.Properties['version']) {
$versionValue = "$($entry.version)".Trim()
if ([string]::IsNullOrWhiteSpace($versionValue)) {
$versionValue = $null
}
}
$exportEnabled = $true
if ($null -ne $entry.PSObject.Properties['export']) {
$exportEnabled = Test-IsTruthy $entry.export
}
$importEnabled = $true
if ($null -ne $entry.PSObject.Properties['import']) {
$importEnabled = Test-IsTruthy $entry.import
}
$specs += [pscustomobject]@{
Type = $typeName
Id = $artifactId
Version = $versionValue
ExportEnabled = $exportEnabled
ImportEnabled = $importEnabled
}
}
}
return @($specs)
}
# Function to export Artifact by Id and Version, with retry logic
function Export-Artifact {
param(
[string]$BaseUrl,
[string]$ArtifactId,
[string]$ArtifactVersion,
[string]$CustomerId,
[string]$ProjectId,
[string]$TypeSegment,
[hashtable]$Headers,
[string]$OutputFile,
[int]$MaxRetries,
[int]$RetryDelaySeconds
)
$exportEndpoint = "$BaseUrl/api/customer/$CustomerId/tenant/$ProjectId/$TypeSegment/$ArtifactId/export/$ArtifactVersion"
Write-Host "Exporting artifact from: $exportEndpoint"
for ($attempt = 1; $attempt -le $MaxRetries; $attempt++) {
try {
Write-VerboseHttpRequest -Method "POST" -Url $exportEndpoint -Headers $Headers -Body $null
$response = Invoke-WebRequest -Uri $exportEndpoint -Method Post -Headers $Headers -OutFile $OutputFile -PassThru -UseBasicParsing
Write-VerboseHttpResponse -Method "POST" -Url $exportEndpoint -StatusCode $response.StatusCode -Headers $response.Headers -Body "<binary content written to file>"
Write-Host "Export saved to $OutputFile"
return $true
}
catch {
Write-Warning "Attempt $attempt failed for artifact '$ArtifactId' version '$ArtifactVersion': $($_.Exception.Message)"
Write-ConsoleHttpErrorDetails -ErrorRecord $_ -Context "Export attempt $attempt for artifact '$ArtifactId'"
Write-VerboseExceptionDetails -ErrorRecord $_ -Context "Export attempt $attempt for artifact '$ArtifactId'"
if (Test-Path -LiteralPath $OutputFile) {
Remove-Item -LiteralPath $OutputFile -Force -ErrorAction SilentlyContinue
}
if ($attempt -eq $MaxRetries) { return $false }
Start-Sleep -Seconds $RetryDelaySeconds
}
}
return $false
}
try {
$headers = Get-AuthHeaders
if ($null -eq $headers) {
throw "Authentication headers could not be acquired."
}
$selectedTypes = NormalizeArtifactTypes -RawArtifactTypes $artifactTypes
if (-not $selectedTypes -or $selectedTypes.Count -eq 0) {
$selectedTypes = @($typeMap.Keys)
}
Write-Verbose "Selected artifact types: $($selectedTypes -join ', ')"
if ([string]::IsNullOrWhiteSpace($outputFolder)) {
if ([string]::IsNullOrWhiteSpace($env:BUILD_ARTIFACTSTAGINGDIRECTORY)) {
$outputFolder = Join-Path (Get-Location).Path "artifacts"
}
else {
$outputFolder = Join-Path $env:BUILD_ARTIFACTSTAGINGDIRECTORY "imports"
}
}
New-Item -Path $outputFolder -ItemType Directory -Force | Out-Null
Write-Host "Using export folder: $outputFolder"
$successCount = 0
$failureCount = 0
$artifactSpecs = Read-ArtifactSpecs -SpecsFilePath $artifactSpecsFile
if ($null -ne $artifactSpecs) {
$filteredSpecs = @($artifactSpecs | Where-Object { ($selectedTypes -contains $_.Type) -and $_.ExportEnabled })
Write-Host "artifactSpecsFile supplied. Skipping ID/version discovery and exporting based on supplied IDs and versions."
Write-Verbose "Artifact specs count (after type + export=true filter): $($filteredSpecs.Count)"
if (-not $filteredSpecs -or $filteredSpecs.Count -eq 0) {
Write-Host "No artifacts in artifactSpecsFile match selected type(s) and export=true: $($selectedTypes -join ', ')."
exit 0
}
foreach ($spec in $filteredSpecs) {
$typeName = $spec.Type
$artifactId = $spec.Id
$typeSegment = $typeMap[$typeName]
$artifactVersion = Resolve-VersionForSpec `
-RequestedVersion $spec.Version `
-ArtifactId $artifactId `
-TypeName $typeName `
-TypeSegment $typeSegment `
-BaseUrl $baseUrl `
-CustomerId $sourceCustomerId `
-ProjectId $sourceProjectId `
-Headers $headers
if ([string]::IsNullOrWhiteSpace($artifactVersion)) {
continue
}
$outFile = Join-Path $outputFolder "$($typeName)_$($artifactId)_$($artifactVersion).zip"
$ok = Export-Artifact `
-BaseUrl $baseUrl `
-ArtifactId $artifactId `
-ArtifactVersion $artifactVersion `
-CustomerId $sourceCustomerId `
-ProjectId $sourceProjectId `
-TypeSegment $typeSegment `
-Headers $headers `
-OutputFile $outFile `
-MaxRetries $maxRetries `
-RetryDelaySeconds $retryDelaySeconds
if ($ok) {
$successCount++
Write-Verbose "Export succeeded for artifact '$artifactId' as type '$typeName' and version '$artifactVersion'."
}
else {
$failureCount++
Write-Warning "Failed to export artifact '$artifactId' as type '$typeName' and version '$artifactVersion'."
}
}
}
else {
Write-Host "No artifactSpecsFile provided. Fetching IDs, then published versions, then exporting latest published version."
$artifactsToExport = @()
foreach ($type in $selectedTypes) {
$typeSegment = $typeMap[$type]
$artifactIds = Get-ArtifactIdsByType `
-BaseUrl $baseUrl `
-CustomerId $sourceCustomerId `
-ProjectId $sourceProjectId `
-ArtifactType $type `
-Headers $headers
foreach ($artifactId in $artifactIds) {
$publishedRecords = Get-PublishedVersionRecordsForArtifact `
-BaseUrl $baseUrl `
-CustomerId $sourceCustomerId `
-ProjectId $sourceProjectId `
-TypeSegment $typeSegment `
-ArtifactId $artifactId `
-Headers $headers
$latestRecord = Get-LatestPublishedRecord -Records $publishedRecords
$latest = $null
if ($null -ne $latestRecord) {
$latest = $latestRecord.Version
}
if ($latest) {
$artifactsToExport += @{
Type = $type
Id = $artifactId
Version = $latest
}
}
else {
Write-Verbose "No published non-unit-test version found for artifact '$artifactId' of type '$type'."
}
}
}
if (-not $artifactsToExport -or $artifactsToExport.Count -eq 0) {
Write-Host "No published artifacts found to export."
exit 0
}
foreach ($artifact in $artifactsToExport) {
$typeName = $artifact.Type
$typeSegment = $typeMap[$typeName]
$artifactId = $artifact.Id
$artifactVersion = $artifact.Version
$outFile = Join-Path $outputFolder "$($typeName)_$($artifactId)_$($artifactVersion).zip"
$ok = Export-Artifact `
-BaseUrl $baseUrl `
-ArtifactId $artifactId `
-ArtifactVersion $artifactVersion `
-CustomerId $sourceCustomerId `
-ProjectId $sourceProjectId `
-TypeSegment $typeSegment `
-Headers $headers `
-OutputFile $outFile `
-MaxRetries $maxRetries `
-RetryDelaySeconds $retryDelaySeconds
if ($ok) { $successCount++ } else { $failureCount++ }
}
}
Write-Host "##vso[task.setvariable variable=exportFolder;isOutput=true]$outputFolder"
Write-Host "Export completed. Succeeded exports: $successCount, Failed exports: $failureCount"
if ($failureCount -gt 0) { exit 1 }
exit 0
}
catch {
Write-VerboseExceptionDetails -ErrorRecord $_ -Context "Export pipeline failure"
Write-Error "An error occurred during export: $($_.Exception.Message)"
exit 1
}
Import
Import is handled by importArtifacts.ps1.
Import Behavior
- Reads exported zip files recursively from artifacts path.
- Resolves artifact type from zip file prefix (
Flows_,Forms_,IntegrationOrchestrations_). - Uses type-specific import endpoints:
POST /api/customer/{customerId}/tenant/{tenantId}/environment/{environmentId}/process/importPOST /api/customer/{customerId}/tenant/{tenantId}/environment/{environmentId}/demform/importPOST /api/customer/{customerId}/tenant/{tenantId}/environment/{environmentId}/integrationorchestration/import
- Retries failed imports up to
maxRetries.
Import Script (Current)
param (
[Parameter(Mandatory = $true)]
[string]$baseUrl,
[Parameter(Mandatory = $true)]
[string]$clientId,
[Parameter(Mandatory = $true)]
[string]$clientSecret,
[Parameter(Mandatory = $true)]
[string]$tenantId,
[Parameter(Mandatory = $true)]
[string]$destCustomerId,
[Parameter(Mandatory = $true)]
[string]$destProjectId,
[Parameter(Mandatory = $true)]
[string]$destEnvironmentId,
[Parameter(Mandatory = $false)]
[Alias("artifactType")]
[object]$artifactTypes,
[Parameter(Mandatory = $false)]
[string]$artifactSpecsFile,
[Parameter(Mandatory = $false)]
[object]$verboseLogging = $false,
[string]$artifactsPath,
[int]$maxRetries = 3,
[int]$delaySeconds = 5
)
$ErrorActionPreference = "Stop"
function Test-IsTruthy {
param([object]$Value)
if ($Value -is [bool]) { return $Value }
$normalized = "$Value".Trim().ToLowerInvariant()
return $normalized -in @('true', '1', 'yes', 'y')
}
if (Test-IsTruthy $verboseLogging) {
$VerbosePreference = "Continue"
}
else {
$VerbosePreference = "SilentlyContinue"
}
$typeMap = @{
Flows = 'process'
Forms = 'demform'
IntegrationOrchestrations = 'integrationorchestration'
}
# Function to acquire authentication headers
function Get-AuthHeaders {
param(
[string]$ClientId = $clientId,
[string]$ClientSecret = $clientSecret,
[string]$TenantId = $tenantId
)
$tokenUrl = "https://login.microsoftonline.com/$TenantId/oauth2/v2.0/token"
Write-Host "Attempting to acquire access token from Azure AD for $tokenUrl"
try{
$token = (Invoke-RestMethod -Method Post `
-Uri $tokenUrl `
-ContentType "application/x-www-form-urlencoded" `
-Body @{
grant_type = "client_credentials"
client_id = $ClientId
client_secret = $ClientSecret
scope = "$ClientId/.default"
} `
-ErrorAction Stop).access_token
if ($token) {
Write-Host "Successfully acquired access token from Azure AD."
return @{ Authorization = "Bearer $token" }
}
}
catch {
Write-Warning "Failed to acquire access token from Azure AD: $($_.Exception.Message)"
return $null
}
throw "No authentication mechanism available."
}
function Write-VerboseExceptionDetails {
param(
[System.Management.Automation.ErrorRecord]$ErrorRecord,
[string]$Context
)
if (-not $ErrorRecord) { return }
Write-Verbose "$Context Exception: $($ErrorRecord.Exception.Message)"
$response = $null
if ($ErrorRecord.Exception.PSObject.Properties['Response']) {
$response = $ErrorRecord.Exception.Response
}
elseif ($ErrorRecord.Exception.InnerException -and $ErrorRecord.Exception.InnerException.PSObject.Properties['Response']) {
$response = $ErrorRecord.Exception.InnerException.Response
}
if ($null -eq $response) { return }
try {
if ($response.PSObject.Properties['StatusCode']) {
Write-Verbose "$Context StatusCode: $([int]$response.StatusCode)"
}
if ($response.PSObject.Properties['Headers']) {
Write-Verbose "$Context Headers: $($response.Headers | Out-String)"
}
$body = $null
if ($response -is [System.Net.Http.HttpResponseMessage]) {
if ($null -ne $response.Content) {
$body = $response.Content.ReadAsStringAsync().GetAwaiter().GetResult()
}
}
elseif ($response.PSObject.Methods.Name -contains 'GetResponseStream') {
$stream = $response.GetResponseStream()
if ($null -ne $stream) {
$reader = New-Object System.IO.StreamReader($stream)
$body = $reader.ReadToEnd()
$reader.Dispose()
$stream.Dispose()
}
}
if (-not [string]::IsNullOrWhiteSpace($body)) {
Write-Verbose "$Context Body: $body"
}
}
catch {
Write-Verbose "$Context Response body could not be read: $($_.Exception.Message)"
}
}
function Get-ErrorBodyFromException {
param([System.Management.Automation.ErrorRecord]$ErrorRecord)
if (-not $ErrorRecord) { return $null }
if (-not $ErrorRecord.Exception.Response) { return $null }
try {
$response = $ErrorRecord.Exception.Response
if ($response.PSObject.Methods.Name -contains 'GetResponseStream') {
$stream = $response.GetResponseStream()
if ($null -ne $stream) {
$reader = New-Object System.IO.StreamReader($stream)
try {
return $reader.ReadToEnd()
}
finally {
$reader.Dispose()
$stream.Dispose()
}
}
}
}
catch {
Write-Verbose "Unable to read error response body: $($_.Exception.Message)"
}
return $null
}
# Function to normalize artifact types input into an array of valid type names
function NormalizeArtifactTypes {
param([object]$RawArtifactTypes)
if ($null -eq $RawArtifactTypes) {
return @()
}
$values = @()
if ($RawArtifactTypes -is [System.Array]) {
$values = $RawArtifactTypes
}
else {
$raw = "$RawArtifactTypes".Trim()
if ([string]::IsNullOrWhiteSpace($raw)) {
return @()
}
if ($raw.Contains(',') -or $raw.Contains(';')) {
$values = $raw -split '[,;]'
}
elseif ($raw.Contains(' ')) {
$values = $raw -split '\s+'
}
else {
$values = @($raw)
}
}
$normalized = @()
foreach ($v in $values) {
$name = "$v".Trim()
if ([string]::IsNullOrWhiteSpace($name)) { continue }
$normalized += $name
}
return @($normalized | Select-Object -Unique)
}
function Read-ArtifactSpecs {
param([string]$SpecsFilePath)
if ([string]::IsNullOrWhiteSpace($SpecsFilePath)) {
return $null
}
$raw = Get-Content -LiteralPath $SpecsFilePath -Raw
$json = $raw | ConvertFrom-Json
$typePropertyMap = @{
Forms = 'forms'
Flows = 'flows'
IntegrationOrchestrations = 'integrationOrchestrations'
}
$specs = @()
foreach ($typeName in $typePropertyMap.Keys) {
$propertyName = $typePropertyMap[$typeName]
$entries = @($json.$propertyName)
foreach ($entry in $entries) {
if ($null -eq $entry) { continue }
$artifactId = "$($entry.id)".Trim()
if ([string]::IsNullOrWhiteSpace($artifactId)) { continue }
$versionValue = $null
if ($null -ne $entry.PSObject.Properties['version']) {
$versionValue = "$($entry.version)".Trim()
if ([string]::IsNullOrWhiteSpace($versionValue)) {
$versionValue = $null
}
}
$exportEnabled = $true
if ($null -ne $entry.PSObject.Properties['export']) {
$exportEnabled = Test-IsTruthy $entry.export
}
$importEnabled = $true
if ($null -ne $entry.PSObject.Properties['import']) {
$importEnabled = Test-IsTruthy $entry.import
}
$specs += [pscustomobject]@{
Type = $typeName
Id = $artifactId
Version = $versionValue
ExportEnabled = $exportEnabled
ImportEnabled = $importEnabled
}
}
}
return @($specs)
}
function Resolve-ZipsToImport {
param(
[System.IO.FileInfo[]]$ZipFiles,
[object[]]$ArtifactSpecs,
[string[]]$SelectedTypes
)
if ($null -eq $ArtifactSpecs) {
return @($ZipFiles)
}
$filteredSpecs = @($ArtifactSpecs | Where-Object { ($SelectedTypes -contains $_.Type) -and $_.ImportEnabled })
if (-not $filteredSpecs -or $filteredSpecs.Count -eq 0) {
Write-Verbose "artifactSpecsFile supplied, but no entries match selected artifact types with import=true."
return @()
}
$zipIndexByTypeAndId = @{}
foreach ($zip in $ZipFiles) {
if ($zip.Name -match '^(Flows|Forms|IntegrationOrchestrations)_(?<id>[^_]+)_(?<ver>.+)\.zip$') {
$typeName = $Matches[1]
$artifactId = $Matches['id']
$artifactVersion = $Matches['ver']
$key = "${typeName}|${artifactId}"
if (-not $zipIndexByTypeAndId.ContainsKey($key)) {
$zipIndexByTypeAndId[$key] = @()
}
$zipIndexByTypeAndId[$key] += [pscustomobject]@{
FileInfo = $zip
Version = $artifactVersion
}
}
}
$selectedZips = @()
foreach ($spec in $filteredSpecs) {
$key = "$($spec.Type)|$($spec.Id)"
$candidates = @()
if ($zipIndexByTypeAndId.ContainsKey($key)) {
$candidates = @($zipIndexByTypeAndId[$key])
}
if (-not $candidates -or $candidates.Count -eq 0) {
Write-Warning "No zip found for spec artifact '$($spec.Id)' of type '$($spec.Type)'."
continue
}
$requestedVersion = "$($spec.Version)".Trim()
if ([string]::IsNullOrWhiteSpace($requestedVersion) -or $requestedVersion.Equals('Latest', [System.StringComparison]::OrdinalIgnoreCase)) {
$selected = @($candidates | Sort-Object { [version]$_.Version } -Descending)[0]
$selectedZips += $selected.FileInfo
Write-Verbose "Resolved import zip for artifact '$($spec.Id)' ($($spec.Type)) to latest available file '$($selected.FileInfo.Name)'."
continue
}
$exact = @($candidates | Where-Object { $_.Version -eq $requestedVersion })
if ($exact.Count -gt 0) {
$selectedZips += $exact[0].FileInfo
}
else {
Write-Warning "No zip found for artifact '$($spec.Id)' of type '$($spec.Type)' with version '$requestedVersion'."
}
}
$dedup = @{}
foreach ($zip in $selectedZips) {
$dedup[$zip.FullName] = $zip
}
return @($dedup.Values)
}
# Function to validate zip file names and resolve artifact type based on naming convention and selected types
function Resolve-ArtifactTypeForZip {
param(
[string]$ZipName,
[string[]]$AllowedTypes
)
if ($ZipName -match '^(Flows|Forms|IntegrationOrchestrations)_.+\.zip$') {
$fileType = $Matches[1]
if ($AllowedTypes -notcontains $fileType) {
throw "Zip '$ZipName' resolved to type '$fileType' but it is not in selected artifact type(s): $($AllowedTypes -join ', ')."
}
return $fileType
}
if ($AllowedTypes.Count -eq 1) {
return $AllowedTypes[0]
}
throw "Cannot determine artifact type for zip '$ZipName'. Expected filename prefix 'Flows_', 'Forms_', or 'IntegrationOrchestrations_'."
}
# Function to import artifact into destination environment with retry logic
function Import-Artifact {
param(
[string]$BaseUrl,
[string]$ZipPath,
[string]$TypeSegment,
[string]$DestCustomerId,
[string]$DestProjectId,
[string]$DestEnvironmentId,
[hashtable]$Headers,
[int]$MaxRetries = 3,
[int]$DelaySeconds = 5
)
$uploadUrl = "$BaseUrl/api/customer/$DestCustomerId/tenant/$DestProjectId/environment/$DestEnvironmentId/$TypeSegment/import"
Write-Host "Uploading $ZipPath -> $uploadUrl"
for ($attempt = 1; $attempt -le $MaxRetries; $attempt++) {
try {
$file = Get-Item -LiteralPath $ZipPath
Invoke-WebRequest -Uri $uploadUrl -Method Post -Headers $Headers -InFile $file.FullName -ContentType "application/zip" -ErrorAction Stop | Out-Null
return @{ Success = $true }
}
catch {
$statusCode = $null
if ($_.Exception.Response -and $_.Exception.Response.PSObject.Properties['StatusCode']) {
$statusCode = [int]$_.Exception.Response.StatusCode
}
$body = Get-ErrorBodyFromException -ErrorRecord $_
$statusText = if ($null -ne $statusCode) { "HTTP $statusCode" } else { "no status code" }
Write-Warning "Attempt $attempt failed for '$ZipPath': $statusText - $($_.Exception.Message)"
if (-not [string]::IsNullOrWhiteSpace($body)) {
Write-Warning "Import error body: $body"
}
if ($_.Exception.Response -and $_.Exception.Response.Headers -and $_.Exception.Response.Headers['x-master-correlation-id']) {
Write-Warning "x-master-correlation-id: $($_.Exception.Response.Headers['x-master-correlation-id'])"
}
if ($attempt -eq $MaxRetries) {
return @{ Success = $false; Error = $_.Exception.Message; Body = $body; StatusCode = $statusCode }
}
Start-Sleep -Seconds $DelaySeconds
}
}
return @{ Success = $false; Error = "Unknown failure for '$ZipPath'." }
}
try {
$baseUrl = $baseUrl.TrimEnd('/')
Write-Host "Using destination base URL: $baseUrl"
$headers = Get-AuthHeaders
if ($null -eq $headers) {
throw "Authentication headers could not be acquired."
}
Write-Host "Authentication headers prepared."
$selectedTypes = NormalizeArtifactTypes -RawArtifactTypes $artifactTypes
if (-not $selectedTypes -or $selectedTypes.Count -eq 0) {
$selectedTypes = @("Flows", "Forms", "IntegrationOrchestrations")
}
Write-Verbose "Selected artifact types: $($selectedTypes -join ', ')"
if ([string]::IsNullOrWhiteSpace($artifactsPath)) {
if ([string]::IsNullOrWhiteSpace($env:BUILD_ARTIFACTSTAGINGDIRECTORY)) {
$artifactsPath = Join-Path (Get-Location).Path "artifacts"
}
else {
$artifactsPath = Join-Path $env:BUILD_ARTIFACTSTAGINGDIRECTORY "imports"
}
}
Write-Verbose "Using artifacts path: $artifactsPath"
if (-not (Test-Path -Path $artifactsPath)) {
Write-Error "Artifacts path '$artifactsPath' does not exist."
exit 1
}
$resolvedArtifactsPath = (Resolve-Path -Path $artifactsPath).Path
$zipFiles = Get-ChildItem -Path $resolvedArtifactsPath -Filter '*.zip' -File -Recurse
if (-not $zipFiles -or $zipFiles.Count -eq 0) {
Write-Host "No zip files found to import under: $resolvedArtifactsPath"
exit 0
}
Write-Verbose "Found $($zipFiles.Count) zip file(s) under '$resolvedArtifactsPath'."
$artifactSpecs = Read-ArtifactSpecs -SpecsFilePath $artifactSpecsFile
if ($null -ne $artifactSpecs) {
Write-Host "artifactSpecsFile supplied. Importing only artifacts listed in specs."
$zipFiles = Resolve-ZipsToImport -ZipFiles $zipFiles -ArtifactSpecs $artifactSpecs -SelectedTypes $selectedTypes
if (-not $zipFiles -or $zipFiles.Count -eq 0) {
Write-Host "No matching zip files found for artifactSpecsFile and selected type(s)."
exit 0
}
Write-Verbose "After applying artifactSpecsFile filter, $($zipFiles.Count) zip file(s) remain for import."
}
$imported = @()
$failed = @()
foreach ($zip in $zipFiles) {
$artifactType = Resolve-ArtifactTypeForZip -ZipName $zip.Name -AllowedTypes $selectedTypes
$typeSegment = $typeMap[$artifactType]
$result = Import-Artifact `
-BaseUrl $baseUrl `
-ZipPath $zip.FullName `
-TypeSegment $typeSegment `
-DestCustomerId $destCustomerId `
-DestProjectId $destProjectId `
-DestEnvironmentId $destEnvironmentId `
-Headers $headers `
-MaxRetries $maxRetries `
-DelaySeconds $delaySeconds
if ($result.Success) {
$imported += $zip.Name
Write-Host "Imported '$($zip.Name)' as '$artifactType'."
}
else {
$failed += @{ File = $zip.Name; Error = $result.Error }
if ($result.Body) {
Write-Verbose "Final failure body for '$($zip.Name)': $($result.Body)"
}
Write-Error "Failed to import '$($zip.Name)'."
}
}
Write-Host "Imported files: $($imported -join ', ')"
if ($imported.Count -gt 0) {
Write-Host "##vso[task.setvariable variable=importedFiles]$($imported -join ',')"
}
if ($failed.Count -gt 0) {
Write-Error "Failed to import $($failed.Count) file(s)."
foreach ($f in $failed) {
Write-Error " - $($f.File): $($f.Error)"
}
exit 1
}
Write-Host "All imports completed successfully."
exit 0
}
catch {
Write-Error "Import failed: $($_.Exception.Message)"
exit 1
}