Book demo Start trial

TFS Integration

Automatic creation of bugs in TFS

An important part of running automated test cases is to monitor, inspect and react on the results. This is best done in the test management and bug tracking system that you use in your daily work.

In this article we describe three different scenarios of how to trigger and create/update bugs in Team Foundation Server (TFS) based on the result of test cases running in LEAPWORK.

 

Using a plugin for VSTS or TFS

One of the easiest ways to integrate LEAPWORK with Visual Studio Team Services or Team Foundation Server 2015 and later, is to use the LEAPWORK plugin. This extension can be installed from the Visual Studio Marketplace. Please visit the direct link, Or alternatively,

This LEAPWORK plugin is a community developed and can be found on GitHub.

 

Creating bugs from Schedules

To run a collection of test cases on one or more environments (machines, devices etc.) on a scheduled basis or by triggering it through the LEAPWORK REST API, you define a Schedule. In a Schedule you can select one or more test cases from a LEAPWORK project and pair them with one or more environments. Further you set at what time and how often the schedule should run.

As part of a Schedule you can define one or more Actions. An example of an action is to send an e-mail, when the schedule has ended. Another is to call a specified web service if more than X number of test cases in the schedule has failed.

In this case we will trigger a PowerShell action when the schedule is done. We will use this action to call a local PowerShell script and pass the result of the schedule to the PowerShell script. An example of how to specify this action is shown in the figure below:

tfs

The command tells LEAPWORK to run a PowerShell script: “c:\scripts\createTFSbugs.ps1”. Download the script (zip) by clicking here: CreateTFSbugs and extract it to the location where you want to execute the script from.

Further we have added a few parameters: 

  • -result: This parameter is read by the PowerShell script and is the result of the entire schedule. To add the result data right click in the command field and select “Insert token -> RESULTS-JSON”.It is VERY important that the inserted token is surrounded by
    @”
    [

    before the token, and
    ]
    “@
    after the token - including the line breaks. This structure ensures that PowerShell transfers the JSON result in the right format.
  • -rootPath: This parameter is read by the PowerShell script and is the file path to the folder containing the videos and screenshots recorded during the execution of the test cases.

The PowerShell script, “CreateTFSbugs.ps1” will connect to the TFS REST API and create or update bugs based on the failed test cases in the Schedule.

In order to use the PowerShell script you need to specify a few settings about you TFS installation:

tfs 2

domain: If you are using Visual Studio Online (VSO), insert your visualstudio.com domain name, e.g. “https://LEAPWORK.visualstudio.com”

tfsPath: The path to your project.

pat: Personal Access Token. You can find more information about setting up a Personal Access Token here.

logfile: The path to the integration log file. All log entries are appended to this file.

The script has been tested against Visual Studio Online, but it’s the same API used for the OnPremise installations of TFS. There might be differences in the version of the REST API. Please consult the MS documentation for further info about this.

 

Triggering schedule from API and create bugs in TFS

Using four simple steps to integrate LEAPWORK with any third-party system through the REST API, automatically creating or updating bugs in TFS when test automation cases are failed in LEAPWORK is very easy.

Download sample PowerShell script.


# LEAPWORK REST API example: Run a schedule, iterate through the results and create bugs in TFS.# 
#
# Author: Claus Topholt.


# Function that finds a schedule in LEAPWORK based on a title, runs it and polls for the results.
function RunScheduleAndGetResults($schedule)
{
    Write-Host "Getting id for schedule '$schedule'."

    # Get the id of the schedule.
    $runScheduleId = "";
    $headers = @{}
    $headers.Add("AccessKey","bTyGAd0UGL70JFQg")
    $runSchedules = Invoke-WebRequest -ContentType "application/json" -Headers $headers "http://localhost:9001/api/v3/schedules" | ConvertFrom-Json
    foreach($runScheduleItem in $runSchedules)
    {
        if ($runScheduleItem.title -eq $schedule) { $runScheduleId = $runScheduleItem.id }
    }
    if ($runScheduleId -eq "") { throw "Could not find schedule '$schedule'." }

    Write-Host "Running the schedule."

    # Run the schedule now.
    $timestamp = [DateTime]::UtcNow.ToString("ddMMyyyy HHmmss")
    Start-Sleep 1
    $runNow = Invoke-WebRequest -Method PUT -ContentType "application/json" -Headers $headers "http://localhost:9001/api/v3/schedules/$runScheduleId/runNow"
    if ($runNow.StatusCode -ne 200) { throw "Could not run schedule." }
    $runNowResponse=$runNow.Content | ConvertFrom-Json
    $runId=$runNowResponse.RunId
    # Get the result, keep polling every 5 seconds until a new result is returned.
    do
    {
        Start-Sleep -Seconds 5

        Write-Host "Polling for run results."

        $runResult = Invoke-WebRequest -ContentType "application/json" -Headers $headers "http://localhost:9001/api/v3/run/$runId" | ConvertFrom-Json         

    } 
    while ($runResult.Status -ne 'Finished')

    Write-Host "Results received."

    return $runResult
}


# Function that creates a bug with a specific title in TFS or updates if it already exists.
function CreateOrUpdateBug($title, $description)
{
    # Create an authorization header using a Personal Access Token.
    $pat = "YOUR PERSONAL ACCESS TOKEN HERE"
    $patEncoded = [Convert]::ToBase64String([Text.Encoding]::ASCII.GetBytes(":$($pat)"));
    $header = @{"Authorization" = "Basic $patEncoded"}

    # See if a bug with the same title already exists.
    $queryAlreadyExists = '{ "query": "SELECT [System.Id], [System.Title], [System.State] FROM WorkItems ' +
                          'WHERE [System.WorkItemType] = ''Bug'' AND [System.Title] = ''' + $bugTitle + ''' AND [System.State] <> ''Removed'' ' + 
                          'ORDER BY [System.Id] DESC" }'
    $urlAlreadyExists = "https://YOURINSTANCE.visualstudio.com/DefaultCollection/YOURPROJECT/_apis/wit/wiql?api-version=1.0"
    $header['Content-Type'] = 'application/json'
    $resultAlreadyExists = Invoke-RestMethod -Headers $header -Method Post -Body $queryAlreadyExists $urlAlreadyExists

    # If the bug already exists, update it.
    if ($resultAlreadyExists.workItems.Count -gt 0)
    {
        Write-Host "Updating existing bug $($resultAlreadyExists.workItems[0].id)."

        # Add a note that this bug was updated or re-opened.
        $timestamp = [DateTime]::UtcNow.ToString("dd-MM-yyyy HH:mm:ss")
        $bugDescription = "Updated or re-opened by LEAPWORK on $timestamp.

" + $bugDescription

        # Update bug.
        $queryUpdateBug = '[ { "op" : "replace", "path" : "/fields/System.Title", "value" : "' + $bugTitle + '" }, ' +
        '{ "op" : "add", "path" : "/fields/Microsoft.VSTS.TCM.ReproSteps", "value" : "' + $bugDescription + '" }, ' + 
        '{ "op" : "replace", "path" : "/fields/System.State", "value" : "New" } ]'
        $urlUpdateBug = "https://YOURINSTANCE.visualstudio.com/DefaultCollection/_apis/wit/workitems/$($resultAlreadyExists.workItems[0].id)?api-version=1.0"
        $header['Content-Type'] = 'application/json-patch+json'
        $resultUpdateBug = Invoke-RestMethod -Headers $header -Method Patch -Body $queryUpdateBug $urlUpdateBug
    }
    else
    {
        Write-Host "Creating new bug."

        # Create new bug.
        $queryCreateNewBug = '[ { "op" : "add", "path" : "/fields/System.Title", "value" : "' + $bugTitle + '" }, ' +
        '{ "op" : "add", "path" : "/fields/Microsoft.VSTS.TCM.ReproSteps", "value" : "' + $bugDescription + '" }, ' + 
        '{ "op" : "add", "path" : "/fields/System.State", "value" : "New" } ]'
        $urlCreateNewBug = 'https://YOURINSTANCE.visualstudio.com/DefaultCollection/YOURPROJECT/_apis/wit/workitems/$Bug?api-version=1.0'
        $header['Content-Type'] = 'application/json-patch+json'
        $resultCreateNewBug = Invoke-RestMethod -Headers $header -Method Patch -Body $queryCreateNewBug $urlCreateNewBug
    }
}


# Run the LEAPWORK schedule "My Test Schedule" and get the results.
$runResult = RunScheduleAndGetResults("My Test Schedule")

# If there are any failed cases in the results, iterate through them.
if ($runResult.Failed -gt 0)
{
    Write-Host "Found $($runResult.Failed) failed case(s)."
    $headers = @{}
    $headers.Add("AccessKey","bTyGAd0UGL70JFQg")
    $runId=$runResult.RunId
    $runItemIds = Invoke-WebRequest -ContentType "application/json" -Headers $headers http://localhost:9001/api/v3/run/$runId/runItemIds | ConvertFrom-Json 

    $rootPath =$runResult.RunFolderPath
    foreach ($runItemId in $runItemIds.RunItemIds)
    {   
   
    $runItems = Invoke-WebRequest -ContentType "application/json" -Headers $headers http://localhost:9001/api/v3/runItems/$runItemId | ConvertFrom-Json 

        if($runItems.FlowInfo.Status -eq 'Failed')
        {

         # Create a title for the bug.
            $bugTitle = "LEAPWORK: " + $runItems.FlowInfo.FlowTitle

            # Create a description that contains the log messages.
            $newline = "\r\n";
           
            $keyFrames = Invoke-WebRequest -ContentType "application/json" -Headers $headers http://localhost:9001/api/v3/runItems/$runItemId/keyframes/1 | ConvertFrom-Json 

            $bugDescription = "Log from LEAPWORK:$newline $newline"
            foreach ($keyframe in $keyFrames)
            {
                if ($keyframe.Level -ge 1)
                {
                    $keyframeTimestamp = get-date($keyframe.Timestamp.LocalDateTime) -Format "dd-MM-yyyy HH:mm:ss"
                    $bugDescription += "$keyframeTimestamp - $($keyframe.LogMessage) $newline"
                }
            }

            # Add path to video and screenshots.
            $mediaPath = Join-Path -Path $rootPath  "$($runItems.RunItemId)"
            $videoPath = Join-Path -Path $mediaPath  "$($runItems.RunItemId).avi"
            $videoPath = $videoPath.Replace('\', '\\')
            $screenshotsPath = Join-Path -Path $mediaPath "Screenshots"
            $screenshotsPath = $screenshotsPath.Replace('\', '\\')
            $bugDescription += "$newline Video: $videoPath $newline"
            $bugDescription += "$newline Screenshots (if any): $screenshotsPath $newline"

           
            # Create or update bug in TFS.
            CreateOrUpdateBug($bugTitle, $bugDescription)
        }
    }

}
else
{
    Write-Host "No failed cases found."
}

The script runs a pre-defined LEAPWORK schedule, polls for the results until they are available, and then loops through all failed cases and creates or updates bugs in Visual Studio Online (Microsoft’s cloud TFS solution) as appropriate.

Please note that the script contains no error handling or logging mechanisms. It is meant only to demonstrate the core functionality of integrating LEAPWORK with TFS.

After running, cases will be created in TFS and can be managed for instance through Visual Studio:

tfs 3

The above script uses Personal Access Tokens (PATs) to authorize access to TFS (Visual Studio Online). For information about how to setup PATs and how to perform TFS REST calls, please see the following links:

https://www.visualstudio.com/en-us/docs/setup-admin/team-services/use-personal-access-tokens-to-authenticate

https://www.visualstudio.com/en-us/docs/integrate/get-started/rest/basics

You can explore the endpoints mentioned above by going to this url: http://localhost:9001/help/index if you have the Controller installed on your own computer.

Explore the full documentation of the LEAPWORK REST API.

If you have any questions, please contact priority support on prioritysupport@leapwork.com.