Tech Articles


Revisiting Continuous Integration with PowerBuilder 2017, Bonobo Git and Jenkins


In a previous blog post I ran through a example of how to use the new command line compiler feature of PowerBuilder 2017 to perform continuous integration using Jenkins and a Git repository.  One of the "next steps" I referenced in the close of that post was the creation of a plug-in for Jenkins that would make creating the build steps for the project much simpler.

Well, I've created such a plugin.  Or more specifically what I did was take the existing MSBuild plugin and customize it to create a number of plugins to do PowerBuilder compiles instead.

In this post I'm going to look at how we can use those plugins, plus some additional changes I've made to the project in Jenkins to make better use of the capability of Jenkins and other Jenkin plugins.

Download and install the Oracscript and PBCompile plugins

The first thing you'll need to do is grab a copy of the OrcaScript and PBCompile plugins.  What you'll need is the orcascript.hpi and pbcompile.hpi files.  There are different versions depending on the version of PowerBuilder you are using

  Orcascript PBCompile
2017

orcascript-plugin-2017

pbcompile-plugin-2017

2019

orcascript-plugin-2019

pbc_compile-plugin-2019


Go into Jenkins and select the "Manage Jenkins" option, and then the "Manage Plugins" link on that page.

On the Plugin Manager page select "Advanced" and then look for the "Upload Plugin" section. Click the "Browse" button there and navigate one of the HPI files you downloaded.

Do it again for the second HPI file.  You'll may need to restart the Jenkins server after installing the plugins before you can use them.

Configure the plugin installations

After the server has been restarted go back into "Manage Jenkins" and then "Global Tool Configuration".

Scroll down that page an you'll see a new "Orcascript" section.  Select the "Add OrcaScript" button.

In the form that appears, provide the installation with a name and the path to the OrcaScript executable (orcascript170.exe, or orcascript190.exe).

Scroll down the page a bit more and you will see a new "PBCompile" section.  Select the "Add PBCompile" button as before.  In the form that appears,provide the installation with a name and the path to the PowerBuilder command line compiler (pbc170.exe or pb190.exe).

Create OrcaScript and PBCompile build steps

Go back into your PowerBuilder project and select the "Configure" link.

Delete all the previous build steps that called batch files.  Now add a new one and notice a new entitled "Create PowerBuilder PBLs from source code".  Select that one.

In the form that appears, select the version of the OrcaScript installation you just created and provide the relative location to the PowerBuilder project file.  If you have PBDs in the library path that you want OrcaScript to ignore, provide them in the Exclude Liblist option.  The open source PFC contains a reference to the PBDOM PBD, so we're excluding that here.

Add another build step and this time select the "Build a PowerBuilder target using PBC" option.

In the form that appears, select the version of the PBCompile that you just created earlier.

You'll also need to add the command line arguments to pass to the PowerBuilder Autocompile utility.  You can get those from the first page of the project object in your PowerBuilder target.

hese plugins provide the same capability as the first two batch files from the previous blog article, namely running OrcaScript to create PBLs from source control and then running the AutoCompile Utility (PBC170.exe) to create the executables and PBD from the PBLs.

Archive the Project Artifacts

What we're going to do now is replace the third batch file, the one that copies the EXE and PBDs into the build folder.  Jenkins has some built in capabilities that make this simpler that I didn't demonstrate in the previous article.

The first thing we're going to do is to tell Jenkins to archive off the EXE and PBDs (Jenkins calls them "artifacts").  To do this, create a Post Build step and select "Archive the Artifacts"

In the new step that is added, enter the following in for the "Files to Archive".

       **/*exe, **/*.pbd

The syntax is Apache Ant include fileset.  This particular setting tells Jenkins to find all of the EXE and PBD files in the workspace - including in any subdirectories - and copy them into the archive location.  When it does so it preserves the subdirectory structure, something we'll deal with in the next step.

Install the Copy Artifacts plugin

Now that we have the PBDs and EXE in the archive location, we want to use the Copy Artifacts Plugin to copy them to the build folder.  First we'll need to install the plugin.  Go back to the Manage Jenkins  -> Manage Plugins page, but this time select the "Available" tab.  Because there are a lot of plugins for Jenkins you might want to use the Filter option in the upper right to pear down the list.

Once again you'll need to restart the server before you can use the plugin.

Create a Copy Artifacts project

After the server has been restarted, create a new project.  We'll fire this project off after the build is complete from our original PowerBuilder project.  You might simply name the new project as the original project name with an " - Output" suffix, as I've done here.

The only thing we're going to do with this new project is create one build step.  Do that, and choose the new "Copy artifacts from another project" option.

  • Project to Copy From:  Select your original PowerBuilder project. 
  • Artifacts to Copy:  You can leave this blank, as it defaults to copying all artifacts.
  • Target Directory:  The he location you want the files copied to, and can reference the Jenkins environmental variables using $VARIABLENAME (as opposed to %VARIABLENAME% that we used in the batch files).  For example, I'm using:

    c:\builds\$JOB_NAME\$BUILD_NUMBER

    In the previous article I was concatenating the job_name and build_number for the directory.  I like this better as I'm now putting the builds in subdirectories under the job name.
  • Flatten Directories:  The one other option you will want to change is to check the "Flatten Directories" option.  That tells the plugin to just copy over the EXE and PBD files and ignore the subdirectory structure.

Create a Post Build step to fire the Copy Artifacts project

Go back into the original PowerBuilder project.  We need to add another Post Build step. This time select "Build other projects".

And in the new post build step specify your "output" project as the project to run after the PowerBuilder project completes.

Test the new configuration

Run a build of your PowerBuilder project just to verify that it's all working.  You'll notice some differences.  In particular, the build information now shows the artifacts that were created and the "output" project as a "Downstream Project".

The build information from the "output" project isn't particularly interesting, but we can check out build folder and see that the build files are being created and stored correctly.

Caveats

This all works great.  The only issue is that (currently) there are a number of options available in the PowerBuilder target that aren't supported by the command line arguments for the PowerBuilder autocompiler.  Well we have some other options.  One is to user PowerGen, and the other is to use the command line options of the PowerBuilder IDE itself.  Fortunately, I also created a plugin for each of those as well.  (That was true when this article was originally published.  It was addressed in later releases of PowerBuilder.) 

PowerGen plugin

If you'd like to use PowerGen instead, grab a copy of the PowerGen plugin.  You'll need the powergencompile.hpi.  Install it as you did the other two plugins above.  Similarly, go to the Global Tool Configuration and create a PowergenCompile installation.

 

PowerGen version  
9.5 (2017 and below)

powergen-plugin-2017

10.0 (2019 and below)  

 

Like the other two plugins, the only thing you need to do here is give the installation a name and provide a path to the PowerGen executable (Pwrgn17.exe).

Go back into your project, delete the PBCompile step and add instead a PowerGen step.

And in the form that appears there selection the install you just created and the relative location of the PowerGen project file you've created for the project.

You'll need to save or copy that PowerGen project file in the workspace.

Caveats

There's a couple of caveats with this approach as well though, both of which I've shared with the folks who make PowerGen.

  • PowerGen doesn't output to the console.  The other utilities we've used so far do, and it's useful for following the progress of a build.  Instead, PowerGen (if the project file is configured for it) outputs to a log file.  What I've done is added *.log to the artifacts.  As a result, the log file PowerGen generates appears in the project summary page and I can click on it to display it after the build.

 

  • Powergen doesn't indicate build errors by returning a non-zero exit code.  Instead, when there are errors it writes out a pgerror.log file.  However, a non-zero exit code (also often referred to as an ERRORLEVEL code) is what Jenkins is looking for from such tools to indicate when their task has failed.  It's certainly possible that from within the plugin I could look for the existence of a pgerror.log and flag the task as failing.  However, in the long run I think it would be cleaner if PowerGen could update its behavior to include the non-zero exit code.

(The above was true at the time the article was originally written.  PowerGen was later revised to address those caveats).

Note that PowerGen also has the capability to generate PBLs from source code like OrcaScript does.  I haven't looking into making use of that capability as I already have OrcaScript doing that for me.

PowerBuilder IDE Plugin

The last option is to use the PowerBuilder IDE itself to do the compile using command line arguments.  To do that, grab a copy of the PowerBuilder IDE plugin.  You'll need the pbidecompile.hpi file.  Install it as you did the others, and then go into Global Tool Configuration and give it a name and the location of the PowerBuilder IDE.

 

PowerBuilder IDE version  
2017

pbide-plugin-2017

2019  

 

 

Go back and replace that last build step with the new "Build a PowerBuilder target with the PowerBuilder IDE command line" option that appears.

And in the form that appears, select the version of the install you just created and indicate the relative location of both the workspace and project file.

Note that the workspace file is a new requirement, and something that isn't part of the source pulled over from the source control server.  You'll have to copy or create a workspace file in the Jenkins workspace in order to use the PowerBuilder IDE command line approach.

Caveats

  • You have to purchase an additional license of the PowerBuilder IDE and install it on the build machine for this option.
  • Like PowerGen, The PowerBuilder IDE doesn't output to the console.  So, like PowerGen, we write the output from the IDE to a log file.
  • You need to copy or create a PBW file for the workspace, as the PBW file isn't one of the source controlled objects.
  • It appears that problems generating a build don't cause the IDE to exit with an error code.  In fact, it seems to prevent the IDE from exiting entirely.  Instead, the build appears to hang as the IDE waits for user interaction.
Comments (24)
Friday, Nov 30 2018

When I'm following these instructions I do not see the orcascript configuration available in my "Global Tool Configuration" window.

I'm able to successfully install both plugins. I can see and configure the PBCompile in my global tool config but not the orcascript. Is there something I'm missing?

0

Tuesday, Jun 04 2019

Hi Bruce,
I have the same problem as Peter.
There also no build step "Create PowerBuilder PBLs from source code" available.
Do you know how to fix this?
Thank you.

0

Tuesday, Jun 04 2019

I did some updates on the source for the plugin, and if you use the latest source they may be an issue. To get it working until I fix the source, I would use the batch file approach I mentioned in the previous article (https://community.appeon.com/index.php/articles-blogs/tutorials-articles/2-powerbuilder/175-continuous-integration-with-powerbuilder-2017-bonobo-git-and-jenkins).

0

Friday, Apr 24 2020

Hi Bruce,
Your effort is much appreciated. It is timely to a major "build automation pipeline migration to Jenkins" project I am currently prototyping. Thank you

Two questions about ORCAScript PlugIn Build configuration:
1) For PBT file: Are these assumed?
scc connect offline
scc set target "" "refresh_all importonly"
2) I don't see options to Create PBLs or other commands such as:
a) create library
b) build executable
c) build library
d) set exeinfo
Are you planning to add these?
Or
Is current best practice to do them as BAT jobs in separate CMD pre and post build actions ?

Thanks again
Yakov Werde
OpenLink Financial
Sr Dev and Dev Ops Dude

Comment was last edited 4 years ago by yakov werde
0

Friday, Apr 24 2020

Hey Yakov!

1. You enter the name of the PBT file in the build configuration:

2. The plugin is hard coded to do " scc refresh target " + refreshType

See: https://github.com/bruce-armstrong/orcascript-plugin-2019/blob/master/src/main/java/hudson/plugins/orcascript/OrcascriptBuilder.java

That rebuilds PBLs from source.

The assumption is that you'll then use the PBC plugin to do the compile from the PBLs, so that's where the OrcaScript plugin stops.

The point of the plugins is to avoid having to write batch files. In the case of the OracSript plugin it creates an orcascript file for you and then runs it. The PBC plugin just runs the PBC utillity directly.

0

Friday, Apr 24 2020

Yea Bruce!

Question: All we are after in the next step is PBD and EXE packaging. Since The PB Project painter doesn't have a "Rebuild None" option won't it just repeat what was just done with Refresh Target?

(Worse yet, since it does NOT have a "3pass option it might nuke what was just done)

**In our case Refresh Target takes about 100 min"

Y

Comment was last edited 4 years ago by yakov werde
0

Friday, Apr 24 2020

The OrcaScript plugin assumes that you don't have your PBLs under source control or decide that you'd still rather build them from raw source code. If you have PBLs already, just use the PBC plugin. The PBC plugin doesn't build PBLs from raw source code.

If you have the project options set to "incremental" I would hope that would limit the amount of duplicate work. I don't know for sure though as I really don't know how PBC works internally and how it would know what should or should not be recompled as a result of the incremental setting .If you have it set to "full" then it would do another, likely unnecessary, recompile.

Yes, refresh target can take a while with a larger application.

0

Friday, Apr 24 2020

Thanks Bruce

From the IDE Project Painter on incremental perspective. I always observe all objects iwhirl by n the output window, even if nothing changed.....

So you recken that (Incremental) might == (None) depending on how it keeps track of what changed (detailed lookup or high level last change timestamp)

Maybe Roland Smith or one of the Appeon engineers could comment .....

I could kick it off and see what happens since I just finished a 2 hr refresh

To ensure a clean bootstrap I always create new PBLs explicitly

Documentation says nothing on this subject:

Does the Refresh Target create new PBLs such that CREATE Library steps are not needed

or

Does it just empty and refresh the pre-existing ones?

I place my bet on doing explicit Build EXE PBD steps

Yakov

0

Friday, Apr 24 2020

>>To ensure a clean bootstrap I always create new PBLs explicitly

We do as well. We're just a bit paraniod.

>>Does the Refresh Target create new PBLs such that CREATE Library steps are not needed

it creates new PBLs such that CREATE Library steps are not needed

0

Friday, Apr 24 2020

>>We do as well. We're just a bit paraniod.
Aren't we all?

>>it creates new PBLs such that CREATE Library steps are not needed
Thanks - I'll keep an eye on it for confirmation

PS: Refresh Target for SVN and GIT drivers create and populate a temporary PBG file for each PBL. They use the directory contents of the corresponding folder ws_objects\pbl.src folder to write the PBG.
This is how they 'shoehorned' the new PBG less (haha) process onto the the older PBG based one.

Y

0

Find Articles by Tag

API Repository 32-bit PostgreSQL ODBC driver JSONGenerator Sort .NET DataStore DevOps Service CI/CD REST SnapObjects Expression Elevate Conference PostgreSQL Class SqlExecutor OAuth Encryption UI SQL OLE PowerBuilder (Appeon) PDFlib Authorization Window SOAP SnapDevelop Migration Interface IDE Source Code PowerBuilder GhostScript Git Windows 10 Icons Export Jenkins Debugger Excel WinAPI Database Import JSON Trial InfoMaker RESTClient License Model Debug Encoding Authentication Platform WebBrowser Database Table Schema Icon Transaction RibbonBar Event Handling C# Open Source Database Table DataWindow Android Resize PowerServer Web TLS/SSL ActiveX TortoiseGit BLOB Performance iOS Windows OS Source Control Text Syntax Configuration 64-bit Messagging Installation NativePDF UI Themes MessageBox Mobile Event RibbonBar Builder Deployment Filter File Debugging DLL External Functions Stored Procedure .NET Assembly Event Handler Oracle Web Service Proxy Linux OS TreeView Database Object PowerScript (PS) PowerServer Mobile COM Validation TFS PowerBuilder Compiler Error HTTPClient SQL Server Database Painter SqlModelMapper UI Modernization Outlook JSONParser Automated Testing Script Testing Menu PBDOM OAuth 2.0 DataType JSON Variable Database Profile Export JSON ODBC Data Charts Branch & Merge DragDrop SVN Import Web API XML CrypterObject Database Connection Graph Database Table Data Array SDK DataWindow JSON OrcaScript Bug Azure Application RichTextEdit Control Design CoderObject .NET Std Framework PFC PDF