Recording test scripts with UPT and replay them with DISA is fun, as we discovered in an earlier article. But what if we want to put the Auto in Test Automation?
Enter Jenkins, at our service.
In this article, we will explore how to install and configure Jenkins for automated test execution in Siebel CRM Innovation Pack 2017 (and probably later versions too) and how to execute and analyze test runs. The roadmap is as follows:
- Download and install Jenkins
- Configure Jenkins Server and Nodes
- Install the Siebel Test Execution plug-in for Jenkins
- Create a project
- Verify Prerequisites
- Create a Test Plan
- Create Test Sets
- Create Master Suites
- Create Automation Execution Configurations
- Schedule Run
- Build Jenkins project
- View Reports
- Create Test Passes
To understand Siebel Testing and for general reading pleasure, please refer to the Testing Siebel Applications guide in bookshelf.
Download and install Jenkins
Go to jenkins.io to download the latest Long Term Support (LTS) version for Windows. What you get is an .msi installer which lays down a Jenkins server on your machine, starts it up and opens a browser (default port is 8080) to log in as the admin user. At first login, the page tells you where to find the password (in a text file in the Jenkins installation folder).
Configure Jenkins Server and Nodes
First, go to the Manage Plugins page, and use the Available tab to locate and download the plugin called "Node and Label Parameter Plugin".
Next, we have to make a few tweaks to the default installation. In the Configure System page, we change the hostname from localhost to the FQDN and add a label "Master" to identify the master server.
The bookshelf guide also tells us to go to the Configure Global Security page and set the TCP Port selection for JNLP agents to "Random".
System Configuration
Then, we use the Manage Nodes page to create one or more slave nodes as required. Follow the instructions in the bookshelf guide to get the configuration settings right and deploy the slave.jar file on the slave machines. The Configuration page for the slave will have the instructions to start the slave as well.
Slave Node/Agent
Once the slave(s) are running, you can see that they are connected to the master and have "idle" status.
Install the Siebel Test Execution plug-in for Jenkins
Now comes the Siebel plugin. Again, go to the Manage Plugins page and this time we use the Advanced tab to upload the STE.hpi file which we shall find on the Siebel Server in the plugins directory. This plugin allows us to create a Build Step in Jenkins project which effectively launches DISA to play the test scripts (see below).
Siebel plugin upload
Create a Project
Now, we create a New Item, which essentially is a project, give it a meaningful name and select Freestyle Project (sounds nice anyway).
New Freestyle Project
In the Configuration page (which pops up once you create the project), go to the Build section and select Siebel Test Execution from the Add Build Step menu. Then provide the parameters for DISA etc as indicated
Siebel Build Step
Notice that the Siebel Server URL ends at /siebel which is correct (despite what bookshelf says *cough*).
Verify Prerequisites
Now that Jenkins is configured, let's check if Siebel is up to the game. On your Siebel Enterprise, verify the following.
- EAI Object Manager is exposed via /eai/lang in the Application Interface profile. This is needed because Jenkins will make Siebel inbound web service calls using /eai/enu
- DISA is deployed, started and fully operational on the slave machines, which – by the nature of DISA – must obviously run on Windows OS. DISA must be set up for Test Automation as discussed in a previous article, including the correct browser drivers to be downloaded to the DISA directory.
- If you use self-signed certificates (as is customary on internal networks), make sure that the Siebel Web Server (AI) .cer file is imported in the JRE's cacerts file (on all hosts hosting Jenkins master or slave nodes) using a command similar to the following:
keytool -importcert -alias siebelcert -keystore $JAVA_HOME/jre/lib/security/cacerts -storepass changeit -file <Path to Siebel AI .cer file>
Create a Test Plan
Test Automation in Siebel CRM is following a hierarchical model which looks like this (simplified):
Test Strategy – Test Plan – Test Set – Test Pass – Test Script
Automation Execution Configuration – Master Suite – Test Set – Test Script
It is optional to create a Test Strategy, but we need a Plan (otherwise there would be no Test Passes to attach to). So in the Siebel Release screen, go to the Test Plan Administration view and create a new plan.
There are quite a few required fields, one of which is labeled Release. The pick list is actually listing Projects. So it might be a good idea to go to the Projects screen first and create a project record if you haven't got one.
Create Test Sets
Creating Test Sets is straightforward. In the Test Sets view, create a new record, associate it with a Test Plan and the Project (Release field). After drilling down, you can associate one or more Test Scripts with your new Test Set.
Create Master Suites
A Master Suite is a collection of Test Sets. You use the Master Suites view to create them in a similar fashion as you created the Test Set(s).
Create Automation Execution Configurations
Next, go to the Automation Execution Configurations view and use the middle applet to create a new record. Associate the Master Suite and fill the required fields such as Application Version (the picklist shows Release Products, which you administer via the Quality screen).
After saving the record, the Server Credentials and User List (both applets show only a single record) are populated with default values.
Edit the records in both applets according to your environment.
Take care of the Endpoint URL which must be a valid Siebel application URL.
The status of the record will be "Hold" until you press the Schedule Run button…
Schedule Run
After clicking the Schedule Run button and acknowledging the message that it will be read only, the record will indeed become read-only with a Status of "Requested". In addition, the Attachments list will show two system-generated files.
Generated Attachments
There are two files. The CSV file contains the accumulated test steps from all test scripts in all test sets of the master suite. The batchconfig.xml file contains instructions for the Jenkins server.
Nothing else happens, you didn't expect anything else to happen, did ya?
Build Jenkins Project
Depending on your real-life setup, the Jenkins project might be scheduled to build regularly or at specific times. In a "Hello World" evaluation scenario, you will likely have to start the build manually. You do this by selecting Build Nowfrom the Project options in the Jenkins admin console.
Build Now
Now the magic happens. Your slave servers will become busy because DISA will start launching browsers and executing the Test Scripts. This will happen even with the lights out.
As usual, the DISA plugin generates reports, takes screenshots etc. When a slave is finished, files are zipped up and uploaded as a new attachment to the Automation Execution Configuration.
In addition, an XML file will be generated which can be used to generate Test Passes (see below).
Here's a screenshot of a successful build in Jenkins, so you get the general idea what goes on behind the scenes:
You will notice that Jenkins invokes a business service named Automation Test Service via REST to retrieve the ROW_ID of the requested test run(s). Then it issues a SOAP request to download the attachments. Subsequently, DISA is invoked and the results are uploaded to Siebel again.
View Reports
As mentioned above, the generated reports are uploaded as a zipped attachment. Simply download it and revel in a collection of log files, HTML summaries and screenshots.
Test Report
Create Test Passes
Finally – and optionally – you can select the XML file that the test run has generated and press the Create Test Passes button. This will generate Test Pass records associated with the respective Test Sets and Test Plans. You can use the Test Pass view to review the generated records. During my tests with 17.0, I got a "wrong value for pick list" error which I could reconcile by adding the "Completed" value to the PS_QA_TEST_STATUS LOV Type.
Summary
Setting up Lights Out Testing with Jenkins for the first time is quite a ride but you are rewarded with a multitude of options to get to the next level of Test Automation and release quality.
