Example Use Case: Steps and Pipeline Section Overview

This section contains a rudimentary example to demonstrate how to use steps and pipelines in Xactly Connect. The Trigger and Order upload ETL processes described later in this document use more sophisticated steps and pipelines. This section provides a gentler introduction using some common features found in Xactly Connect.

The example contains two steps and a single pipeline. The pipeline will query Incent payment data for the JAN­2015 monthly period and place the results into a custom Xactly Connect table. The example uses isql, an open source command line, ODBC customer and Xactly’s ODBC driver.

 

Steps

Steps are the basic building blocks for assembling ETL processes within Connect. A step can contain a single XSQL expression. Steps are stored on the Xactly Connect server for later use.
Creating a step object does not execute the step. The CREATE STEP syntax creates a step object on the Xactly Connect server for later use. “Invoking” a step causes the Xactly Connect server to execute the command stored within a step.

Step 1: Create Step to Set Monthly Period Name in a Variable
When invoked, the s_example_set_period_name step will set a variable to the Incent monthly period name associated with the date, January 31, 2015. LookupPeriodName() is an XSQL scalar function that returns an Incent period name for the data and period type specified.

Step 2: Create Step to Query Incent Tables for Specified Monthly Period
When invoked, the s_example_create_pay_extract step will join three Incent tables and insert the result set into a custom Xactly Connect table. If the target table does not exist, Xactly Connect will create it on the fly based on the column names, values, and data types found in the SELECT.

 

Pipelines

Pipelines organize multiple steps into a single processing unit. In addition to steps, pipelines may contain other pipelines. Pipelines are executed only when “invoked”. Creating a pipeline simply creates the pipeline object on the Xactly Connect server for later invocation.

Create a Pipeline Object

First, create a pipeline object, p_example_pay_extract. The pipeline object is stored on the Xactly Connect server.

Add Steps to the Pipeline

Next, add the two steps created previously to the pipeline. When the pipeline is invoked, the steps will fire asynchronously in the order they were added to the pipeline.

Invoke the Pipeline

Finally, invoke the pipeline to execute it on the Xactly Connect server. Recall, the pipeline contains two steps. The first step determines the monthly period name to process. The second step queries Incent data for the specified period and inserts the results into a custom table.

Review the Pipeline Results

The following examples show two ways to check the pipeline results. One method is to query the target table from the pipeline. The second example queries Xactly Connect invocation history log using the show invocation details command.