JOBBUILDER FOR DATASTAGE
Metadata-driven generation of jobs and sequences
With JobBuilder for DataStage you generate executable jobs and sequences based on metadata. The implementation of ETL processes is accelerated by 100 %. Changes to the domain-oriented specification are maintained in the metadata and lead to a comfortable re-generation of the job at the push of a button. If required, generated jobs for hot fix activities can be processed manually and delivered to production.
Structured methodical approach in design and development.
Specification model: The technical specification in a standardized mapping document is the starting point for an interpretation-free and documented start into the implementation.
ETL framework: The use of reusable modules (e.g. via shared containers or buildop stages) for standard tasks such as change detection, harmonization, lookups and surrogate key assignment guarantees the correct and standardized implementation of processes. The wheel is only invented once!
Job Templates: the components of the framework are arranged to customer-specific, precisely fitting process scenarios: Templates for the data flow from source to target.
Central changes to components and templates can be transferred to all ETL processes at the push of a button via generation.
Simple user interface for controlling the JobBuilder Engine
The web-based JobBuilder Frontend performs all essential configuration and administration tasks:
- Creating and configuring the directories for the metadata
- Start, monitoring and logging of job generation
- Import of generated jobs and sequences into the Service Tier of the IBM Information Server
High-quality traceable data integration processes
The job templates standardized in the framework map all necessary and meaningful work steps for the clearly structured data flow from source to target in the DWH Core DWH. The generated processes are easy to understand and highly traceable across projects.
Job Builder for DataStage Architecture
- directory configuration
- Start of generation
- Generation monitoring
Job Builder Backend:
- Reading in the metadata
- Generation of jobs and sequences
- Export to service animal and logging
Defined customer-specific job templates generate jobs and sequences for loading into the core DWH
- 1:1 – Mappings
- All conversion rules according to DataStage Transformersyntax
- customer-specific higher-value transformations via shared container
Job generation starts in the frontend:
- Selection of one or more mapping specifications
- You can optionally overwrite the template defined in the mapping specification.
Quick overview of generation orders:
- Filtering of orders by author or status
- Display of detailed error messages
- Import of the generated job to DataStage
The JobBuilder generates executable jobs and sequences from the metadata and the selected job templates.
- Insert the job parameter
- Assignment of links with columns
- Adoption of the mapping specifications from the mapping specification in Transformer Stage
- Duplication of links and stages
- Complete the Full job description with generation information
shorter development period
The generation of DataStage jobs and sequences accelerates the realization of ETL processes by 100 %. The use of components and job templates facilitates familiarization with existing processes. The maintenance of existing processes is carried out efficiently without lengthy manual activities. The test effort is reduced due to quality-checked standard components.
The use of standard components and the flexible arrangement in job templates guarantees the customer precisely fitting processes from specification to implementation. The structured design and development process leads to traceable ETL processes.
Automating the development process reduces training time in data integration processes and the costs of initial implementation and maintenance.