Back in December I presented a new conference talk on Oracle Data Integrator in a DevOps world. The slides are available from the TECH 15 conference part of the UKOUG website, or if you are not a member, contact me and I’ll see if I can get them to you.
Most of my talk was about scripting processes so that we can easily reproduce them in an automated deployment world. One of the joys of ODI is that the API exposed through the SDK is so rich; for example, we can find repository objects such as projects and then iterate though the project exporting the most recently generated scenario file for each mapping (or interface depending on ODI version), package or procedure. In think that there is only one thing that you can do in the ODI studio that you are unable to do in scripting and that is managing ODI versions; a feature that I seldom, if ever use.
One of the points I raised in my talk is that scripted source control is not the same as scripted software release. We are dealing with different object types held in different parts of the repositories. I will blog more about this later in 2016. For now I am just going to mention a bit about which things sit where in the repository. Knowing this helps us to design strategies for automating development source control and for automating deployments and code promotion.
ODI is completely metadata driven – everything sits in a database repository; in fact two repositories, a MASTER repository and a work repository. Although it is possible to have a single master to multiple work repositories I favour one master, one work as this often fits better to enterprise data access rules that keep developers away from real data and also maintains a degree of platform isolation so that upgrades and patches to the ODI software can be cascaded through platforms and not as one step that affects multiple environments. That said there is something to be said for multiple developers working in their own ‘sandbox’ development work repositories whilst sharing a common master repository – another topic for later.
Now we have established my one-to-one preferences I should also mention that we have two flavours of work repository: the design repository and its subset the execution (runtime) repository.
The master repository contains the kind of things we may want to share across multiple work repositories. The two big things are the security components and the topologies, that is the physical and logical connection details for all of our source and target data stores. The one thing that is not here is the design of the data model (of that more later) The master repository is also used for ODI versions which are in effect named copies of work repository artefacts that can be downloaded to work repositories to replace existing objects.
The execution part of the work repository holds the runtime scenarios and load plans together with the associated execution logs. In the case of the ‘design’ variant of the work repository we also store the objects we edit on the designer tab of the navigator in ODI Studio, that is the models and data stores and all of the project-related code objects.
This gives us three fundamental areas where we can develop scripting.
- The Topology for both developer repository set up and deployments.
- The Code and Models for developer use, such as source control and developer environment set up.
- The Scenarios and Load Plans for deployment processes
.More on this in other blogs in this series
DW_PETE Talks Data 