Avoiding Variables in ODI File Copy

Copying files in ODI is a trivial task. There are many ways to do it (ODIFileCopy, Procedures, OSCommand etc.) One of the biggest issues though is when you need to move from environment to environment. Frequently I have implemented the file copy using variables and I have seen enough implementations to know that is fairly common. It's pretty easy, you just have to make sure the variable is set right on the target system and it generaly works well.

The problem is it requires the developer/administrator to have to remember additional steps or create code relying on context to determine the value of the variable. Upon thinking about this at my current project I wondered if this functionality could be acheived with an Knowledge Module (KM).

The benenfit of using a KM is that the source and destination are controlled by the physical/logical topology and context. This makes the functionality easily portable between environments. The other benefit is it avoids having to create a bunch of variables.

IKM File Copy



Copy File

import os
import java.sql as sql
import java.lang as lang
import shutil

def getPath(path, item):
    path = path.strip()
    if path[len(path) - 1:] == "\\":
	    path = path[:len(path) - 1]
    if item[:1] == "\\":
        item = item[1:]
	return os.path.join(path, item)

source = getPath(r"<%=odiRef.getSrcTablesList("", "[SCHEMA]", "", "") %> ", r"<%=odiRef.getSrcTablesList("", "[RES_NAME]", "", "") %>")
target = getPath(r"<%=odiRef.getTargetTable("SCHEMA")%> ", "<%=odiRef.getTargetTable("RES_NAME")%>")
shutil.copy(source, target)


The first thing to know is that this can be used with any file that is defined as a model in ODI. Even if the file is binary you will just have a to create a surrogate column in the model (ODI requires something to be mapped to use an IKM even if the map is unused.) Next its advisable to set the staging area to the same as the source so that an LKM can be avoided.
Eliminate the LKM

Next setup the mapping in to enable the IKM, remember that this does not do anything no transformation is being performed.
Setup a dummy mapping

Last set the IKM and you are ready to go!

More info

As usual I have added this to github for public consumtion. The full contents of the KM are below.