A VTK pipeline primer (part 2)

05 Oct 2014

In a previous blog, I covered some of the preliminaries for understanding how VTK's pipeline works. In this article, we will see the pipeline in action and start dissecting the execution path to understand the inner-workings of algorithms.

Let's start with a very simple pipeline.

a pipeline

We'll use the following code for the source and the algorithm.

import vtk
from vtk.util.vtkAlgorithm import VTKPythonAlgorithmBase

class MySource(VTKPythonAlgorithmBase):
    def __init__(self):
        VTKPythonAlgorithmBase.__init__(self,
            nInputPorts=0,
            nOutputPorts=1, outputType='vtkPolyData')

    def RequestInformation(self, request, inInfo, outInfo):
        print "MySource RequestInformation:"
#        print outInfo.GetInformationObject(0)
        return 1

    def RequestUpdateExtent(self, request, inInfo, outInfo):
        print "MySource RequestUpdateExtent:"
#        print outInfo.GetInformationObject(0)
        return 1

    def RequestData(self, request, inInfo, outInfo):
        print "MySource RequestData:"
#        print outInfo.GetInformationObject(0)
        return 1

class MyFilter(VTKPythonAlgorithmBase):
    def __init__(self):
        VTKPythonAlgorithmBase.__init__(self,
            nInputPorts=1, inputType='vtkPolyData',
            nOutputPorts=1, outputType='vtkPolyData')

    def RequestInformation(self, request, inInfo, outInfo):
        print "MyFilter RequestInformation:"
#        print outInfo.GetInformationObject(0)
        return 1

    def RequestUpdateExtent(self, request, inInfo, outInfo):
        print "MyFilter RequestUpdateExtent:"
#        print outInfo.GetInformationObject(0)
        return 1

    def RequestData(self, request, inInfo, outInfo):
        print "MyFilter RequestData:"
#        print outInfo.GetInformationObject(0)
        return 1

Then we instantiate and execute this pipeline with the following code.

s = MySource()

f = MyFilter()
f.SetInputConnection(s.GetOutputPort())

f.Update()

This will print the following.

MySource RequestInformation:
MyFilter RequestInformation:
MyFilter RequestUpdateExtent:
MySource RequestUpdateExtent:
MySource RequestData:
MyFilter RequestData:

The most interesting thing here is the order of execution of the various methods representing pipeline passes. Here are the order of the passes:

RequestInformation
RequestUpdateExtent
RequestData

Furthermore, notice during RequestInformation and RequestData, the method execution happens starting at the upstream algorithm (MySource) and continues downstream. On the other hand, during RequestUpdateExtent, execution starts downstream and continues upstream. Why this is so will become more clear as we discuss each pass.

RequestInformation

This is the meta-data pass. This is the pass where the sources (usually readers) tell downstream algorithms about what is available for them to request. Some examples about meta-data include whole extent (see this blog for example), timesteps available, ensemble members available, data blocks available etc. This meta-data usually originates in the readers (not necessarily required to though) and is either copied downstream or modified. For example, a subsetting filter can reduce the whole extent during its RequestInformation telling downstream that it can access a smaller subset that what the reader can provide. Or a filter that integrates a value over time may remove the timesteps meta-data since its output represent the whole time range. We'll see plenty of examples of this in upcoming blogs.

To demonstrate how this works, let's modify our example a bit. Let's create a meta-data key with:

from vtk.util import keys

metaDataKey = keys.MakeKey(keys.DataObjectMetaDataKey, \
  "a meta-data", "my module")

and change our source to create a meta-data instance with this key as follows

class MySource(VTKPythonAlgorithmBase):
    def RequestInformation(self, request, inInfo, outInfo):
        print "MySource RequestInformation:"
        outInfo.GetInformationObject(0).Set(metaDataKey, vtk.vtkPolyData())
        print outInfo.GetInformationObject(0)
        return 1

Let's also change our filter to print its output information in RequestInformation as follows

class MyFilter(VTKPythonAlgorithmBase):
    def RequestInformation(self, request, inInfo, outInfo):
        print "MyFilter RequestInformation:"
        print outInfo.GetInformationObject(0)
        return 1

Now if we execute the RequestInformation pass with

f.UpdateInformation()

we get the following output

MySource RequestInformation:
vtkInformation (0x7fe5aadb56b0)
  ...
  a meta-data: vtkPolyData(0x7fe5aae5a410)


MyFilter RequestInformation:
vtkInformation (0x7fe5aad4ade0)
  ...
  a meta-data: vtkPolyData(0x7fe5aae5a410)

Note that the output information of MyFilter contains the meta-data generated by MySource. Certain keys are automatically copied by the pipeline from upstream to downstream during the RequestInformation pass. For example, any key that is an instance of vtkInformationDataObjectMetaDataKey is copied automatically. Now say that MyFilter behaves in such a way that the meta-data coming from upstream needs to change for downstream. For example, if it computes a subset of its input, the extent meta-data would have to be changed. Here is how something like this can be accomplished.

29 class MyFilter(VTKPythonAlgorithmBase):
...
35     def RequestInformation(self, request, inInfo, outInfo):
36         print "MyFilter RequestInformation:"
37         print outInfo.GetInformationObject(0)
38         metaData = inInfo[0].GetInformationObject(0).Get(
39             metaDataKey)
40         newMetaData = metaData.NewInstance()
41         newMetaData.ShallowCopy(metaData)
42         someArray = vtk.vtkCharArray()
43         someArray.SetName("someArray")
44         newMetaData.GetFieldData().AddArray(someArray)
45         outInfo.GetInformationObject(0).Set(metaDataKey, newMetaData)
46         print outInfo.GetInformationObject(0)
47         return 1

Here, on line 38, we extract the meta-data from the input information, we make a copy of it on line 40-41, we add a new array to the copy on lines 42-44 and on line 45, we overwrite the output meta-data with the copy. If we run this example, the output will look like the following.

vtkInformation (0x7f9eab59bd30)
  ...
  a meta-data: vtkPolyData(0x7f9eab59f840)


MyFilter RequestInformation:
vtkInformation (0x7f9eab59c0b0)
  ...
  a meta-data: vtkPolyData(0x7f9eab59f840)


vtkInformation (0x7f9eab59c0b0)
  ...
  a meta-data: vtkPolyData(0x7f9eab5a0500)

Note how the pointer referenced by "a meta-data" is different on the third printout. Here is a graphical presentation of what is going on.

meta-data

Next, let's look at a more realistic example. See this blog for details. Our pipeline looks like this:

hdf5 pipeline

HDF5Source's RequestInformation is as follows:

def RequestInformation(self, request, inInfo, outInfo):
        f = h5py.File(self.__FileName, 'r')
        dims = f['RTData'].shape[::-1]
        info = outInfo.GetInformationObject(0)
        info.Set(vtk.vtkStreamingDemandDrivenPipeline.WHOLE_EXTENT(),
            (0, dims[0]-1, 0, dims[1]-1, 0, dims[2]-1), 6)
        return 1

RequestSubset's RequestInformation is as follows:

def RequestInformation(self, request, inInfo, outInfo):
    info = outInfo.GetInformationObject(0)
    info.Set(vtk.vtkStreamingDemandDrivenPipeline.WHOLE_EXTENT(), \
        self.__UpdateExtent, 6)
    return 1

You will notice the same design pattern:

HDF5Source creates meta-data in RequestInformation,
This meta-data is copied to the output of RequestSubset by the pipeline,
RequestSubset overwrites this meta-data in RequestInformation to fit its output.

Although the meta-data involved as well as the copying logic (specially when multiple inputs and/or outputs are involved) can get fairly complicated, if you understand this basic pattern, you will be able to decipher much of what is going on during RequestInformation.

I will cover RequestUpdateExtent and RequestData in upcoming blogs. Afterwards, we will be fully equipped to explore what can be achieved using VTK's pipeline.

Berk Geveci About Archive Feed

A VTK pipeline primer (part 2)

RequestInformation