Modules how to python

We will here explain how to develop a python module. The purpose of the module is to concatenate two files. You can use the IDE to generate a skeleton for your module.

Get started

We will first go in the modules/nodes (or wherever you want to put your module) directory of dff sources and create a directory called filescat :

cd dff_sources/modules/nodes
mkdir filescat

Do not call your module 'test', it can generate some conflicts.

Then we can modify the file dff_sources/modules/nodes/CMakeList.txt and add the line :

add_subdirectory (filescat)

This cmake directives is used to indicate to make command that it must enter the filescat directory during the compilation.

In the directory filescat we will need to create an other file CMakeLists.txt containing :

set(catfiles_srcs
  __init__.py
  catfiles.py
)

install_file(catfiles ${catfiles_srcs})

and a last file called __init__.py containing :

__all__ = ['catfiles']

Python

Once theses two steps are done our environment is ready and we can start coding.

import

First we will have to create a catfiles.py file. In this file, we will add the following imports :

# version of the module
__dff_module_catfiles_version__ = "1.0.0"
 
from struct import unpack
 
from api.vfs import *
from api.module.module import *
from api.types.libtypes import Variant, VList, VMap, Argument, Parameter, typeId
from api.vfs.libvfs import *

class catfiles

In this class, inheriting the class Module, we will define which options the driver will need to run. In our case, the two file names we want to concatenate are passed in parameter to the module :

class catfiles(Module):
"""This module is designed to concatenate 2 files."""
  def __init__(self):
    Module.__init__(self, "catfiles", CatFiles)
 
    # module configuration. We are adding two mandatory arguments, which are the two nodes we will concatenate.
    self.conf.addArgument({"input": Argument.Required|Argument.List|typeId.Node,
                           "name": "files",
                           "description": "these files will be concatenated in the order they are provided",
                           "parameters": {"type": Parameter.Editable,
                                          "minimum": 2,
                                          "maximum": 2}
    self.tags = "Node"

Your module musts have the same name than the file. For example, if the file is called foo.py, the module has to be named Foo otherwise it won't be loaded properly.

class CatFiles

The second step is to implement the mfso methods, and so create a class inheriting mfso :

class CatFiles(mfso):
    # Constructor. we init the name of the driver. Do not forget the self.__disown()__ or 
    # python garbage collector will delete the object
    def __init__(self):
       mfso.__init__(self, "catfiles")
       self.name = "catfiles"
       self.__disown__()
 
    # the job of the module will be done in this method
    def start(self, args):
 
       # first we get the two nodes passed in parameters to the driver (a list)
       self.files = args['files'].value()
 
       self.file1 = self.files[0]
       self.file2 = self.files[1]
 
       # we concatenate the two names and calculate the total size
       name = self.file1.name() + "-" + self.file2.name()
       size = self.file1.size() + self.file2.size()
 
       # creation of the node corresponding to the concatenated two files (see the class below)
       self.cat_node = CatfilesNode(name, size, None, self, self.file1, self.file2)
 
       # the self.cat_node.__disown__() will insure that python garbage collector wont delete the object
       self.cat_node.__disown__()
 
       # and finally the registerTree
       self.registerTree(self.file1.parent(), self.cat_node)

class CatfilesNode

in this class we will have to implement the fileMapping and extendedAttributes method

class CatfilesNode(Node):
   def __init__(self, name, size, parent, mfso, file1, file2):
      Node.__init__(self, name, size, parent, mfso)
 
      # we will need file1 and file2 in the file mapping so we save them as attributes
      self.file1 = file1
      self.file2 = file2
      self.__disown__() # to avoid python garbage collector to delete the object
 
   # the fileMapping method of the node is called when the node is opened.
   # We push the content of the two nodes (2 chuncks)
   def fileMapping(self, fm):
      fm.push(0, self.file1.size(), self.file1, 0)
      fm.push(self.file1.size(), self.file2.size(), self.file2, 0)
 
   # in the extended attributes we will store the names and sizes of the  two original files
   # so DFF users can be able to see which are the "original" 2 files
   def _attributes(self):
      f1_size = Variant(self.file1.size())
      f2_size = Variant(self.file2.size())
      f1_name = Variant(self.file1.name())
      f2_name = Variant(self.file2.name())
 
      # to avoid a deletion by the garbage collector.
      attr.thisown = False
      f1_size.thisown = False
      f2_size.thisown = False
      f1_name.thisown = False
      f2_name.thisown = False
 
      # and we push our attributes in the attr vmap
      attr = VMap()
      attr.thisown = False
 
      attr["1st file name"] = f1_name
      attr["2nd file name"] = f2_name
      attr["1st file size"] =  f1_size
      attr["2nd file size"] = f2_size
 
      return attr

Conclusion

And that's it ! Nothing complicated.