Hi,
I'd like to create a dynamic cmd-action.  I attempted to do this using
the environment setup in the following example.
After 'doit ffmpeg' the new.flv file should contain:
'ffmpleg -doitexample 20'

#dodo.py

import re

origflv='orig.flv'
origxml='orig.xml'
newflv="new.flv"

class MetaData():
    def __init__(self):

        self.fps=-1

    def setup(self):
        try:
            fh=open(origxml,"r")
            self.meta=fh.read()
            fh.close()
        except IOError:
            #this is run before the task can generate the xml file
            #in that case just exit
            print "file does note exist"
            return None

        fps_re=re.compile(r"""<framerate>(.*)</framerate>""")
        fps_reobj=fps_re.search(self.meta)
        if fps_reobj:
            self.fps=fps_reobj.group(1)

        print self.fps

metaA=MetaData()

def task_metaxml():
    "generate the metadata xml file for post-processing (get framerate
to determine ffmpeg params for encoding)"
    #cmd="""yamdi -i {vid} -x {xml}""".format(vid=origflv,xml=origxml)
    cmd="""echo "<framerate>20</framerate>" > {xml}""".format
(xml=origxml)
    return {'actions': [cmd],
            'targets': [origxml],
            'clean':True,
            }
            #'dependencies':[origflv],

def task_ffmpeg():
    cmd="echo ffmpeg -doitexample {fps} > {out}".format
(fps=metaA.fps,out=newflv)
    return {'actions': [cmd,],
            'targets': [newflv],
            'dependencies':[origxml],
            'clean':True,
            'setup':[metaA]
            }
            #'dependencies':[origflv,origxml],

Hi,

No. environment setup is not supposed to be used for that.

what's wrong
===========

Here is an explanation why your code does NOT work. I notice many
people get this wrong and I was already planning to improve the docs
about this.

"def task_xxx()"... is not the task itself. it is a "task generator".
the task is the "dictionary returned by the task generator. The task
generators are executed when dodo.py is loaded. So

 cmd="echo ffmpeg -doitexample {fps} > {out}".format(fps=metaA.fps,out=newflv)

will be executed before metaA.setup

how to solve your problem
=================

As I understood... you are trying to use the result of a computed
value in task. So lets say the Metada.setup should be a task =>
task_get_metadata.

So we want to be able to use a value calculated in a task in another
task. This is not really supported on doit right now. Lets take a look
in the options we have (starting from a hack to a feature proposal in
doit):

1 - use subprocess
------------------------
if task_get_metadata is executed only when you call task_ffmpeg, or it
is not very expensive operation. You could combine everything in one
python-task:

   fps = get_fps(origxml)
   subprocess.call(["ffmpeg", "-doitexample", fps] ...

this is far from ideal but can solve your problem right now.

2- save computed value in a file
------------------------------------------
you could save the value of task_get_metadata in a file and then read
it on task_ffmepg. actually the reason i created doit was to avoid
this kind of file manipulations...

3- make doit save computed values from task
------------------------------------------------------------
actually I've done some work towards this... doit saves a dictionary
with some info for every task. so adding and saving some data on it
should be quite easy. what i still didnt figure out is how the
interface should be.

Please tell what you think about this...

task_get_metada would return a dictionary like
   return {'fps':20}
, doit would save this...

a cmd-action could access this using string formatting. i.e.
def task_ffmpeg():
def task_ffmpeg():
   cmd="echo ffmpeg -doitexample %(fps)s"
   return {'actions': [cmd,],
           'values': {'fps': ':get_metadata.fps'} # ===> tells doit to
get value saved on task_get_metadata
           'targets': [newflv],
           'dependencies':[origxml],
           'clean':True,
           }

for python-action you would access this values through function patameters like

def task_ffmpeg():
   def myaction(fps):  print pfs
   return {'actions': [(myaction,),],
           'values': {'fps': ':get_metadata.fps'} # ===> tells doit to
get value saved on task_get_metadata
           'targets': [newflv],
           'dependencies':[origxml],
           'clean':True,
           }

doit would take care of passing the 'values' dictionary  to the
python-action function.

what do you think?

cheers,
  eduardo

Thanks for your explanation.  I'm now using the subprocess workaround,
but would prefer something similar to your proposal (3) to pass
variable values to the cmd-action.

Some comments on proposal 3
---------------------------

- Passing a large number of parameters

  'values':{'fps':':get_metadata.fps','A':':get_metadata.A',
        'B':':get_metadata.B',...}

   How about a method of passing all the values from the task
(get_metadata) without explicitly defining them?

- Passing values from python-action to cmd-action within same task

  It seems the order of execution can be changed depending on the
actions definition.

  'actions': [(python_action),cmd],
  'actions': [cmd,(python_action)],

  Perhaps the python_action could compute the result value and pass
values to the cmd-action.

Thanks,
LJN

thanks for the feedback.

FYI I am going to try to push doit 0.5 by the end of this month
but I am quite busy  and I guess this wont make into it 0.5... But I
will definitely look into it at
some point... or you could try to take a look in it by yourself. I
guess it wont be that hard.

cheers

On Thu, Nov 12, 2009 at 6:24 PM, Eduardo Schettino

FYI I've created a bug report for this:
https://bugs.launchpad.net/doit/+bug/486569

I finally found some time to implement this. Now you can use
saved/computed values from another task. Here is a trivial example

--------------------------------

DEFAULT_TASKS = ['use']

def task_compute():
    def comp():
        return {'x':5,'y':10, 'z': 20}
    return {'actions': [(comp,)]}

def task_use():
    return {'actions': ['echo %(x)s %(z)s - 5 20'],
            'taskargs': {'x':'compute.x',
                         'z':'compute.z'},
            'verbosity': 2,
            }
---------------------------------

you need latest trunk to test this. I used the name "taskargs" though
i dont like it. suggestions are welcome.

the value can anything that can be encoded in json. if you need many
values just put them in a list or dictionary

the actions are always executed in the order they are passed. passing
values within the same task is not supported up to now...

cheers,
  eduardo

On Fri, 15 Jan 2010 00:37 -0200, "Eduardo Schettino"

Great - I converted my task to the new method.

taskargs
getargs ?

With the subprocess workaround I built the shell command within the
python-action.  The shell cmd was several lines and used .format() to
substitute many variables.  In converting the task I had to move the
shell cmd definition to within the task return statement.

based on your example:
-----------------------

DEFAULT_TASKS = ['use']

constantvar1='1'

def task_compute():
    def comp():
        return {'x':5,'y':10, 'z': 20}
    return {'actions': [(comp,)]}

def task_use():
    return {'actions': ['echo %(x)s %(z)s {c1} - 5 20
    1'.format(c1=constantvar1)],
            'taskargs': {'x':'compute.x',
                         'z':'compute.z'},
            'verbosity': 2,
            }
-----------------------

The return line is awkward in that it uses .format() as well as the %
substitution.  
    return {'actions': ['echo %(x)s %(z)s {c1} - 5 20
    1'.format(c1=constantvar1)],

Of course I could have passed all the variables (constantvar1) from the
compute task.

I tried another variation:
-----------------------

DEFAULT_TASKS = ['use']

constantvar1='1'

def task_compute1():
    def comp():
        return {'x':5,'y':10, 'z': 20}
    return {'actions': [(comp,)]}

def task_compute2():
    def buildcmd(x,z):
        cmd="echo {x} {z} {c1} - 5 20 1".format(x=x,z=z,c1=constantvar1)
        return {'cmd':cmd}

    return {'actions': [(buildcmd)],
            'taskargs': {'x':'compute1.x',
                         'z':'compute1.z'},
            'verbosity': 2,
            }

def task_use():
    return {'actions': ['%(cmd)s'],
            'taskargs': {'cmd':'compute2.cmd'},
            'verbosity': 2,
            }

# combining tasks compute2 and use
# the following would require support for passing from python-action to
cmd-action
#- build the shell command outside the return statement
#- group the python function that builds the shell command in the same
task as the cmd-action

#def task_use()
#    def buildcmd(x,z):
#        cmd="echo {x} {z} {c1} - 5 20
1".format(x=x,z=z,c1=constantvar1)
#        return {'cmd':cmd}
#    return {'actions': [(buildcmd),'%(cmd)s'],
#      ???   'taskargs': {'cmd':'this.cmd'},
#            'verbosity': 2,
#            }

-----------------------

Thanks for the feature.  

LJN

i guess i would use

double %
-------------------

DEFAULT_TASKS = ['use']

constantvar1='1'

def task_compute():
   def comp():
       return {'x':5,'y':10, 'z': 20}
   return {'actions': [(comp,)]}

def task_use():
   return {'actions': ['echo %%(x)s %%(z)s %(c1)s - 5 20 1' %
                       {'c1':constantvar1}],
           'taskargs': {'x':'compute.x',
                        'z':'compute.z'},
           'verbosity': 2,
           }

---------------------------------------------

or

create substring for each command option
----------------------------------------------

DEFAULT_TASKS = ['use']

constantvar1='1'

def task_compute():
    def comp():
        return {'x':5,'y':10, 'z': 20}
    return {'actions': [(comp,)]}

def task_use():
    opt_x = '%(x)s '
    opt_z = '%(z)s '
    opt_c = '%s ' % constantvar1
    args = '- 5 20 1'
    return {'actions': ['echo ' + opt_x + opt_z + opt_c + args],
            'taskargs': {'x':'compute.x',
                         'z':'compute.z'},
            'verbosity': 2,
            }

you could also combine taskargs from 2 different tasks:

DEFAULT_TASKS = ['use']

constantvar1='1'

def task_constants():
    def consts():
        return {'c1': constantvar1}
    return {'actions': [(consts,)]}

def task_compute():
    def comp():
        return {'x':5,'y':10, 'z': 20}
    return {'actions': [(comp,)]}

def task_use():
    return {'actions': ['echo %(x)s %(z)s %(c1)s - 5 20 1'],
            'taskargs': {'x':'compute.x',
                         'z':'compute.z',
                         'c1':'constants.c1'},
            'verbosity': 2,
            }

the implementation of "this.something" would be actually completely
different from using taskargs from other tasks. taskargs values are
saved only after all task actions are completed successfully.

I guess this feature 'this.something' is not strictly necessary for
any use-case (as far as i can see, but i might be wrong). i am trying
to avoid adding features to doit that would increase its complexity
and brings little gain in functionality.

cheers,
  eduardo