Oozie <job-xml> 无法在具有子工作流操作的全局部分中工作

Question

我正在尝试缩短我的 oozie 工作流程，我指的是这个 cloudera 帖子。 http://blog.cloudera.com/blog/2013/11/how-to-shorten-your-oozie-workflow-definitions/

根据这篇文章，我们可以在全局部分添加 job.xml 文件，但它对我不起作用。

   <global>
   <job-tracker>${jobTracker}</job-tracker>
   <name-node>${nameNode}</name-node>
   <job-xml>job1.xml</job-xml>
   <configuration>
        <property>
            <name>mapred.job.queue.name</name>
            <value>${queueName}</value>
        </property>
    </configuration>
</global>

------------主工作流程---------- --------

<?xml version="1.0" encoding="UTF-8"?>
<workflow-app xmlns="uri:oozie:workflow:0.4" name="main-wf">
   <global>
    <job-xml>/user/${wf:user()}/${examplesRoot}/apps/map-reduce/job.xml</job-xml>
   </global>
    <start to="main-node"/>
    <action name="main-node">
      <sub-workflow>
         <app-path>/user/${wf:user()}/${examplesRoot}/apps/map-reduce/workflow.xml</app-path>
      </sub-workflow>
      <ok to="end" />
      <error to="fail" />
   </action>
    <kill name="fail">
        <message>MR failed, error message[${wf:errorMessage(wf:lastErrorNode())}]</message>
    </kill>
    <end name="end"/>
</workflow-app>

-----------子工作流程---------- --------

<?xml version="1.0" encoding="UTF-8"?>
<workflow-app xmlns="uri:oozie:workflow:0.4" name="OozieBDU-wf">
    <start to="wordcount-node"/>
    <action name="wordcount-node">
        <map-reduce>
            <job-tracker>${jobTracker}</job-tracker>
            <name-node>${nameNode}</name-node>
        <prepare>
         <delete path="${nameNode}/user/${wf:user()}/${examplesRoot}/output-data/${outputDir}"/>
        </prepare>
            <configuration>
                <property>
                    <name>mapred.job.queue.name</name>
                    <value>${queueName}</value>
                </property>
        <property>
               <name>mapred.mapper.new-api</name>
               <value>true</value>
        </property>
        <property>
               <name>mapred.reducer.new-api</name>
               <value>true</value>
             </property>
        <property>
               <name>mapreduce.map.class</name>
               <value>com.yumecorp.WordCount$TokenizerMapper</value>
        </property>
        <property>
               <name>mapreduce.reduce.class</name>
               <value>com.yumecorp.WordCount$IntSumReducer</value>
        </property>
             <property>
               <name>mapred.output.key.class</name>
               <value>org.apache.hadoop.io.Text</value>
             </property>
             <property>
               <name>mapred.output.value.class</name>
               <value>org.apache.hadoop.io.IntWritable</value>
             </property>
             <property>
               <name>mapred.input.dir</name>
               <value>/user/${wf:user()}/${examplesRoot}/input-data/decodeAndProfile</value>
             </property>
             <property>
               <name>mapred.output.dir</name>
               <value>/user/${wf:user()}/${examplesRoot}/output-data/${outputDir}</value>
             </property>
             <property>
               <name>mapreduce.job.outputformat.class</name>
               <value>org.apache.hadoop.mapreduce.lib.output.SequenceFileOutputFormat</value>
             </property>
             <property>
               <name>mapred.mapoutput.key.class</name>
               <value>org.apache.hadoop.io.Text</value>
             </property>
             <property>
               <name>mapred.mapoutput.value.class</name>
               <value>org.apache.hadoop.io.IntWritable</value>
             </property>
            </configuration>
        </map-reduce>
        <ok to="end"/>
        <error to="fail"/>
    </action>
    <kill name="fail">
        <message>MR failed, error message[${wf:errorMessage(wf:lastErrorNode())}]</message>
    </kill>
    <end name="end"/>
</workflow-app>

-----------job.xml---------- ---------

<configuration>
            <property>
               <name>jobTracker</name>
               <value>pari.hhh.com:8021</value>
            </property>
            <property>
               <name>nameNode</name>
               <value>hdfs://pari.hhh.com:8020</value>
            </property>
            <property>
               <name>queueName</name>
               <value>default</value>
            </property>
            <property>
               <name>examplesRoot</name>
               <value>wordcount</value>
            </property>
            <property>
               <name>outputDir</name>
               <value>map-reduce</value>
            </property>    
</configuration>

-----------job.properties------ --------- examplesRoot=wordcount oozie.wf.application.path=${nameNode}/user/${user.name}/${examplesRoot}/apps/map-reduce/mainworkflow.xml

-----------oozie 命令 ---------- -------- oozie job -config job.properties -run

-----------错误----------- -------------- OB[0000012-150131091133585-oozie-oozi-W] ACTION[0000012-150131091133585-oozie-oozi-W@wordcount-node] ActionStartXCommand 中的 ELException javax.servlet.jsp.el.E​​LException：无法解析变量 [jobTracker] 在 org.apache.oozie.util.ELEvaluator$Context.resolveVariable(ELEvaluator.java:106) 在 org.apache.commons.el.NamedValue.evaluate(NamedValue.java:124) 在 org.apache.commons.el.E​​xpressionString.evaluate(ExpressionString.java:114) 在 org.apache.commons.el.E​​xpressionEvaluatorImpl.evaluate(ExpressionEvaluatorImpl.java:274) 在 org.apache.commons.el.E​​xpressionEvaluatorImpl.evaluate(ExpressionEvaluatorImpl.java:190) 在 org.apache.oozie.util.ELEvaluator.evaluate(ELEvaluator.java:203) 在 org.apache.oozie.command.wf.ActionStartXCommand.execute（ActionStartXCommand.java:188） 在 org.apache.oozie.command.wf.ActionStartXCommand.execute(ActionStartXCommand.java:63) 在 org.apache.oozie.command.XCommand.call(XCommand.java:281) 在 org.apache.oozie.service.CallableQueueService$CompositeCallable.call(CallableQueueService.java:323) 在 org.apache.oozie.service.CallableQueueService$CompositeCallable.call(CallableQueueService.java:252) 在 org.apache.oozie.service.CallableQueueService$CallableWrapper.run(CallableQueueService.java:174) 在 java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) 在 java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) 在 java.lang.Thread.run(Thread.java:745)

有人可以帮忙吗？？

干杯 帕里

Answer 1

要将属性从工作流传递到其子工作流，您需要将<propagate-configuration/> 标记添加到<sub-workflow> 操作。

例如：

<action name="main-node">
  <sub-workflow>
     <app-path>/user/${wf:user()}/${examplesRoot}/apps/map-reduce/workflow.xml</app-path>
     <propagate-configuration/>
  </sub-workflow>
  <ok to="end" />
  <error to="fail" />
</action>