hadoop - Getting YARN action application ID in subsequent action -


i running oozie workflow , doing map distributed model fitting within map-reduce action. there many mappers, have written code compiles yarn logs of mapper tasks using yarn logs -applicationid application_x application_x parent application id of map tasks. want make summarization part of workflow need application_x dynamically application id of previous action. there way can this?

i have not tested this, think can workflow el function:

wf:actionexternalid(string node)  returns external id action node, or empty string if action has not being executed or has not completed yet. 

so in node after map reduce job has completed, should able use likel

wf:actionexternalid('mapred-node-name') 

i suspect return job_xxx instead of application_xxx, can handle ok.


Comments

Popular posts from this blog

java - Jasper subreport showing only one entry from the JSON data source when embedded in the Title band -

serialization - Convert Any type in scala to Array[Byte] and back -

SonarQube Plugin for Jenkins does not find SonarQube Scanner executable -