hadoop - Getting YARN action application ID in subsequent action -
i running oozie workflow , doing map distributed model fitting within map-reduce action. there many mappers, have written code compiles yarn logs of mapper tasks using yarn logs -applicationid application_x
application_x parent application id of map tasks. want make summarization part of workflow need application_x
dynamically application id of previous action. there way can this?
i have not tested this, think can workflow el function:
wf:actionexternalid(string node) returns external id action node, or empty string if action has not being executed or has not completed yet.
so in node after map reduce job has completed, should able use likel
wf:actionexternalid('mapred-node-name')
i suspect return job_xxx instead of application_xxx, can handle ok.
Comments
Post a Comment