[Solved-3 Solutions] How to keep PIG job log file when it is successful ?




Problem:

How to keep PIG job log file when it is successful ?

Solution 1:

  • By default errors are logged to pig.logfile which can be set in $PIG_HOME/conf/pig.properties. If we want to log status messages also, then we have to prepare a valid log4j properties

E.g: rename log4j.properties.template to log4j.properties in $PIG_HOME/conf and Set the following:

log4j.logger.org.apache.pig=info, B
# ***** A is set to be a ConsoleAppender.
#log4j.appender.A=org.apache.log4j.ConsoleAppender
# ***** A uses PatternLayout.
#log4j.appender.A.layout=org.apache.log4j.PatternLayout
#log4j.appender.A.layout.ConversionPattern=%-4r [%t] %-5p %c %x - %m%n
# ***** B is set to be a FileAppender.
log4j.appender.B=org.apache.log4j.FileAppender
#log4j.appender.B.File=/home/user/pig-distrib/logs/pig_success.log
log4j.appender.B.File=/home/user/pig-distrib/logs/pig.log
log4j.appender.B.layout=org.apache.log4j.PatternLayout
log4j.appender.B.layout.ConversionPattern=%-4r [%t] %-5p %c %x - %m%n
log4j.appender.B.Append=true

Solution 2:

  • Once the log4j.properties file is prepared, we have to open the pig.properties file
log4jconf=$PIG_HOME/conf/log4j.properties

Solution 3:

  • One of the Easy way is to redirect output
pig -f pig-file 2> xyz.log  -- this will give just the logs

Related Searches to How to keep PIG job log file when it is successful.