[Solved-3 Solutions] How to keep PIG job log file when it is successful ?
Problem:
How to keep PIG job log file when it is successful ?
Solution 1:
- By default errors are logged to pig.logfile which can be set in $PIG_HOME/conf/pig.properties. If we want to log status messages also, then we have to prepare a valid log4j properties
E.g: rename log4j.properties.template to log4j.properties in $PIG_HOME/conf and Set the following:
log4j.logger.org.apache.pig=info, B
# ***** A is set to be a ConsoleAppender.
#log4j.appender.A=org.apache.log4j.ConsoleAppender
# ***** A uses PatternLayout.
#log4j.appender.A.layout=org.apache.log4j.PatternLayout
#log4j.appender.A.layout.ConversionPattern=%-4r [%t] %-5p %c %x - %m%n
# ***** B is set to be a FileAppender.
log4j.appender.B=org.apache.log4j.FileAppender
#log4j.appender.B.File=/home/user/pig-distrib/logs/pig_success.log
log4j.appender.B.File=/home/user/pig-distrib/logs/pig.log
log4j.appender.B.layout=org.apache.log4j.PatternLayout
log4j.appender.B.layout.ConversionPattern=%-4r [%t] %-5p %c %x - %m%n
log4j.appender.B.Append=true
Solution 2:
- Once the log4j.properties file is prepared, we have to open the pig.properties file
log4jconf=$PIG_HOME/conf/log4j.properties
Solution 3:
- One of the Easy way is to redirect output
pig -f pig-file 2> xyz.log -- this will give just the logs