Issue
I am running a program with the hadoop jar
command. However, to make that program run faster, I need to increase Hadoop's heap size. I tried the following, but it didn't have any effect (I have hadoop version 2.5), even on the same terminal.
export HADOOP_HEAPSIZE=16384
export HADOOP_OPTS="-Xmx16g"
Another way I reckon is to add the following to the mapred-site.xml file, but I am unfortunately not the admin, so can't do it.
<property>
<name>mapred.child.java.opts</name>
<value>-Xmx16384m</value>
</property>
Is there any other method as well to do this?
Solution
I solved this problem by modifying the HADOOP_CLIENT_OPTS environment variable, as shown below.
export HADOOP_CLIENT_OPTS="-Xmx16g $HADOOP_CLIENT_OPTS"
Note that the program I am using runs only on the Master node. In other words, its not a distributed application.
Answered By - pythonic
0 comments:
Post a Comment
Note: Only a member of this blog may post a comment.