Installing HDFS on FreeBSD Latest

In this tutorial, we will go through the steps of installing HDFS (Hadoop Distributed File System) on the FreeBSD operating system. We will be downloading HDFS from the official website at http://hadoop.apache.org/ and installing it on a FreeBSD Latest version.

Prerequisites

  • A FreeBSD Latest instance with root access.
  • Java 8 or higher installed on the system.

Steps

Follow the steps below to install HDFS on FreeBSD:

  1. Install the required dependencies:

    # pkg install bash openjdk8 maven ant

  2. Download and extract the Hadoop package:

    # cd /usr/local
# wget https://downloads.apache.org/hadoop/common/hadoop-3.3.1/hadoop-3.3.1.tar.gz
# tar -xzvf hadoop-3.3.1.tar.gz
# ln -s hadoop-3.3.1 hadoop

  3. Set the Hadoop environment variables by creating a new file named hadoop-env.sh in the /usr/local/hadoop/etc/hadoop directory:

    # cd /usr/local/hadoop/etc/hadoop
# cp hadoop-env.sh.template hadoop-env.sh
# vi hadoop-env.sh

    Add the following lines to the file:

    export JAVA_HOME=/usr/local/openjdk8
export HADOOP_HOME=/usr/local/hadoop
export HADOOP_CONF_DIR=${HADOOP_HOME}/etc/hadoop

    Save and exit the file.

  4. Update the Hadoop configuration files:

    # vi core-site.xml

    Add the following lines to the file:

    <configuration>
    <property>
        <name>fs.defaultFS</name>
        <value>hdfs://localhost:9000</value>
    </property>
</configuration>

    Save and exit the file.

    # vi hdfs-site.xml

    Add the following lines to the file:

    <configuration>
    <property>
        <name>dfs.replication</name>
        <value>1</value>
    </property>
    <property>
        <name>dfs.namenode.name.dir</name>
        <value>/usr/local/hadoop_data/hdfs/namenode</value>
    </property>
    <property>
        <name>dfs.datanode.data.dir</name>
        <value>/usr/local/hadoop_data/hdfs/datanode</value>
    </property>
</configuration>

    Save and exit the file.

  5. Create the Hadoop data directories:

    # mkdir -p /usr/local/hadoop_data/hdfs/namenode
# mkdir -p /usr/local/hadoop_data/hdfs/datanode

  6. Format the HDFS filesystem:

    # /usr/local/hadoop/bin/hdfs namenode -format

  7. Start the HDFS filesystem and the related services:

    # /usr/local/hadoop/sbin/start-dfs.sh

    This will start the following HDFS services:

    • NameNode (on port 9000)
    • DataNode

    To stop these services, use the following command:

    # /usr/local/hadoop/sbin/stop-dfs.sh

Conclusion

In this tutorial, we have installed HDFS on FreeBSD Latest and started the filesystem and related services. You should now be able to use HDFS for storing and processing large amounts of data.