Docker-hadoop

This project contains Dockerfiles for setting up a basic hadoop cluster.

Images

The project contains hadoop modules in corresponding directories:

Basic Hadoop image:

• hadoop (All other images are built base on this image.)

HDFS images:

• namenode
• datanode

YARN images:

• resourcemanager
• nodemanager

Mapreduce images:

• mapred
• historyserver

Hadoop all-in-one image:

• single

Spark image:

• spark

Build images

Run command:

./build-all.sh

Hadoop Configuration:

Set container’s hadoop configuration by setting envirnment HADOOP_CONF_DATA

Example docker-compose.yml

There is a docker-compose.yml file provided to run demo examples.

Run with:

docker-compose up

Destory with

docker-compose down

Web ui