项目作者: Conradyen

项目描述 :
A implementation of Hadoop map reduce, map side join, reducer side join on sample data set.
高级语言: Java
项目地址: git://github.com/Conradyen/Hadoop-MapReduce.git
创建时间: 2020-03-09T16:27:37Z
项目社区:https://github.com/Conradyen/Hadoop-MapReduce

开源协议:

下载


Hadoop-MapRexuce

commonFriend.java

Take input in the format of

  1. <User><TAB><Friends>

output mutul foriend for the rolloring userID pairs.
( 0,1 )( 20,28193 )( 1,29826 )( 6222,19272 )( 28041,28056 )

TopTenCommonFriend.java

Take input in the format of

  1. <User><TAB><Friends>

output 10 user pairs thant has the most common friends.

CommonFriendDOB.java

Take input in the format of

  1. <User><TAB><Friends>

And user data int the format of

  1. column1 : userid
  2. column2 : firstname
  3. column3 : lastname
  4. column4 : address
  5. column5: city
  6. column6 :state
  7. column7 : zipcode
  8. column8 :country
  9. column9 :username
  10. column10 : date of birth.

output of name and birthday of common friend of a user pair.

MaxAge.java

Take input in the format of

  1. <User><TAB><Friends>

And user data int the format of

  1. column1 : userid
  2. column2 : firstname
  3. column3 : lastname
  4. column4 : address
  5. column5: city
  6. column6 :state
  7. column7 : zipcode
  8. column8 :country
  9. column9 :username
  10. column10 : date of birth.

And do the following.

Step 1: Calculate the maximum age of the direct friends of each user.

Step 2: Sort the users based on the calculated maximum age in descending order as described in step 1.

Step 3. Output the top 10 users from step 2 with their address and the calculated maximum age.

To Run

Change all file path in run.sh and cleanup.sh

  1. chmod u+x run.sh
  2. ./run.sh

cleanup.sh

cleanup.sh will delete all output files.

  1. chmod u+x cleanup.sh
  2. ./cleanup.sh
  • Note that all file are only tested on single node.