mlflow-demo

Simple Demo of MLflow Project with LightGBM

Data

Adult Data Set from UCI Machine Learning Repository

Variable	Explanation
age	continuous
workclass	Private, Self-emp-not-inc, Self-emp-inc, Federal-gov, Local-gov, State-gov, Without-pay, Never-worked
fnlwgt	continuous
education	Bachelors, Some-college, 11th, HS-grad, Prof-school, Assoc-acdm, Assoc-voc, 9th, 7th-8th, 12th, Masters, 1st-4th, 10th, Doctorate, 5th-6th, Preschool
education-num	continuous
marital-status	Married-civ-spouse, Divorced, Never-married, Separated, Widowed, Married-spouse-absent, Married-AF-spouse
occupation	Tech-support, Craft-repair, Other-service, Sales, Exec-managerial, Prof-specialty, Handlers-cleaners, Machine-op-inspct, Adm-clerical, Farming-fishing, Transport-moving, Priv-house-serv, Protective-serv, Armed-Forces
relationship	Wife, Own-child, Husband, Not-in-family, Other-relative, Unmarried
race	White, Asian-Pac-Islander, Amer-Indian-Eskimo, Other, Black
sex	Female, Male
capital-gain	continuous
capital-loss	continuous
hours-per-week	continuous
native-country	United-States, Cambodia, England, Puerto-Rico, Canada, Germany, Outlying-US(Guam-USVI-etc), India, Japan, Greece, South, China, Cuba, Iran, Honduras, Philippines, Italy, Poland, Jamaica, Vietnam, Mexico, Portugal, Ireland, France, Dominican-Republic, Laos, Ecuador, Taiwan, Haiti, Columbia, Hungary, Guatemala, Nicaragua, Scotland, Thailand, Yugoslavia, El-Salvador, Trinadad&Tobago, Peru, Hong, Holand-Netherlands
target	>50K, <=50K

Feature Engineering

Since this demo project doesn’t focus on machine learning algorithms, so only simple feature engineering steps are implemented. More specifically,

target: transform ‘>50K’ as 1 and ‘<=50K’ as 0
fill missing categorical value with ‘Missing’ and apply label-encoder
fill missing numerical value with median value from training set
since the data set is imbalanced, AUC is used as the metric

Modeling

Gradient Boosting model from LigthGBM are implemented. Check the documentation for details.

How to run the code

Method 1:

$ git clone https://github.com/JifuZhao/mlflow-demo.git
$ mlflow run mlflow-demo \
  -P train_path=absolute path/mlflow-demo/data/train.csv \
  -P test_path=absolute path/mlflow-demo/data/test.csv \
  -P num_boost_round=1000 -P learning_rate=0.1 -P num_leaves=31 \
  -P max_depth=-1 -P min_data_in_leaf=20

Note: You need to change the file path to be your local absolute path.

After the code successfully runs, you can get the following results:

Method 2:

$ git clone https://github.com/JifuZhao/mlflow-demo.git
$ cd mlflow-demo/
$ python train.py ./data/train.csv ./data/test.csv 1000 0.1 31 -1 20

After running, you can view the result through MLflow UI:

$ mlflow ui

In the browser, go to http://127.0.0.1:5000, you will get the following results.

mlflow-demo

Data

Feature Engineering

Modeling

How to run the code

Example MLflow projects: