VS>> linux-scheduler>> 返回
项目作者： iotool

项目描述 ：
VSLS - Very simple linux scheduler based on filesystem without installation / without database
高级语言： Shell
项目地址: git://github.com/iotool/linux-scheduler.git
创建时间： 2017-10-19T13:46:59Z
项目社区：https://github.com/iotool/linux-scheduler
开源协议：GNU Lesser General Public License v3.0
下载


VSLS - Very Simple Linux Scheduler

  • based on simple shellscript
  • no database or installation required

Scheduling

Linux Scheduler

I’ve built this scheduler for environments where you can’t install any software.

schedule your chain:

  • linux feature: cron jobs
  • workaround: while sleep loop

If you are not allowed to use cron, you can create a shellscript like master.sh that contain an endless loop. Don’t forget a sleep delay to reduce the system workload. Inside this loop you can parse system time and speficy crontab like execution rules.

start your chain:

  1. $ cd ~/scheduler/job
  2. $ . start.sh
  3. start master . /home/myuser/scheduler/job/master.sh
  4. master loop 1439 nohup /home/myuser/scheduler/job/chain.sh >/dev/null 2>&1 &
  5. master loop 1438 nohup /home/myuser/scheduler/job/chain.sh >/dev/null 2>&1 &

You can start your chain manually or by cron.

cancle your chain:

  1. $ cd ~/scheduler/job
  2. $ . cancle.sh
  3. cancle master

Use cancle to force exit the main loop of the master script. You don’t need to kill processes. With cancle you are able to terminate your chain that is running on clustered infrastructure with shared filesystems.

monitor your chain:

  1. $ cd ~/scheduler/status
  2. $ ls -l -t --time-style=+'%c '
  3. -rw------- 1 myuser user    8 Thu 19 Oct 2017 09:44:52 PM CEST  master_end
  4. -rw------- 1 myuser user   54 Thu 19 Oct 2017 09:44:52 PM CEST  master_start
  5. -rw------- 1 myuser user   43 Thu 19 Oct 2017 09:43:12 PM CEST  job3_end
  6. -rw------- 1 myuser user   45 Thu 19 Oct 2017 09:43:11 PM CEST  job3_start
  7. -rw------- 1 myuser user   44 Thu 19 Oct 2017 09:43:11 PM CEST  job2b_end
  8. -rw------- 1 myuser user   44 Thu 19 Oct 2017 09:43:09 PM CEST  job2a_end
  9. -rw------- 1 myuser user   46 Thu 19 Oct 2017 09:43:08 PM CEST  job2b_start
  10. -rw------- 1 myuser user   46 Thu 19 Oct 2017 09:43:08 PM CEST  job2a_start
  11. -rw------- 1 myuser user   43 Thu 19 Oct 2017 09:43:08 PM CEST  job1_end
  12. -rw------- 1 myuser user   45 Thu 19 Oct 2017 09:43:07 PM CEST  job1_start

The schedule is using files to controle the workflow.

Steps = program

  • single processing step
  • sequential program execution

stepx.sh:

  1. #!/bin/sh
  2. perl /home/myuser/step_x.pl

Steps are regular shellskripts that execute atomic logic of your process. You don’t have to modify this code because we use separate job script as wrappers. You don’t have to move existing shellscript into the scheduler folder.

Jobs = process

  • overall process flow
  • parallele job processing

Just copy existing job and rename jobname, specify the stepscript / dependencies and add your new jobs to chain. Modify the end script if necessary.

jobx.sh:

  1. jobname=jobx
  2. stepscript=/step/stepx.sh
  3. schedulerdir=/home/myuser/scheduler
  4. parentcount=0
  6. if [[ -e "${schedulerdir}/status/${jobname}_start" ]]; then parentcount=$(( $parentcount + 1 )); fi
  7. if [[ -e "${schedulerdir}/status/${jobname}_end" ]]; then parentcount=$(( $parentcount + 1 )); fi
  9. if [[ ! -e "${schedulerdir}/status/master_start" ]]; then parentcount=$(( $parentcount + 1 )); fi
  10. if [[ ! -e "${schedulerdir}/status/joba_end" ]]; then parentcount=$(( $parentcount + 1 )); fi
  11. if [[ ! -e "${schedulerdir}/status/jobb_end" ]]; then parentcount=$(( $parentcount + 1 )); fi
  13. # ...

Each job detect its dependencies (e.g. “joba” and “jobb”). The script is simply counting each missing parent. All jobs should run only one time (start and/or end must not exists).

chain.sh:

  1. #!/bin/sh
  2. schedulerdir=/home/myuser/scheduler
  3. nohup ${schedulerdir}/job/job1.sh  >/dev/null 2>&1 &
  4. nohup ${schedulerdir}/job/job2a.sh >/dev/null 2>&1 &
  5. nohup ${schedulerdir}/job/job2b.sh >/dev/null 2>&1 &
  6. nohup ${schedulerdir}/job/job3.sh  >/dev/null 2>&1 &
  7. nohup ${schedulerdir}/job/jobx.sh  >/dev/null 2>&1 &
  8. nohup ${schedulerdir}/job/end.sh   >/dev/null 2>&1 &

Add new jobs to the chain shellscript. There is no sort order for dependencies between jobs, because each job check its dependencies by itself.

end.sh:

  1. #!/bin/sh
  2. schedulerdir=/home/myuser/scheduler
  3. parentcount=0
  4. if [[ ! -e "${schedulerdir}/status/jobx_end" ]]; then parentcount=$(( $parentcount + 1 )); fi
  5. if [[ ! -e "${schedulerdir}/status/master_start" ]]; then parentcount=$(( $parentcount + 1 )); fi
  6. if [[ -e "${schedulerdir}/status/master_end" ]]; then parentcount=$(( $parentcount + 1 )); fi
  7. if [[ $parentcount -eq 0 ]]
  8. then
  9.   echo "end reached"
  10.   echo "ende" > ${schedulerdir}/status/master_end
  11. else
  12.   echo "end not reached"
  13. fi

End is the predefined latest job of each chain where you have to define your end dependencies. The end shellscript force the master shellscript to exit its main loop.