项目作者: yorodm

项目描述 :
Plumber is a Python ETL framework
高级语言: Python
项目地址: git://github.com/yorodm/plumber.git
创建时间: 2019-02-20T19:13:53Z
项目社区:https://github.com/yorodm/plumber

开源协议:MIT License

下载


Plumber

Build Status

Plumber is a simple Python ETL framework for Python3

TL;DR: A code sample

  1. from plumber import pipe # API síncrona.
  2. @pipe.extractor
  3. def read_file():
  4. file_name = os.environ['FILENAME']
  5. with open(file_name) as f:
  6. for x in f.readlines():
  7. yield process_line(x)
  8. @pipe.transformer
  9. def csvfy(element):
  10. yield ','.join(map(str,element))
  11. class SaveData(pipe.Writer):
  12. def __init__(self, filename):
  13. self.filename = filename
  14. def setup(self):
  15. self._file = open(f,'w')
  16. def cleanup(self):
  17. self._file.close()
  18. def write(x):
  19. self._file.write(x)
  20. tuberia = pipe.Pipe(
  21. read_file,
  22. csvfy,
  23. SaveData("prueba.csv")
  24. )
  25. tuberia.run()