存储>> d_boutique>> 返回
项目作者： mrifat

项目描述 ：
A distributed in memory data store
高级语言： Elixir
项目地址: git://github.com/mrifat/d_boutique.git
创建时间： 2019-12-22T00:19:25Z
项目社区：https://github.com/mrifat/d_boutique
开源协议：
下载


D-Boutique

D-Boutique is a distributed key value store built in Elixir, it uses Erlang’s built-in term storage.

D-Boutique was built as an attempt to understand how distributed systems works and to learn more about Elixir and Erlang ecosystem.

D-Boutique is an umbrella project consisting of two supervised applications.

Getting Started

Setup your machine with Erlang 22.2 and Elixir 1.9.4.

The easiest way to do that is to use a universal language version manager, i.e: asdf.

OSX

  1. $> brew install asdf
  3. $> echo -e "\n. $(brew --prefix asdf)/asdf.sh" >> ~/.bash_profile
  4. $> echo -e "\n. $(brew --prefix asdf)/etc/bash_completion.d/asdf.bash" >> ~/.bash_profile

Erlang

You may encounter an SSL error, in that case please refer to asdf-erlang for possible solution.

  1. $> asdf plugin-add erlang https://github.com/asdf-vm/asdf-erlang.git
  3. $> brew install autoconf
  4. $> asdf install erlang 22.2

Elixir

  1. $> asdf plugin-add elixir https://github.com/asdf-vm/asdf-elixir.git
  2. $> asdf install elixir 1.9.4

Linux

  1. $> git clone https://github.com/asdf-vm/asdf.git ~/.asdf
  3. $> cd ~/.asdf
  4. $> git checkout "$(git describe --abbrev=0 --tags)"
  6. $> echo -e '\n. $HOME/.asdf/asdf.sh' >> ~/.bashrc
  7. $> echo -e '\n. $HOME/.asdf/completions/asdf.bash' >> ~/.bashrc

Erlang

Assumes you want a full installation of support packages, please refer to asdf-erlang for more details.

  1. $> apt-get -y install build-essential autoconf m4 libncurses5-dev libwxgtk3.0-dev libgl1-mesa-dev libglu1-mesa-dev libpng-dev libssh-dev unixodbc-dev xsltproc fop
  2. $> asdf install erlang 22.2

Elixir

  1. $> apt-get -y install unzip
  2. $> asdf plugin-add elixir https://github.com/asdf-vm/asdf-elixir.git
  3. $> asdf install elixir 1.9.4

Development Configuration

At its current state D-Boutique identifies the cluster by the computer name so you will want to change the cluster_id in config/config.exs to match your computer name.

  1. config :boutique,
  2.   cluster_id: "YOUR_COMPUTER_NAME",
  3.   port: "THE_PORT_NUMBER_YOU_WANT_TO_USE"

Running the Project

Getting the dependencies and running the tests.

  1. # Get the dependencies.
  2. $> mix deps.get
  4. # Run tests
  5. $> mix test

To run the distributed tests you need to make sure that you have separate nodes started, each node is responsible for a range of data stores based on their
bucket names, in the test environment the nodes are divided as:

a..g node

h..n node

o..u node

v..z node

  1. # cd into boutique app
  2. $> cd apps/boutique
  4. # start the first node
  5. $> iex --sname a-g -S mix
  7. # start the second node
  8. $> iex --sname h-n -S mix
  10. # start the third node
  11. $> iex --sname o-u -S mix
  13. # start the fourth node from the umbrella root to start the TCP server
  14. $> cd ../../
  15. $> iex --sname v-z -S mix test --only distributed

If you want to run the project on different computers,
make sure the computers are on the same network and have the same ~/.erlang.cookie value.

Check Erlang’s epmd for more information.

Running the project in a cluster:

  1. # cd into boutique app
  2. $> cd apps/boutique
  4. # start the first node
  5. $> iex --sname a-g -S mix
  7. # start the second node
  8. $> iex --sname h-n -S mix
  10. # start the third node
  11. $> iex --sname o-u -S mix
  13. # start the fourth node from the umbrella root to start the TCP server
  14. $> cd ../../
  15. $> iex --sname v-z -S mix
  16. 14:27:57.773 [info]  Accepting connections on port 4040

Running the project in a single node, from the umbrella root, run:

  1. $> iex -S mix
  2. 14:27:57.773 [info]  Accepting connections on port 4040

Making requests

  1. # Use telent or any similar application protocol to connect over the TCP server
  2. # feel free to have multiple telnet sessions to play around with.
  3. $> telnet 127.0.0.1 4040 # or the port you have in your config.
  4. $> CREATE shopping
  5. OK
  7. $> PUT shopping milk 3
  8. OK
  10. $> GET shopping milk
  11. 3
  12. OK
  14. $> DELETE shopping milk
  15. OK
  17. $> ^]
  18. telnet> quit
  19. Connection closed.

Documentation

To generate the documentation for the projects run:

  1. $> mix docs

Documentation files are generated in docs/, open docs/index.html to read the API reference, or any module of interest.

Projects

Boutique

This project starts a supervision tree that handles creating and updating the state of data stores using Elixir Agents and GenServers.

Boutique Server

A TCP Server, listens to a port until the port is available and get hold of the socket.
Once a client connection is established on the port it accepts it and proceeds to read the client requests and writes responses back.

Notes

This project was created for educational purposes only, I created it to learn more about distributed systems, the Elixir process life cycle, concurrency, and state mutation.

Check out Elixir’s Introduction to Mix for more information.

Due the beam VM and OTP implementation, all nodes on the network can access any process in any other node on the cluster by default which is a security concern,
in general communication between nodes should happen using SSL or any kind of encrypted messaging you’d like to use.
Erlang provides a powerful :rpc module that can be used for secure communication between nodes of a cluster. Currently this project does not use the :rpc module.

Useful Resources: