项目作者: tboeghk

项目描述 :
Experiments with Solr autoscaling
高级语言: HCL
项目地址: git://github.com/tboeghk/solr-aws-autoscaling.git
创建时间: 2020-02-12T11:49:24Z
项目社区:https://github.com/tboeghk/solr-aws-autoscaling

开源协议:MIT License

下载


🧪 Solr AWS observability & autoscaling experiments

This repo holds a playground for Solr Cloud observability & autoscaling
experiments with AWS and Terraform. It’s idea is to quickly spin up a
Zookeeper/Solr ensemble and evaluate SolrCloud autoscaling triggers in conjunction with AWS autoscaling group events.

The Solr Autoscaling Framework is very complicated and seems pretty
overengineered. That’s why I use this playground to test Solr
Autoscaling policies.

🚨 The autoscaling framework in its current form is deprecated
and will be removed in Solr 9.0.

Project goal

Use this project as a blueprint to:

  • scale Solr autoscaling groups at speed (~90s up and running)
  • utilize Cloud-Init and SystemD to properly launch and terminate
    Solr instances.
  • See Prometheus metrics and alarms in action
  • See Jaeger distributed tracing in action
  • Experiment with Solr autoscaling settings

Dependencies

Before you start, provide your AWS access and secret key
to Terraform either static in a secrets.auto.tfvars file or in environment
variables.

  1. cat << EOF > secrets.auto.tfvars
  2. aws_access_key = "YOUR_AWS_ACCESS_KEY"
  3. aws_secret_key = "YOUR_AWS_SECRET_KEY"
  4. aws_region = "eu-west-1"
  5. EOF

Up and running

💰 You are about to create resources in AWS that actually
cost money. Just be aware of that when scaling your cluster
beyond infinity …

  1. # create VPC and basic security groups
  2. cd tf-workspaces/vpc && tf init && tf apply
  3. # create monitoring instances
  4. cd ../monitoring && tf init && tf apply
  5. # create Zookeeper and Solr cluster
  6. cd ../solr && tf init && tf apply
  7. # retreive Solr urls
  8. cd ../aws-solr-instances && tf init && tf apply -auto-approve && cd ../..
  9. # create film sample collection
  10. ./solr-create-collection.sh

Experimenting

  1. ./solr-configure-autoscaling.sh

Tear down

  1. cd terraform-workspaces/aws-solr-instances && tf destroy -auto-approve
  2. cd ../aws-solr && tf destroy -auto-approve
  3. cd ../aws-vpc && tf destroy -auto-approve

Local experimenting

  1. $ curl -sfLo jaeger-core.jar https://repo1.maven.org/maven2/io/jaegertracing/jaeger-core/1.8.0/jaeger-core-1.8.0.jar
  2. $ curl -sfLo jaeger-thrift.jar https://repo1.maven.org/maven2/io/jaegertracing/jaeger-thrift/1.8.0/jaeger-thrift-1.8.0.jar
  3. $ curl -sfLo libthrift.jar https://repo1.maven.org/maven2/org/apache/thrift/libthrift/0.16.0/libthrift-0.16.0.jar
  4. $ curl -sfLo solr-jaegertracer-configurator.jar https://repo1.maven.org/maven2/org/apache/solr/solr-jaegertracer-configurator/8.11.1/solr-jaegertracer-configurator-8.11.1.jar
  1. $ curl "http://localhost:8983/solr/admin/collections?action=CLUSTERPROP&name=samplePercentage&val=100"
  2. $ ./src/main/solr/scripts/solr-create-collection.sh