spark>> spark-monitoring-client>> 返回
项目作者： talal830

项目描述 ：
A Restful Java Client for Spark Monitoring API
高级语言： Java
项目地址: git://github.com/talal830/spark-monitoring-client.git
创建时间： 2018-01-02T11:35:14Z
项目社区：https://github.com/talal830/spark-monitoring-client
开源协议：
下载


Spark Monitoring API Client

Java Client for Spark Monitoring API written with Retrofit.

Getting Started

Tested with Spark V2.2.0

Usage

Spark Client with default history server i.e, http://127.0.0.1:18080/api/v1/:

  1. // Create client
  2. SparkClient client = SparkClient.createHistoryClient();
  4. // Get all completed applications
  5. Optional<List<Application>> applications = client.getApplications(SparkRestService.APPLICATION_STATUS_COMPLETED);
  6. List<Application> applicationList = applications.get()
  8. // Get application Id at first index
  9. String appId = applicationList.get(0).getId()     
  11. // Get Jobs against application   
  12. Optional<List<Job>> jobs = this.client.getJobs(appId);
  14. // Get Stages against application
  15. Optional<List<Stage>> stages = this.client.getStages(appId);

Creating SparkClient using Builder:

  1. SparkClient client = new SparkClient.Builder()
  2.         .setBaseUrl("http://127.0.0.1:18080/api/v1/") // History Server Url
  3.         .addConverterFactory(MoshiConverterFactory.create()) // For other converters see Retrofit
  4.         .get();

Using SparkRestService:

  1. // Create client //
  2. ...
  4. SparkClient client = SparkClient.createHistoryClient();
  6. ...
  8. // Get SparkRestService via client
  9. SparkRestService restService = client.getRestService();
  12. // Make API calls
  13. Call<List<Application>> getAppsCall = restService.getApplications();
  14. Response<List<Application>> getAppsResponse = getAppsCall.execute();
  15. List<Application> applicationList = getAppsResponse.body();

For further details see SparkClientTest and SparkRestServiceTest

Install

Repository

  1. git clone https://github.com/talal830/spark-monitoring-client.git
  2. cd spark-monitoring-client
  3. mvn clean install

Dependency

  1. <dependency>
  2.     <groupId>org.apache.spark</groupId>
  3.     <artifactId>spark-monitoring-client</artifactId>
  4.     <version>2.2.0</version>
  5. </dependency>

Built With

  • Retrofit - A type-safe HTTP client for Android and Java

TODO

Add streaming calls:

  • /applications/[app-id]/streaming/statistics
  • /applications/[app-id]/streaming/receivers
  • /applications/[app-id]/streaming/receivers/[stream-id]
  • /applications/[app-id]/streaming/batches
  • /applications/[app-id]/streaming/batches/[batch-id]
  • /applications/[app-id]/streaming/batches/[batch-id]
  • /applications/[app-id]/streaming/batches/[batch-id]/operations/[outputOp-id]

License

This project is licensed under the MIT License