Measurement Kit

(deprecated) Network measurement engine

As of 2020-03-15, Measurement Kit is deprecated. This date has been chosen
arbitrarily such that we could write:

Friends, OONItarians, developers, lend me your ears;
I come to bury Measurement Kit, not to praise it.
The bugs that software have live after them;
The good is oft interred with their remote branches;

And, of course, some good, old piece of art is also in order:

The rewrite of Measurement Kit in Go has been going on for quite some time now
as ooni/probe-engine. As part of this
rewrite, we considered all the use cases addressed by Measurement Kit, as documented
by issue #1913.

We will most likely never release v0.11.0 of Measurement Kit. We will keep
maintaining the 0.10.x version until 2021-03-14 (which is another
interesting date). In the following, we’ll discuss
what options you have for replacing Measurement Kit in your use case. The
current README reflects the situation “here and now”. We are still working to
provide a smooth upgrade path. Please, let us know if the upgrade path we have
designed is troubling you by voting/contributing to the issues indicated below.

(The content of the old README.md is still available as OREADME.md.)

Changes in the settings JSON

The ooni/probe-engine implementation exposes similar APIs to Measurement Kit
and specifically honours the data format of Measurement Kit v0.10.11.
There should be no differences in the emitted events. There are however some
differences in the settings as discussed below.

You should now add the following three keys to the settings JSON:

{
  "assets_dir": "",
  "state_dir": "",
  "temp_dir": ""
}

where assets_dir is the directory where to store assets, e.g.
GeoIP databases; state_dir is the directory where to store the
authentication information used with OONI orchestra; temp_dir
is the directory where to store temporary files. If these three
keys are not present, the test will fail during the startup
phase (i.e. it will run for a very short time and you will see
a bunch of failure.startup events emitted).

Also, the Go code does recognize all the settings recognized by
Measurement Kit, but we have only implemented the settings required
by OONI. All the other settings, when used, cause a failure during
the experiment startup phase. If a not implemented setting is causing
you issues, let us know by voting in the corresponding bug tracking
issue.

Android

In your app/build.gradle file, replace

  implementation "org.openobservatory.measurement_kit:android-libs:$version"

with

  implementation "org.ooni:oonimkall:$version"

(The new package is called oonimkall because it’s a OONI probe-engine based
implementation of the mkall API for iOS. In turn, mkall means that we
are bundling together all the MK APIs (i.e. the API for running experiments
and all the ancillary APIs). For historical reasons android-libs was
named before we defined the concept of mkall and was never renamed.)

The following differences apply between android-libs and oonimkall:

1. the import path is oonimkall and you can use it directly as a scope
   for the classes, rather than doing import oonimkall.Foo;

2. the MKAsyncTask class is replaced by oonimkall.Task and the
   MKAsyncTask.start factory is replaced by oonimkall.Oonimkall.startTask;

3. the MKGeoIPLookupResults and MKGeoIPLookupTask classes are
   replaced by oonimkall.GeoLookupResults and oonimkall.GeoLookupTask;

4. the MKOrchestraResults and MKOrchestraTask classes cannot be
   replaced, because it seems we are moving away from the orchestra model
   and, in going forward, we will only use orchestra internally inside of
   probe-engine to authenticate probes when they fetch input;

5. the MKReporterResults and MKReporterTask classes are replaced
   by oonimkall.CollectorResults and oonimkall.CollectorTask;

6. the MKResourcesManager class cannot be replaced, because the new
   code manages resources differently, by downloading them when needed
   into the assets_dir directory mentioned above;

7. the MKVersion class cannot be replaced because version pinning in
   Go makes it much simpler to know which version of what software we compile;

8. oonimkall throws Exception in much more cases than the code in
   android-libs that instead was using RuntimeException (using the latter
   was actually an anti-pattern and we are fixing it with the new code).

The following diff shows how to update code that runs an experiment, which
is probably the most common use case of android-libs:

--- MK.java    2020-04-10 11:54:53.973521643 +0200
+++ PE.java    2020-04-10 11:55:50.915205613 +0200
@@ -1,10 +1,8 @@
 package com.example.something;
-import io.ooni.mk.MKAsyncTask;
-
 public class Example {
-    public static void run(settings String) {
-        MKAsyncTask task = MKAsyncTask.start(settings);
+    public static void run(settings String) throws Exception {
+        oonimkall.Task task = oonimkall.Oonimkall.startTask(settings);
         for (!task.isDone()) {
             String event = task.waitForNextEvent();
             System.out.println(event);

The most striking difference is that the function to start a task
will explicitly throw Exception on failure. The old code
would instead throw RuntimeException, as mentioned above. The
required settings have slightly changed, as discussed above.

iOS

In your Podfile replace

    pod 'mkall', :git => 'https://github.com/measurement-kit/mkall-ios.git',
                 :tag => '$version'

with

    pod 'oonimkall', :podspec => 'https://dl.bintray.com/ooni/ios/oonimkall-$version.podspec'

The changes are similar to the ones described above for Android except
that the oonimkall. prefix is Oonimkall for iOS. The following diff
shows how you should be upgrading your MKAsyncTask code:

--- MK.m    2020-04-10 12:06:14.252573662 +0200
+++ PE.m    2020-04-10 12:08:18.520924676 +0200
@@ -1,11 +1,15 @@
 #import <Foundation/Foundation.h>
-#import <mkall/MKAsyncTask.h>
+#import <oonimkall/Oonimkall.h>
-void run(NSDictionary *settings) {
-    MKAsyncTask *task = [MKAsyncTask start:settings];
-    while (![task done]) {
-        NSDictionary *ev = [task waitForNextEvent];
+NSError *run(NSString *settings) {
+    NSError *error = nil;
+    OonimkallTask *task = OonimkallStartTask(settings, &error);
+    if (error != nil) {
+        return error;
+    }
+    while (![task isDone]) {
+        NSString *ev = [task waitForNextEvent];
         if (ev == nil) {
             continue;
         }

The most striking differences are the following. First, the function
that starts a task now fails explicitly (e.g., if the settings
are not valid JSON). Second, the new code takes in input and emits in
output serialized JSONs rather than NSDictionary *. You are welcome to
adapt code from MKAsyncTask
to reimplement the previous behaviour. Also, remember that
some extra mandatory settings are required, as described above.

Command Line

The miniooni binary
mostly has the same CLI of the measurement_kit binary you could build from
this repository. The following list describes the main differences between
the two command line interfaces:

• miniooni by default appends measurements to report.jsonl while
  measurement_kit uses a file name including the experiment name and the
  datetime when the experiment was started;

• miniooni uses the -i, --input <input> flag to uniformly provide input for
  every experiment, while in measurement_kit different experiments use
  different command line flags after the experiment name (e.g., the -u <URL>
  flag is used by MK’s Web Connectivity);

• miniooni allows you to specify a proxy (e.g. Tor, Psiphon) with
  -P, --proxy <URL> that will be used for interacting with OONI services,
  but no such option exists in measurement_kit;

• miniooni does not yet implement -s, --list that lists all the
  available experiments;

• miniooni does not implement -l, --logfile <path> but you can use
  output redirection and tee to save logs anyway;

• miniooni does not implement --ca-bundle-path <path>, --version,
  --geoip-country-path <path>, --geoip-asn-path <path>, because
  these resources are now downloaded and managed automatically;

• miniooni does not implement --no-resolver-lookup;

• miniooni writes state at $HOME/.miniooni.

We automatically build miniooni for windows/amd64, linux/amd64,
and darwin/amd64 at every commit. The Linux build is static and does not depend
on any external shared library. You can find the builds by looking into the
GitHub actions of probe-engine
and selecting for cli-windows, cli-linux, or cli-darwin. If you want us
to attach such binaries to every release, please upvote
the related issue.

Shared Library

The libooniffi
package of ooni/probe-engine is a drop-in replacement for the Measurement
Kit FFI API. The new API is defined by the
ooniffi.h header. It is ABI compatible with MK’s API. The
only required change is to replace the mk_ prefix with ooniffi_. This
diff shows the changes you typically need:

--- MK.c    2020-04-10 12:32:13.582783743 +0200
+++ PE.c    2020-04-10 12:32:36.633038633 +0200
@@ -1,19 +1,19 @@
 #include <stdio.h>
-#include <measurement_kit/ffi.h>
+#include <ooniffi.h>
 void run(const char *settings) {
-    mk_task_t *task = mk_task_start(settings);
+    ooniffi_task_t *task = ooniffi_task_start(settings);
     if (task == NULL) {
         return;
     }
-    while (!mk_task_is_done(task)) {
-        mk_event_t *event = mk_task_wait_for_next_event(task);
+    while (!ooniffi_task_is_done(task)) {
+        ooniffi_event_t *event = ooniffi_task_wait_for_next_event(task);
         if (event == NULL) {
             continue;
         }
-        printf("%s\n", mk_event_serialization(event));
-        mk_event_destroy(event);
+        printf("%s\n", ooniffi_event_serialization(event));
+        ooniffi_event_destroy(event);
     }
-    mk_task_destroy(task);
+    ooniffi_task_destroy(task);
 }

Of course, you also need to take into account the changes to
the settings documented above.

You can generate your own builds with:

# macOS from macOS or Linux from Linux
go build -v -tags nomk -ldflags='-s -w' -buildmode c-shared -o libooniffi.so ./libooniffi
rm libooniffi.h  # not needed
cp libooniffi/ooniffi.h .  # use this header
# Windows from Linux or macOS with mingw-w64 installed
export CGO_ENABLED=1 GOOS=windows GOARCH=amd64 CC=x86_64-w64-mingw32-gcc
go build -v -tags nomk -ldflags='-s -w' -buildmode c-shared -o libooniffi.dll ./libooniffi
rm libooniffi.h  # not needed
cp libooniffi/ooniffi.h .  # use this header

Let us know if you want us to automatically publish libooniffi dynamic
libraries by upvoting the related issue.