The primary output of an Execution are the results (i.e. Result instances) produced by the MinerTools employed in the Scenario. Additionally, the user can retrieve a so called Executionlog, a list of logging statements that was recorded during the Execution by the MINER Server as well as the ToolProxies and MinerTools.

Introduction

Retrieving the results of an Execution is a central aspect of running experiments with MINER.

Incremental and limited queries

There are 2 fundamental aspects of the result query system in MINER.

Incremental: Result retrieval in MINER is generally incremental, i.e. a query for results only returns results that accumulated at the MINER Server since the last query.

Limited: The result set returned by the Server is possibly limited, i.e. it may contain less results than available at the Server. The number of results returned per query is determined by the minimum of 3 parameters: a server-side query limit; a user-configurable limit; and naturally the number of results available at the Server.

The server-side query limit is configured by the operator of the MINER installation (configuration parameter: queryLimit). In exists in order to prevent memory exhaustion at the Server. The user-configurable limit can optionally be defined by the user per MinerTool Result.

Defining which results to query

As explained in section “Defining the requested results“, a user must explicitly request a specific result class in the Scenario definition:

ResultHandle hLossRatio = action.produce("lossRatio")

As a Scenario can be run many times, the ResultHandle alone doesn’t sufficiently identify the results to be queried. It is additionally necessary to define from which Execution the lossRatio results have to be retrieved. Therefore, a result query is always tied to one specific pair of (ResultHandle, Execution) instances.

Querying results

Due to the importance of retrieving results, MINER provides several convenience methods for doing so. In the end, it is always the Query interface (and its implementations) through which results are retrieved from the MINER Server. It can be employed directly by a user as explained below in section “Querying Results via the Query interface“.

Many times however, the user doesn’t need (and want) to manage the Query objects himself. Therefore, as a first layer of convenience, the Execution class provides several methods for querying results (without having to deal with Query objects). See the description below in section “Querying results via the Execution class“.

Finally, there is a second convenience layer that is attractive when results shall not only be retrieved but also printed or saved to a file in a single method call. To this end, the Results class provides several methods as explained below in section “Querying results via the Results class“.

Querying results via the Results class

In the simplest case, a user wants to retrieve all results of an Execution and save them to files. This can be achieved in a single line:

Results.getAndSaveAllResults(exec);

This method retrieves all available results for each ResultHandle that has been created in the Scenario definition via action.produce(…). While the underlying result query is incremental (it always is), this method keeps getting blocks of results until no more are available at the Server.

As with most other methods of the Results class, each …Save… method has a corresponding …Print… method that prints the results to stdout:

Results.getAndPrintAllResults(exec);

The results can also retrieved for a specific set of ResultHandles:

Results.getAndPrint(exec, resultHandle1, resultHandle2, ...);
Results.getAndSave(exec, resultHandleCollection);

As can be seen in the Javadoc of the Results class, multiple ResultHandles can be passed as a Collection or via varargs.

There is another pair of methods that gets the results, prints/saves them and returns them to the user:

List<Result> myResults = Results.getAndPrintAndReturn(exec, handle);
List<Result> otherResults = Results.getAndSaveAndReturn(exec, handle);

The Results class is fully integrated with the ResultFormatter interface and there are additional methods that enable the usage of a specific formatter, e.g.:

ResultFormatter myFormatter;
Results.getAndSave(exec, myFormatter, resultHandle);

Obviously, these result saving methods automatically choose the name of the output files. The way the output file name is determined is described below in section “Automatic naming of result files“.

Querying results via the Execution class

A user may want to avoid having to create and save ResultHandle objects for potentially many pairs of (Execution, ResultHandle). As a convenience, the Execution class offers several methods for result querying. This frees the user from working with Query implementations directly. Under the hood, the Execution class creates those objects itself and calls the appropriate methods on behalf of the user.

The following methods are offered by the Execution class

exec.setQueryLimit(resultHandle, limit);
int userLimit = exec.getQueryLimit(resultHandle);
boolean moreResults = exec.hasNextResults(resultHandle);
List<Result> results = exec.nextResultBlock(resultHandle);
List<Result> results = exec.nextResults(resultHandle);
List<Result> results = exec.allResults(resulHandle);

It should be obvious how these methods map to the Query interface described below.

It is worth pointing out the difference between exec.nextResults() and exec.allResults(). The exec.nextResults(handle) method returns all result instances that have become available since the previous invocation of exec.nextResults(handle). In contrast, exec.allResults(handle) retrieves all result instances (i.e. from the first to the last) that are currently available at the MINER Server – independent of any previous queries. Internally, both methods repeatedly invoke nextResultBlock() as long as hasNext() returns true to cope with the result query limit.

The method exec.allResults() has no side-effect (in neither direction) on any (previous/parallel/subsequent) incremental result retrievals on resultHandle (e.g. via nextResults(), allNextResults(), etc.) because internally a new Query implementation is created and used to retrieve all existing results.

Querying Results via the Query interface

The Query interface provides the foundation for all result queries and is used by the various convenience methods described above. A user can obtain an implementation of the Query class and subsequently make use of it to get maximum control over the result retrieval process.

Query query = exec.createQuery(resultHandle)

As described above, the user may define a query limit to constrain the maximum number of results retrieved from the MINER Server per invocation of nextResultBlock().

query.setLimit(numResults);
query.getLimit();

To check if more results are available at the Server:

boolean moreResults = query.hasNext()

The result retrieval is achieved with the nextResultBlock() method:

List<Result> results = query.nextResultBlock();

This method incrementally retrieves the next block of result data that became available at the Server since the previous invocation of the method. As explained above, the returned result list does possibly not contain all results available at the Server as the maximum size of the result list is limited by min(userLimit, serverLimit, availableResults). The hasNext() method tells if more results are available at the Server.

To retrieve all results that became available at the Server since the previous retrieval, nextResultBlock() can be invoked repeatedly as long as hasNext() is true. As this is a common use case, the Query interface provides this functionality in the method nextResults():

List<Result> results = query.nextResults();

Note that the size of the results list can now be larger than the user- or server-imposed query limit. Depending on the scenario, this might lead to heap space problems at the client.

The Result class

All methods that query results from the MINER Server return a list of results: List<Result>. A Result consists of a timestamp, a value, and proxy that references the ToolProxy where the result was produced.

The timestamp denotes the number of milliseconds since January 1, 1970, 00:00:00 GMT. There is a generic getter getValue() that returns the value as a String. Additionally, there are getters that try to convert the String value to an int / long / double.

The class provides a toString() implementation that converts a Result to the String in the following format:

{timestamp}{separator}{value}[{separator}{proxy}]

The default separator is “\t”. It can be modified via

Result.SEPARATOR = ",";

The name of the ToolProxy is only included if it is not null.

Example:

1306316979500 441 tp1
1306316980000 237 tp1
1306316980499 222 tp1
1306316980999 222 tp1

Formatting Results with a ResultFormatter

While the Result class implements the toString() method for basic needs, the Clientlib also provides a far more flexible way to convert a Result to a String. This support comes in the form of the ResultFormatter interface as well as a few implementations of it: ResultFormatterSimple, ResultFormatterDate, and ResultFormatterDeltaTime. Please see the Javadoc of these classes for a description. Of course, a user is free to create alternative implementations of this interface.

The interface contains a single method format() that takes a Result object and returns a String. The introduction of this interface was motivated by the fact that there is no single “right” or “best” String representation of a result. Typically, the requirements for screen output are different than for saving results to file. In the later case, for example, saving the timestamp in milliseconds is desired. On the other hand, these numbers make little sense on screen where a date/time format is much easier to read.

The following table shows the output produced by 3 different ResultFormatter implementations.

1312456651773 -- 0 -- tp1
1312456652773 -- 1 -- tp1
1312456653779 -- 2 -- tp1
1312456654780 -- 3 -- tp1
1312456655781 -- 4 -- tp1
1312456656782 -- 5 -- tp1
1312456657783 -- 6 -- tp1
1312456658784 -- 7 -- tp1
1312456659785 -- 8 -- tp1
1312456660786 -- 9 -- tp1

00:00.191    0
00:01.191    1
00:02.197    2
00:03.198    3
00:04.199    4
00:05.200    5
00:06.201    6
00:07.202    7
00:08.203    8
00:09.204    9

0.0 0
1.0 1
1.006 2
1.001 3
1.001 4
1.001 5
1.001 6
1.001 7
1.001 8
1.001 9

The Results class makes heavy use of the ResultFormatter interface. It is always utilized when a Result has to be converted to a String. Most methods of the Results class come in pairs of 2: one variant where the user passes a ResultFormatter implementation, and another one where this parameter is omitted. In the second case, the method relies on a default implementation. The Results class maintains 2 of them: one for printing results (a ResultFormatterDate) and one for saving results (a ResultFormatterSimple). These default implementations can be replaced by the user:

ResultFormatter myPrintFormatter = ...
ResultFormatter mySaveFormatter = ...
Results.setPrintFormatter(myPrintFormatter);
Results.setSaveFormatter(mySaveFormatter);

Utility methods provided by the Results class

The Clientlib ships with the Results class (note the trailing s) that contains static utility methods for working with results. It supports users in printing and saving results and provides methods for combing result retrieval and printing/saving in a single call as shown above in section “Querying via the Results class“. The Results class works hand in hand with the ResultFormatter interface.

Printing results

The Results class provides methods for printing and saving Result instances:

List<Result> results;
Results.print(results);
Results.save(results, file);
Results.append(results, file);

Saving results

A user may want to specify a base directory for storing results.

Results.setSaveDir(File directory)

Results are then saved to this directory in the following cases:

• the user calls a save/append method and passes a filename as a String (not as a File!)

  Results.save(List<Result> results, String filename);
  Results.append(List<Result> results, String filename);

• the user calls a save method and doesn’t pass a filename at all

  Results.getAndSaveAllResults(...);
  Results.getAndSave(...);
  Results.getAndSaveAndReturn(...);

The saveDir is not used in calls where the user passes the output file as a File instance:

Results.save(List<Result> results, File file);
Results.append(List<Result> results, File file);

Automatic naming of result files

The Results class offers several methods that save results to a file. Some of those methods don’t take a filename as an argument.  In this case, the output file is determined by the method itself.

The algorithm takes into account a savePattern (if defined for a resultHandle as explained in section “Defining a save pattern“) and the saveDir (if defined). The algorithm creates a File instance that is constructed from 3 parts: the directory, the filename, and the suffix.

We have to distinguish between binary and non-binary results, as for binary results, each result instance is saved to a separate file.  On the contrary, all instances of a non-binary result, e.g. lossRatio, are saved to 1 file.  Therefore, the naming scheme differs for binary and non-binary results.

The output file is determined in the following way:

• Directory
  If the savePattern begins with an absolute path, this path is the target directory for the result file. Otherwise, if a saveDir was previously defined via setSaveDir(), this directory is the target directory. Else, the target directory is the working directory. If the savePattern begins with a relative path, this path is relative to the directory determined in the previous step (saveDir or working directory).
• Filename
  • Non-binary result
    If the savePattern defines a name (after a path and before a suffix, if any), this name is used as the filename. Otherwise, the filename is set as the name of the result and if the result has a selector, the filename is set as “name-selector”, respectively.
  • Binary result
    If the savePatterndefines a name (after a path and before a suffix, if any), this name is used as the beginning of the filename. Then, the following string is appended:

    {proxyName}-{resultName}[-{resultSelector}]-{count}-{timestamp}

• Suffix
  If the savePattern ends with a suffix (e.g. “.log”), this suffix is used for the result file. Otherwise, unless disableAutoSuffix() has been called before, a suffix is appended to the result file name (“.bin” for binary results, “.txt” for all other types).

Examples

The following table shows the target file for some combinations of result name, savePattern and saveDir.  CWD refers to the current working directory. The lossRatio result is non-binary result whereas raw is a binary result.

Note that in the last lossRatio example the saveDir is ignored because the savePattern contains an absolute path.

Combined query and printing/saving

Results.getAndSave(...);
Results.getAndPrint(...);
Results.getAndSaveAllResults(...);
Results.getAndPrintAllResults(...);
Results.getAndPrintAndReturn(...);
Results.getAndSaveAndReturn(...);

See the description in section”Querying via the Results class” above.

Saving queries

The Results class provides methods that allow a user to marshall a Query to a String. This String can later be converted back to a Query.   This functionality is needed if the definition and scheduling of a MINER Scenario is made in 1 application and the retrieval of results from Executions of this Scenario is done in another application.

To marshall a Query to a String:

Results.marshallQuery(execution, resultHandle)

Multiple queries can be marshalled to one file in 1 step using:

Results.marshallQueriesToFile(file, execution, resultHandle...)

To reconstruct a Query from a String:

Query resultQuery = Results.unmarshallQuery(String)

Querying the Executionlog

The Executionlog is a list of logging statements that was recorded during the Execution by the MINER Server as well as the ToolProxies and MinerTools.

The Executionlog is retrieved via:

Executionlog log = exec.getLog()

This method returns an Executionlog object which contains all the log records of this Execution so far. Note that unlike result retrieval, getting the log is not incremental, i.e. this method always returns the complete log.

The Executionlog contains a list of LogEntry objects each of which represents a logging record. A LogEntry consists of a timestamp, a logging level, a message, and an origin that indicates where the log message was generated.

Executionlog log = exec.getLog();
for (LogEntry entry : log.getEntries()) {
    Date timestamp = entry.getTimestamp();
    String level = entry.getLevel();    // INFO, WARN, ERROR
    String msg = entry.getMessage();
    String origin = entry.getOrigin();  // e.g. [execId=14/tp1/a1/TestTool]

}

The Executionlog class offers some methods to query the number of log records with WARN or ERROR level (and if such records exist) without having to parse each entry.

log.hasWarning();
log.hasError();
log.hasWarningOrError();
log.getNumWarning();
log.getNumError();

Advanced topic: restricting queries

As described in section “Defining multiple ToolProxies” there are cases where the user binds an Action to multiple ToolProxies. Basically, querying results in such in case works exactly in the same way as if an Action is bound to a single ToolProxy.

Internally, the Clientlib retrieves the results from the MINER Server in multiple round-trips because a result query to the Server is tied to exactly 1 ToolProxy. The Clientlib collects these results per ToolProxy and returns a single unified list of results to the user.

It is possible for a user to restrict the set of ToolProxies for which the Clientlib queries results from the Server. There are 2 use cases where a user may want to make use of this optimization.

1. The user knows from the documentation of a MinerTool that results are produced by a single ToolProxy only. As an example, the IPPMTool needs to be bound to exactly 2 ToolProxies: one acts as a traffic source, the other one as a traffic sink. As stated in the documentation, results are exclusively produced by the traffic sink. Therefore, querying the MINER Server for results produced by the ToolProxy that hosts the traffic source will always return an empty list. To optimize the query process, the user defines that results are only queried for the receiver-side ToolProxy.

   Action a = task.addAction("IPPM");
   a.setTool("at.srfg.miner.tool.ippm.IPPMTool");
   a.setToolconfig(new File("etc/ippm.xml"));
   a.addProxy("sender", "receiver");
   ResultHandle delay = a.produce("owdApp");
   delay.setQueryProxies("receiver");

2. The user is (currently) only interested in results produced by a limited set of ToolProxies. As an example, assume there is a MinerTool that captures traffic from a network interface as a binary file. The Action is bound to multiple ToolProxies and on each of them the trace file is generated. However, in a first step the user only needs to analyse the trace from linux3.

   Action a = task.addAction("Capture");
   a.setTool("at.srfg.miner.tool.sniffer.SnifferTool");
   a.setToolconfig(new File("etc/sniffer.xml"));
   a.addProxy("linux1");
   a.addProxy("linux2");
   a.addProxy("linux3");
   a.addProxy("linux4");
   ResultHandle dump = a.produce("raw");
   dump.setQueryProxies("linux3");
   List<Result> linux3Dump = exec.nextResults(dump);

   A user may decide to also analyse the dumps from 2 other hosts in a next step.

   dump.setQueryProxies("linux1", "linux4");
   Query query = exec.createQuery(dump);
   List<Result> dumps = query.nextResults();

   Note that in this case the user must explicitly create a new Query implementation and can’t use the exec.nextResults() convenience method (as the Execution internal Query implementation evaluates a ResulHandle’s queryProxies only at construction time).