The Controller — Intel® SDE documentation

# The Controller

The controller is a flexible and powerful way for users to define a slice of execution which they want to profile by Intel® SDE or an Intel® SDE based tool. The controller component delivers controller events, and the tool is responsible to support the events, and define its behavior upon getting the event.

The controller supports a sophisticated definition for alarms and events. The user defines the controller events in a control chain. This defines a set of alarms which will trigger the events to be fired by the controller. All Intel® SDE active tools will receive these events, and act accordingly.

By default, if no start event is defined by the user in the command line, then a default start event is armed for each thread.

The controller provides a set of predefined alarms which include:

Name | Description
--- | ---
icount | count instructions where all the repeat string iterations are counted
instcount | count instructions where each repeat string instruction is count as a single count
address | a real virtual address or a function name (symbol) or image+offset
ssc | execution of a code sequence consisting of 2 instructions where the first has an immediate that identifies this marker
itext | execution of an instruction denoted by a sequence of raw bytes which is interpreted as an instruction
int3 | execution of an embedded INT3 instruction
isa_extension | execution of an instruction which belong to this XED ISA extension name
isa_category | execution of an instruction which belong to this XED ISA category name
isa_class | execution of an instruction which has this XED ICLASS name
enter_func | entering to a function with this name
exit_func | returning from a function with this name
image_load | the time the image load callback was detected by pin
image_enter | execution of the first instruction in the image
cpuid | execution of a CPUID instruction with a special input (as defined by the input registers)
magic | execution of a code sequence also used by other simulation tools which has two input values that identify the marker
pcontrol | entering the MPI_Pcontrol function with specific string argument that identify the marker
timeout | number of seconds to trigger event
system_call | execution of a system call with an input value as the system call number
interactive | the user interactively send the event to the process from another window/shell on the same machine

## Details

The controller exposes a knob: -control which gets the actual definition in a string argument. This knob can be defined multiple time in the Intel® SDE command line. This is a flexible way for the user to control the region being profiled by the tool.

Each instance of the “-control” knob, defines an alarm-chain as:

    % sde -control <event>:<alarm>:<value>[:count<int>][:tid<int>][:bcast] ...

A full definition of the syntax, and the available alarm-types can be found below.

This defines that the <event> will be fired by the controller, once the <alarm> identified by the <value> is executed. If :count<int> is used, the event will be fired only when the alarm identified by the <value> is executed for the <int> time.

Examples:

    % sde -skx -mix -control start:icount:100 -- myapp

The controller will fire a start event once 100 instructions are executed, and this will trigger the start of the collection of mix histogram analysis.

    % sde -skx -mix -control stop:address:foo:count3 -- myapp

The collection of mix histogram analysis will start at the start of the process, and the controller will fire a stop event, once we reach the symbol ‘foo’ for the 3rd time.

## Multiple alarms in a chain

Each instance of the -control knob defines, an alarm chain (see description below).

Alarm chain is a sequence of alarms, separated by the ‘,’ character, where each alarm is activated only after the previous one was fired. So the ‘,’ character applies an order between the proceeding alarm and the following alarm .

Example:

    % sde -skx -mix -control start:icount:1000,stop:icount:2000 -- myapp

In this example the start event will be fired after 1000 instructions and the stop event after 2000 more instructions (i.e. at 3000).

The controller knob can be specified multiple times, each defining an independent alarm-chain.

    % sde -skx -mix -control start:icount:1000 -control stop:icount:2000 -- myapp

In this example the start event will be fired after 1000 instructions and the stop event after 2000 instructions from the start of the run.

In addition, one can set the entire chain to start only after some other chain has finished, using the “,name:<name>” and the “,waitfor:<name>” modifiers.

## Pre-Condition

In some cases we want the event to be fired only after a certain condition. For example: we want to start event to be fired when a function foo is called from the function bar. This can be done with the pre-condition event type. This event doesn’t actually calls to the tool (i.e. fires an event) but only arms the next alarm in the chain.

Example:

    % sde -skx -mix -control precond:address:bar,start:address:foo,stop:icount:100000 -- myapp

The controller will arm the start event after calling to the bar function. Only now, when foo is called, the start event will be fired and the region start. The stop event will be triggered after 100000 instructions have been executed.

## Repeating an alarm multiple times

By default the alarm is “armed” only once for each thread. Once the event is fired, this thread won’t trigger an additional event, for this alarm. If one adds the “,repeat:<int>” to the end of the alarm, it will be activated <int> times. A repeat token without an argument means that the alarm will be fired every time it is executed.

Example:

    % sde -skx -mix -control start:address:foo,stop:icount:1000 -- myapp

The controller will fire a start event at the first call to the function foo, and the region will be terminated after 1000 instructions were executed. No more events will be fired even if the function foo is called again and again.

    % sde -skx -mix -control start:address:foo,stop:icount:1000,repeat:3 -- myapp

In this example, the controller will fire a start event at the call to the function foo, and the region will be terminated after 1000 instructions were executed. Then it will “re-arm” the address alarm, and fire a start event again at the next call to the foo function. The region will be terminated again after additional 1000 instructions were executed. This is repeated 3 times. As opposed to using the “:count3” syntax, which will fire a single start event, only once the start condition was executed 3 time.

Each tool might act differently on repeating of the region. In most cases the analysis will aggregate across all the active regions.

When the repeat modifier doesn’t include the repeat count, the region is started and stopped every time the conditions are met.

    % sde -skx -mix -control start:address:foo,stop:icount:1000,repeat -- myapp

In this example, the mix analysis tool will include the aggregation of all the regions starting from the call to the foo function and terminate after 1000 instructions.

## Specifying C++ Names

The Address alarm can be defined in three options:

- Absolute address specified by the ‘0x’ prefix.

- Image relative offset specified by an image name, the plus sign (‘+’) and an offset.

- Symbol name specified by a stream of characters without the plus sign.

By default the symbol name is searched as is, meaning that it matches the C language notation. When the intent is to give C++ name, Intel® SDE provides two alternatives. The default behavior will take the input symbol as a function name by removing all the C++ decorations. This means that it removes name spaces, class name, and the operands.

In some cases the same class method appears in multiple classes and the user wish to define a specific method. For this case, the user can use the at sign (‘@’) as the first character of the input symbol, and as a notation for name spaces and class separator (‘::’). Specifying the operands is still not supported.

For example, the application has a name space called MySpace and a class name MyClass and method called start. The Intel® SDE command will look like the following.

    % sde -skx -mix -control start:address:@MySpace@MyClass@start -- myapp

The same level of support also exists for the enter function and exit function alarms.

## Multi-threading

The alarm-chains are handled separately for each thread. The controller “arms” the <alarm-type> separately for each thread, and the alarm’s value is counted separately for each thread.

When the “:tid<int>” syntax is used, the preceding alarm is armed only for the thread with the input thread ID.

When the “:global” token is used, the alarm will be counted in all threads.

Adding the “:bcast” token, will cause the controller to specify that the event is marked with the broadcasting attribute (upon the arrival of the event). This means that when the alarm is executed, it fires the event to all active threads. The tool can use this information to decide if to profile all the threads based on this event or only the triggering thread. The effect of adding the “bcast” token depends on the way the tool handles this information. In Addition, an alarm defined with :bcast, will ‘arm’ the following alarm in the chain, for all threads.

Going back to the example above:

    % sde -skx -mix -control start:icount:100 -- myapp

For each thread that reaches an icount of 100, the controller will fire a start event to the tool. The event will be delivered with the tid of the triggering thread. Adding the “:tid<int> token as in the following example:

    % sde -skx -mix -control start:icount:100:tid3 -- myapp

Means that only when the application thread with pin thread ID 3 reach the icount of 100, the controller will fire the start event to the tool. Other application threads will not get the start event.

### Control Syntax Summary

The full definition of an alarm chain:

  -control <alarm-chain>[,repeat[:<int>]][,name:<name>][,waitfor:<name>]

alarm-chain ::= <alarm>[,<alarm-chain>]

alarm ::= <event>:<type>:<value>[:count<int>][:tid<int>][:bcast][:global]

event ::= start|stop|threadid|precond

type ::= icount|address|ssc|itext|isa-extension|isa-category|isa-class|int3|magic|pcontrol|enter_func|exit_func|img_load|img_enter|syscall|interactive

### Values per alarm type:

Alarm | Value
--- | ---
icount | unsigned long int
instcount | unsigned long int

address

- <name> [symbol/function name]

- <name>+<offset> [image name + offset]

- 0x<hex address> [pc address]

ssc | hex (w/o 0x): read more about SSC marks in Using SSC Marks
itext | hex: raw bytes of the instruction
int3 | no argument (the int3 instruction is skipped)
isa-extension | string: the instruction’s extension as specified by XED
isa-category | string: the instruction’s category as specified by XED
isa-class | string: the instruction’s ICLASS as specified by XED
enter_func | string: function name without namespace and params
exit_func | string: function name without namespace and params
img_load | string: image name
img_enter | string: image name
cpuid | hex: CPUID leaf (value in eax) match input value
magic | int.int: the instruction XCHG ebx,ebx and input/output values are defined by the numbers
pcontrol | string: the second argument to the MPI_Pcontrol function called by the application
timeout | int: number of seconds
syscall | int: number of the system call
interactive | no argument (requires additional knob (see below)

Note

Using the address alarm with image and offset, the ‘+’ sign is the key to distinguish between a function name and an image name. The image name can be full path or only the base name of the image.

### Interactive Controller

Using the interactive controller requires two windows: one to run sde with the application and the interactive controller, the second to specify the start/stop event by using the controller client. Here is an example:

    Application:
    %  <kit>/sde -hsw -mix -control start:interactive:bcast,stop:interactive:bcast \
       -interactive-file /tmp/myapp_ctrl -early-out -- myapp

    Main process pid: 32564
        ** using file: /tmp/myapp_ctrl.32564
        ** listening to port: 34106

    Controlller start event:
    % <kit>/misc/sde_cntrl_client /tmp/myapp_ctrl.32564

    Controlller stop event:
    % <kit>/misc/sde_cntrl_client /tmp/myapp_ctrl.32564

## Alarms accuracy

The controller will fire the event once the alarm reaches the triggering condition. The instruction count alarm specifically works in a basic block granularity. The event will be fired at the beginning of a basic-block, when the current icount + the number of instructions within the basic-block, exceed the value defined in the controller, for triggering the event.

Most of the location based events (e.g. address, ssc, and more) where modified to be more accurate and they are triggers in the specific instruction and not at the basic block level.

Note

Using the controller for specifying region of interest for tracing with pinplay has special handling. The start and stop events always act on all the threads. Due to implementation limitations there is a small delay between the exact instruction on which the event is fired and when it is actually start or stop the region.

## Special Alarm - Uniform:

Uniform sampling is a way to collect periodic regions at constant length from a long running application. The uniform sampling controller can be triggered at the start of the process or at certain event. It is defined as uniform:<period>:<length>:<count>

period: number of instructions before next sampling starts.
length: number of instructions to sample.
count: number of samples.

The alarm can be used to define multiple regions based on instruction count.

The Uniform Alaram Tokens

The uniform sampling controller follow the same modifiers defined in the general controller options.

repeat[<int>]: number of iterations of the chain (when no number provided, execute till the end of the process).

count<int>: delay firing the event to only the Nth execution of the alarm, the counting happen for each thread unless global is also specified.

tid<int>: the thread to monitor, events on other threads are ignored.

bcast: inform the tool that the event should be processed for all threads (this behavior is tool specific).

name<string>: the name of the chain, other chains can wait for this chain to finish before it starts.

waitfor<string>: start the chain only after the chain with the specified name has finished.

global: Count alarms summary for all threads and not in a specific thread. this token can not appear with tid token.

## Controller knobs

`-control`

  Define control events and conditions

`-controller_default_start`

  Add default START event [default 1]

`-controller_log`

  Emit the controller’s events log [default 0]

`-controller_olog`

  File name to emit controller’s event log [default controller-log.txt]

`-controller_rep_nonstop`

  Do not stop on repeat string instructions [default 0]

`-interactive_file`

  Name of file used for interactive communication (Linux)

`-interactive-event`

  Name of event used for interactive communication (Windows)

## Backward Compatibility

Most of the knobs from the old controller are still supported by the new controller. You are encouraged to use the new controller as it provides better control.

`-length`

  Number of instructions to execute before stopping

`-skip`

  number of instructions to skip

`-start_address`

  Address and count to trigger a start

`-start_category`

  Trigger a start on seeing an instruction from this XED ISA category

`-start_extension`

  Trigger a start on seeing an instruction from this XED ISA extension

`-start_int3`

  Trigger a start on seeing an ‘int 3’ instruction, requires a count argument

`-start_itext`

  Trigger a start on seeing specified instruction bytes (max 15Bytes)

`-start_ssc_mark`

  Trigger a start on seeing specified SSC marker, big-endian hex without an 0x prefix

`-stop_address`

  Address and count to trigger a stop

`-stop_category`

  Trigger a stop on seeing an instruction from this XED ISA category

`-stop_extension`

  Trigger a stop on seeing an instruction from this XED ISA extension

`-stop_int3`

  Trigger a stop on seeing an ‘int 3’ instruction, requires a count argument

`-stop_itext`

  Trigger a stop on seeing specified instruction bytes (max 15Bytes)

`-stop_ssc_mark`

  Trigger a stop on seeing specified SSC marker

## Controller support in SDE tools

As mentioned above, the controller is a self contained component within SDE. All aforementioned details, is related to when and what events are fired by the controller. This section will discuss which SDE tools support using the controller, and how do they handle events fired by the controller.

All analysis tools behave similarly: The tool collects data per thread. Based on an arrival of a start event triggered by that thread. The tool will stop the collections of data for that thread, when a stop event arrives for that thread. If the tool receives an event with “bcast” as the tid, rather than the triggering tid, the tool will apply this event on all threads.

The effect of the events are effective right at the point of arrival.

## Tracing Support

The tracing tool handles a global region. Meaning, at each given time, we are either in a region, and tracing all threads, or we are outside a region, and not tracing any thread. Another special behavior of the tracing tool is the transition from in/out a region. The effect of the event arrival isn’t immediate.

Following is a description of what happens in the tracing tool upon an arrival of a start event. When the thread which “caught” the event reaches the end of it’s current Basic-Block (BBL), it stops and calls all other threads to stop execution. Each thread is stopped at the end of it’s next BBL. Once all treads are stopped, Intel® SDE “removes” the current instrumentation and change the mode to “in-region”, meaning that all executing code will be instrumented with the trace generation instrumentation.

At this point, all threads are resumed. The same process is done for stopping the trace generation.

Note

The tracing tool ignores any event that arrive after recieving the event and resuming the execution (steps 1 and 4). The controller behavior is orthogonal to the way abalysis tools handle the events. So the controller will continue firing events based on the alarms defined by the user in the command line. The tracing tool will ignore them if a previous event arrived, and its processing hasn’t been completed yet.

By default, sde analysis tools follow the same behavior when running together with the tracing tool. You can cancel this by adding ‘-pinplay-control 0’, this will keep the behavior of the sde analysis tools as described above.