Skip to content

Services

Functionalities under services are a bunch of standalone features, which are not alarms, metrics and notifications, but aiding devops daily tasks. These features turn WombatOAM to be an active agent. That means users can retrieve live information about running systems (such as the state of a gen_server), recover from outages for instance by forcing GC to save more memory, or fix misconfiguration issues by changing configuration parameters.

The services provided fall into 2 categories:

  • explorer: inspecting the subject on a read-only way,
  • executor: executed for their side-effect.

The scope of each service is either one node or one node family. Each service has zero or more arguments. The started instance of a service is called a request.

The provided services include

  • Etop like process listing.
  • Inspecting processes.
  • Evaluating Erlang/Elixir expressions.
  • Soft purge modules.
  • Garbage Collect Processes.

There are some boolean properties of services worth mentioning:

  • Exclusiveness: if a service is exclusive only one request of the same service can be run on the same node or node family at the same time. This is because the request either uses some global resource or to prevent multiple instances using too much resources that would have impact on the managed node.
  • Periodicity: most services are executed only once with their result shown as output or executed for their side-effects. Periodic services, however, keep running until they are stopped regularly updating their output data (for example Processes list). A Running request can be stopped explicitly by the user or it also terminates if it is left unattended (i.e. no browser is subscribed to display its output). This inactivity timeout is one minute.

Explorer

Retrieve either static or periodically updated information about a certain process or about the VM. Has none or only a very small side-effect on the managed node.

The following services are available:

  • Process Manager. Presents information about Erlang processes similar to the information presented by "top" in UNIX. See details in the Process Manager section.

  • Process info. Shows the process information of the process given by its pid or registere name.

  • Process messages. Shows the message queue of the process given by its pid or registere name.

  • Process stack trace. Shows the stack trace of the process given by its pid or registere name.

  • Process dictionary. Shows the dictionary of the process given by its pid or registere name.

  • Process state. Shows the state (if available) of the process given by its pid or registere name.

  • Table Visualizer. Shows information about the ETS tables and their contents. See details in the Table Visualizer section.

Process Manager

For each process the following information is displayed:

  • Pid. The process identifier.

  • Reds. The number of reductions executed by the process.

  • Memory. The size of the process, in bytes, obtained by a call to process_info(Pid, memory).

  • MsgQ. The length of the message queue of the process.

Configuration possibilities are:

  • Refresh interval in seconds. Time interval (in seconds) between each update of the output.

  • Number of listed processes. Number of rows (processes) to display.

  • Order by column. Identifies what information to sort by.

  • Order direction. Identifies which direction to sort by (Ascending/Descending).

Various information can be retrieved by clicking on any Pid, such as:

  • Process info.

  • Process messages.

  • Process stack trace.

  • Process dictionary.

  • Process state.

Table Visualizer

For each ETS table the following information is displayed:

  • Id. The identifier of the table.

  • Name. The name of the table.

  • Type. The table type (set, ordered_set, bag or duplicate_bag).

  • Objects. The number of objects inserted in the table.

  • Size (words). The number of words allocated to the table.

  • Owner pid. The pid of the owner of the table.

  • Owner name. The registered name of the owner process of the table.

Configuration possibilities are:

  • Refresh interval in seconds. Time interval (in seconds) between each update of the output.

  • Order by column. Identifies what information to sort by.

  • Order direction. Identifies which direction to sort by (Ascending/Descending).

By clicking on the id of the ETS table, the contents of the table can be retrieved.

This service limits the size of the data that it displays. By default:

  • Only the first 200 ETS tables are shown.
  • Only the first 100 rows are shown in an ETS table.
  • Only the first 10 columns are shown in an ETS table.

These values can be modified in wombat.config:

1
2
3
  {set, wo_plugins, plugins, observer_ets, table_count_limit, 200}.
  {set, wo_plugins, plugins, observer_ets, row_limit, 100}.
  {set, wo_plugins, plugins, observer_ets, col_limit, 10}.

Executor

Real commands (e.g. Garbage Collect Processes, Evaluate Erlang/Elixir expressions). Executed for their side-effects. May not have any meaningful result as output. As these services give much freedom to the WombatOAM users to modify the monitored nodes for safety reasons it might be useful to disable all executors by adding the following line to the wombat.config.

1
{set, wo_services, enable_executor, false}.

The following services are provided:

  • Evaluate Erlang expression. It evaluates the user given Erlang expression on the managed node and returns the result.

  • Soft purge modules. Soft purge all modules from the managed node (see code:soft_purge/1).

  • Garbage Collect Processes . Force garbage collection on all processes in a controlled way. It asks the VM to perform garbage collection on GcGroupSize processes sequentially. Then, it sleeps for GcSleepTime milliseconds to weaken the negative effect of GC on the managed node.

  • Terminate process. Sends an exit signal with the given exit Reason to a process. The process can be given by PID or registered name.

  • Terminate shell processes. Sends an exit signal with the kill reason to all processes currently executing a function from the shell module. This kills all shells and the processes linked to them.

  • Recover from partitioned mnesia. If mnesia running on the current managed node is partitioned, this service restarts mnesia on all other nodes running this distributed mnesia and configures those nodes to load the table data from the current managed node to achieve consistent data in the tables. Warning: if there's different data in the tables in the other nodes (i.e. data written into the tables after mnesia was partitioned), that data is lost. Select carefully the node where you execute this service as the data of that node will be replicated.

The following additional service is provided on Elixir nodes:

  • Evaluate Elixir expression. It evaluates the user given Elixir expression on the managed node and returns the result.