External Agents

Choria Server supports External Agents, these let you write agents hosted by the Choria Server in any language. These agents are forked on demand and receive the request in a temporary file and write their reply in a temporary file as JSON.

Overview

File Layout

An external agent can be written in any language and the related files go in the same directory as other Ruby agents, here is a basic helloworld agent on disk in /opt/puppetlabs/mcollective/plugins/mcollective/agent:

-rwxr-xr-x 1 root root 315 Sep 12 11:18 helloworld
-rw-r--r-- 1 root root 819 Sep 12 10:40 helloworld.ddl
-rw-r--r-- 1 root root 915 Sep 12 10:40 helloworld.json

For compiled agents, like those written in Go, we support multi-arch binary selection.

$ find agents
agents
agents/helloworld.json
agents/helloworld.ddl
agents/helloworld
agents/helloworld/helloworld-darwin_amd64
agents/helloworld/helloworld-linux_amd64
agents/helloworld/helloworld-freebsd_amd64

Here we will pick the binary based on the go GOOS and GOARCH variables.

Both the DDL and JSON files are required and can be generated using the new choria plugin generate ddl helloworld.json helloworld.ddl utility.

Communication Protocols

Communication from the Choria Server to your agent is done via files on disk and environment variables. When run these variables will be set:

Your agent will also be called with 3 arguments:

helloworld <request file> <reply file> <protocol>

The working directory will be your OS temporary directory.

Logging

Any output your agent produces on STDOUT is logged at INFO level on the server. Any STDERR output is logged as ERROR level on the server.

Distribution

These modules can be packaged and distributed using the standard Plugin Packaging for Puppet.

Activation

At server start your agent will be called with an activation check, this gives you the chance to verify your dependencies and decide if the agent should be active on the particular node.

The CHORIA_EXTERNAL_PROTOCOL will be set to io.choria.mcorpc.external.v1.activation_request, you should only respond to activation checks when this is set.

The CHORIA_EXTERNAL_REQUEST file will look like this:

{
  "$schema": "https://choria.io/schemas/mcorpc/external/v1/activation_request.json",
  "protocol": "io.choria.mcorpc.external.v1.activation_request",
  "agent": "helloworld"
}

And you should reply with this in the CHORIA_EXTERNAL_REPLY:

{
  "activate": true
}

Any reply not in this format or non 0 exit code from your code will result in the agent not being activated on this node. Your program will have maximum of 2 seconds to process an activation check.

Requests

Once activated any requests sent to your agent will result in CHORIA_EXTERNAL_PROTOCOL set to io.choria.mcorpc.external.v1.rpc_request.

The CHORIA_EXTERNAL_REQUEST file will look like this:

{
  "$schema": "https://choria.io/schemas/mcorpc/external/v1/rpc_request.json",
  "protocol": "io.choria.mcorpc.external.v1.rpc_request",
  "agent": "helloworld",
  "action": "ping",
  "requestid": "034c527089f746248822ada8a145f499",
  "senderid": "dev1.devco.net",
  "callerid": "choria=rip.mcollective",
  "collective": "mcollective",
  "ttl": 60,
  "msgtime": 1568281519,
  "data": {
    "msg": "hello"
  }
}

Few things to note here:

• requestid is unique per request and will appear in audit logs and elsewhere
• callerid will have been verified by the security system so you can rely on this being valid
• msgtime is seconds since 1970 in UTC timezone
• data is free form data, typically key=val of whatever the client sent
• data will have been verified subject to the DDL validations, data types and default values set as appropriate

Your reply should be written to CHORIA_EXTERNAL_REPLY and must look like this:

{
  "statuscode": 0,
  "statusmsg": "OK",
  "data": {
    "result": "hello"
  }
}

The statuscode is standard MCollective Protocol status codes:

And data is free form data being sent back to the client. Note however the data too will be validated against the DDL output section, defaults will be set and more.

Helper Libraries

We hope to see helper libraries for various languages from the community, we’ll maintain a list of some here:

• Python - Contributed by Ben Roberts as part of working on the early POC
• Golang - Dependency free Golang package, part of the Choria project