Choria Blog

It’s been a while since we had release announcements and it’s been a surprisingly busy period.

The main focus here has been on a number of stability and bug fixes, we’ve had some users dig in really deep into various aspects of the system and a number of bugs were squashed.

Past the quality of life stuff I have started reworking Choria Server Provisioning which will set us on a path to having a good Puppet free story, I have some POCs lying around of a Kubernetes based Broker, CA, and Provisioner that will give a really smooth path forward - provisioning is now compiled in to the FOSS stack by default and can be enabled using a JWT token, more on that in a future post.

We also include a Tech Preview of NATS JetStream support and significantly moved our event formats over to Cloud Events v1.0 format.

Thanks especially go to Alexander Hermes for his deep dive into all aspects of the client side playbooks. Deep dives into a product and filing some tickets, discussing the model on slack etc it hugely time consuming and very often this kind of community contribution flies under the radar but I find it more valuable than code, huge props to Alexander.

Other shout outs to Ben Robert, Yury Bushmelev, Romain Tartière and Vincent Janelle

The upcoming Choria Server release will support publishing fleet data to JetStream a new Streaming Server from the folks who built NATS - the core technology of the Choria Broker.

This technology is currently in tech preview and so we’ve added early support for it so Choria users can test it for their needs.

Support for JetStream will be in the nightly build tonight and a release will be soon.

Read on for some background on the why and how.

When we first introduced our Lifecycle Events we mentioned efforts by cloudevents.io to standardize a format and set of transports but that it was a bit early in their life to adopt it.

Cloud Events is now a CNCF Incubator project and they reached CloudEvents 1.0 which means there is now a standard to work around that is stable. A good introduction to the concept can be found in The CloudEvents Spec Seeks to Bring Uniformity to Event Data Descriptions.

Since then adoption of this format has exploded - a whole bunch of companies and projects support ingesting and routing CloudEvents, examples: solo.io with their event powered API gateway, OpenFaaS, Serverless.com Event Gateway, Oracle Cloud, Azure, Knative and many many more.

So for us the time is ripe to adjust the format of our events to support CloudEvents 1.0. As of the next release of the Choria Server the Life Cycle events and Autonomous Agent Events are all been transitioned to CloudEvents format.

The latest release of Choria includes support for external agents, so now writing agents in any language is possible. For those of us who prefer to use Python to manage our systems, we have a new library, py-mco-agent which is intended to make it as simple as possible to write Choria agents. The simplest possible agent can be written with just a few lines of code.

Today I have a small few bug fixes to ship, these will affect only people who are experimenting or using our newly announced External Agents support, others can safely ignore this.

Thanks to Ben Roberts for his assistance with these releases

Today we are doing a series of long overdue Puppet Module releases as well as a reasonably significant Choria Server update.

Significantly this delivers the new External Agents I blogged about recently.

Read on for all the details!

MCollective has for a long time been extendible using purely Ruby means. This was fine in an earlier age where it seemed Ruby was going to rule the ops space but turned out that isn’t how things ended up, and even then there were lots of real interest in extending it using Python for example.

In the last 2 weeks this conversation came up again and in many respects the situation was worse now since the new Choria Server, being written in Go, is not extendible by external plugins, we supported the old Ruby agents but that was it. We had plans to support Tengo and Yaegi as ways to add agents to Choria but neither of those got past POC.

There were major hurdles to actually doing this in the new Go system:

• no action_policy type authorization system
• no data plugins
• no data aggregators
• limited DDLs
• do not have inputs and outputs properly mapped
• have no DDL validators
• have no way to set default data in requests

These missing features all worked fine for the Ruby agents though - since the MCollective compatability layer would just start up a subset of old MCollective on demand and so gain access to its versions of all of the above features. We did not need any of it.

In the last 3 weeks we addressed most of these missing features in the pure go daemon:

• We have shellsafe, ipv4address, ipv6address, ipaddress and regex validators. Analysis showed this hits 90%+ of what was ever used
• We have data aggregators for average, summary (including booleans) and a new chart one. Analysis showed these would hit 90%+ of what was ever used
• We have greatly extended the DDLs with full input, output and aggregate awareness
• We can set defaults in both inputs and outputs and it’s all type aware
• We can do full validation of requests based on the DDLs
• We have a action_policy plugin thats 1:1 compatible except for compound statements (planned)
• We can generate ruby DDLs from JSON ones

You’ll notice we are making deliberate choices about what we support here rather than expose a 100 extension points to the user - over extendibility was a real problem in the past - we now favour a batteries included approach where what people need is always available, new validators or aggregators would be via PR submission to the Go code and everyone benefit.

Above points represent a huge push in features, at this point if we add new ways to write agents they would get all these features for free and suddenly the prospect of doing just that is a lot more palatable. But why stop at supporting a specific language like tengo? Why not support all languages - especially with new movement in things like webasm?

That’s exactly what I did in a new feature called External Agents and it will be available in the next release, read on for the full details.

DDL files have been a utter bane of existence for MCollective users - and since Choria provide a compatability layer likewise for Choria users. It’s not limited to Choria though most remote invocation systems have these kinds of files and honestly they are all horrible - think WSDL, OpenAPI/Swagger etc, editing and maintaining any of these is really sub optimal.

I have some plans to improve the situation, as much as we reasonably can, the first of which will land in our next release - 0.12.1 - which revolves around generating these files interactively.

The next releases will start coming in over the next week or three, we’re getting going with quite a major release for the Choria Server and Broker and a few related packages, I’ll introduce some of the changes here today.

Choria Release 0.12.0 is available today, you can get it by updating your Hiera data choria::version.

The latest release of Choria will have a new security provider, called pkcs11! This blog post will go over how to use it in various configurations. But first, a review of what pkcs11 is and how it’s useful.

What is pkcs11?

pkcs11 stands for “Public Key Cryptography Standard #11”. It’s a set of standards for how to interact with a cryptographic token. You may have heard of HSMs or smart cards. pkcs11 is how software interacts with these things.

Why should I use pkcs11?

You may be compelled to use it due to the environment you work in. Yubikeys and CACs are being used more and more in large-scale environments. But it’s a good idea to investigate the use of these things if you already aren’t. The power of HSMs is that the sensitive cryptographic material is generated on the hardware and never leaves it. So instead of opening your private key file and signing hashes with it, you’re handing the hash to your Yubikey, which signs it and returns the data. There are compliance advantages too (because of the stronger security). Some HSMs are FIPS-compliant, which some computing environments require.