Tag Archives: Distributed Agents

Distributed Agent improvements in CS 10.X U 201506

As I had promised in a linked-in status update, I would write something about the changes to distributed agents in the latest updates from Open Text.  Over the weekend, I installed the latest 10.0 release, and today I installed the latest 10.5 release.  At first, it wasn’t obvious what the change was between them. However, the change is there, under System Administration        –>Distributed Agent Status.  Unfortunately it is only available in the 10.5 release of U201506, not the 10.0 release.   On this page, which previously just listed the status of the worker threads, you can now set a black-out period.  So what does this do for us exactly?

The handy thing about specifying a black-out period for DA’s is that the ones which are the most process intense won’t be consuming your system resources during business hours when you want Content Server to be at its most responsive. The main DA’s that come with Content Server are the ones that rebuild your facet and custom columns tables, and the ones that purge deleted items from DTree (now called DTreeCore) – all operations that are potentially database intensive.

Presumably, a task that is running would not suddenly halt when the outage window time arrives but rather finish what it’s doing and exit.  This should be OK as DA tasks are intended to be short (i.e. 1-10 seconds in duration).  What is not clear is whether a job chain would simply pick up where it left off, or would the job need to be rescheduled. If it is the former, that would be pretty brilliant, and this would have little or no impact on 3rd party vendors.

This is a good feature, but the one drawback is that it is all or nothing.  That is, it prevents all DA tasks from running during an outage period. It is conceivable that facet and column updates would need to continue during the day while the purge tasks could be deferred to after hours. It is conceivable that a system administrator may want to prevent facets from updating but need columns to update during business hours (or vice versa).  Admittedly this is more of an issue for any 3rd party Technology Partner that develops a module for Content Server that uses Distributed Agents.

Prior to the introduction of this outage window for DA’s, partner developers would need to introduce their own semaphores to manage any outages such as not running during business hours. In my company’s product, BMUP, outages are managed through an administration screen which stores a weekly configuration in the KINI table, and a job table that keeps track of each DA task that is launched by user actions. Any task that is launched during a time when DA tasks aren’t supposed to run would be scheduled for later. For any job chain that was still running, each individual task would detect the outage, exit, then the master task would reschedule itself to carry on later. For the core DA job chains that Open Text provides, this level of control is probably overkill. After all, the column & facet jobs are automatically spawned, triggered by changes in Content Server.

Perhaps this is already in the works, but the next change from Open Text to the DA framework, might be the ability to increase the granularity so that particular task types  can have different outage periods.

I will be later writing more about programming with Distributed Agents, particularly because it is in my view an underrated and underutilized framework for doing some really good scalable development in Content Server. Kudos to OpenText for allowing DA’s to be restricted to not run during pre-defined periods.