Sizing the Xi-Text Shared Memory Queue

What the shared memory is for

Xi-Text keeps its live queue in System V shared memory so that every command can read it without going through the scheduler. The segments are created when the scheduler starts and removed when it stops cleanly.

In normal operation there are three:

• a fixed-size job information segment holding the queue header and the job hash tables;
• a job data segment holding one slot per job;
• a printer segment holding the printer table.

A fourth appears briefly while a segment is being enlarged, and disappears again when the copy is finished.

The segments are sized at start-up, and they do grow

The scheduler allocates a number of slots - jobs or printers - and the byte size of the segment follows from that number. When the slots run out, the scheduler creates a larger segment at the next key, copies the contents across and removes the old one. Growth is a normal, routine operation and not a last resort.

Growth is nevertheless the thing to size against rather than to rely on. It copies the whole segment while the queue is locked, and - see What happens when the slots run out below - Xi-Text has no way to decline a request it cannot fit.

Setting the initial allocation

spstart takes two positional numbers, in this order, and passes them to the scheduler. There is no -j option and no -p option.

spstart <jobs> <printers>

The shipped service unit runs:

/usr/local/bin/spstart 2000 50

A third number is a usage error. If either number is omitted or given as zero, the compiled-in default is used instead.

Making the setting permanent

Editing the service unit or the start-up script works, but the setting the product itself regards as the site's own is a system variable in the master configuration file, /etc/xi/textconfig. That file is read by every program, so the value applies however the scheduler happens to be started.

Each line is a name, an equals sign and a value:

NUMJOBS = 20000
NUMPTRS = 200

The command-line values override the file, so a service unit that passes numbers will win over anything set here. Set the numbers in one place, not both.

The defaults, and why they may not be what you expect

The compiled-in defaults are considerably larger than the numbers the shipped service unit passes: 100,000 jobs and 5,000 printers. Starting the scheduler with no numbers at all does not give a small queue.

One further rule: a saved queue larger than the number you asked for raises the allocation. At start-up the scheduler sizes the segment from the saved queue file if that is bigger than your figure, so an installation recovering a large queue does not silently truncate it.

How much memory this actually costs

A slot is a fixed-size structure, so the segment is roughly the slot count multiplied by the slot size, rounded up. Read the real figures off your own system rather than estimating them - the bytes column of ipcs -m is the actual allocation:

ipcs -m

Xi-Text's segments have keys beginning 0x58691.

The size column is a Linux one. Solaris, AIX and HP-UX print only the type, identifier, key, mode, owner and group by default; add -b there to get the segment size. ipcs -mb shows the size on all four platforms.

In practical terms, on any machine with a few gigabytes of memory the allocation is small enough that sizing it to cover your peak comfortably costs nothing worth measuring. Sizing it below your peak costs a queue-locking copy at best. Set it generously - but set it from your own peak, not from a table.

What grows, when, and by how much

The default growth steps are large, because they are derived from the large compiled-in defaults rather than from the number you passed. A site that starts with spstart 2000 50 and fills it does not grow to 2,100; it grows by the full INCJOBS step. Set INCJOBS explicitly if that matters to you.

Xi-Text grows silently. Nothing is written to /var/spool/xi/spd/spshed_reps when a segment is enlarged, so there is no record to search for and the only evidence is a changed segment size in ipcs -m.

What happens when the slots run out

This is the reason to size Xi-Text conservatively. A failed allocation is fatal. There is no "queue full" refusal on the job side; if a segment cannot be created or enlarged, the scheduler writes a message to /var/spool/xi/spd/spshed_reps and exits, taking printing with it. The messages to look for are:

Trouble creating shared memory id ...
Possibly some other software is using it all up, or the maximum shared
memory size allocated in your kernel is too small.

Trouble attaching job data shared memory - ...
Trouble attaching printer shared memory - ...
Failed to create job info shm segment - ...

The last of those is also the message you get when the previous shutdown was not clean and the segment is still present. Clear it with xt-ripc -d before restarting.

Growth is a safety net, not a plan

In the shipped release, client programs do not reliably follow every kind of segment growth. Specifically, the address published for readers after a growth is not always the address of the new segment; on Xi-Text this affects the job data segment. The observable effect is that after a growth of that segment, commands that read the queue may show stale contents or fail to open it, until the scheduler is restarted.

The practical consequence is a simple rule and it is the main advice in this article:

Set the initial allocation to cover your peak, so that the segments never need to grow in normal running. Restarting the scheduler with a larger initial figure is the reliable way to enlarge a queue; growth is what stops a busy day turning into an outage, not what you plan around.

Checking what is allocated now

List the segments:

ipcs -m

Identify them by key: 0x58691003, 0x58691004 and 0x58691005. Keys move up by two while a segment is being replaced, so a segment may appear at the next key up after a growth.

Filter by key, not by owner. The segments are owned by root, not by spooler, so an owner filter finds nothing:

ipcs -m | grep 0x58691

The IPC tool reports the same thing more usefully, including which segment is which:

xt-ripc

Check what the scheduler was started with:

ps -ef | grep spshed

Kernel parameters

Xi-Text does not read SHMMAX, SHMALL or SHMMNI, and it has no tuning file of its own. On a current Linux kernel the shared-memory limits are effectively unlimited by default and there is nothing to change; the allocations are megabytes, not gigabytes.

Where a limit does bite - an older kernel, or a system where an administrator has lowered the limits deliberately - the behaviour is worth knowing:

• If the kernel rejects the size as too large, the scheduler halves its request and tries again. You end up with fewer slots than you asked for, with no message.
• If the kernel is out of memory or out of segments, the request fails outright and the scheduler exits.
• If a segment already exists at the key, the scheduler steps to the next key and retries, up to fifty times. Stale segments from an unclean shutdown therefore do not usually block a start - but they do confuse the client programs, so clear them with xt-ripc -d.

The other Unix platforms Xi-Text supports - Solaris, AIX and HP-UX - do have shared-memory limits that can be set low, and there the kernel's own documentation is the place to raise them.

Changing the allocation

A change to the initial allocation takes effect at the next start of the scheduler. Stop the product, change the figure, start it again.

sstop -y
spstart 20000 300

Make the same change in /etc/xi/textconfig or in the service unit, or the next reboot will go back to the old figure.

Summary

• spstart <jobs> <printers>, or NUMJOBS and NUMPTRS in /etc/xi/textconfig.
• Size for your peak. The memory cost of a generous figure is negligible; the cost of a segment growing under load is not.
• Xi-Text grows silently, and exits if it cannot grow. There is no queue-full refusal.
• Identify the segments by key in ipcs -m, or with xt-ripc. An owner filter does not work.