Setting Up and Managing Xi-Batch Command Interpreters

What a Command Interpreter Is

Every Xi-Batch job runs under a command interpreter: a named entry that records the program the scheduler runs on the job's behalf, the arguments that program is given first, and two defaults a new job inherits. The scheduler pipes the text of the job to that program's standard input, so any program that reads its instructions from standard input can be one.

The entries live in a single file, cifile, in the batch spool directory - by default /var/spool/xi/batch. It holds one fixed-length binary record per interpreter. It is a binary file, and the way to read or change it is through the product's own commands: btcilist, btcichange, and the interpreter screen in btq.

The first entry is the standard shell. It cannot be deleted, and it is what a job falls back to whenever the interpreter it names cannot be found. On a new installation the scheduler creates the file with that one entry:

• Name: sh
• Program: the SHELL setting from /etc/xi/batchconfig, which the installation sets to the shell found on the machine, usually /bin/sh
• Arguments: -s
• Nice: 24
• Load level: the batch account's special create load level, 1000 unless it has been changed

When a Change Takes Effect

An interpreter added or altered while the scheduler is running takes effect without a restart and without a signal.

The scheduler opens cifile once, when it starts. After that it re-reads the whole file whenever it has to resolve an interpreter name and the file's modification time has moved on - once for every job it starts, and on every pass over the queue for jobs owned by another host. btcichange and the btq screen write the changed record back into the same file, so the next job to start sees the new value.

Two limits are worth knowing.

The re-read follows the file the scheduler opened, not the path. If cifile is replaced rather than edited - restored from a backup by moving a file into place, or written by any tool that creates a new file and renames it over the old one - the running scheduler and its execution processes go on reading the file they already had, indefinitely, and nothing reports it. Restore through btcichange, or through the script xb-ciconv produces, or stop the scheduler first.

The modification time is recorded to the second. A second change made within the same second as the previous read is not seen until some later change moves the time on again. This shows up only in scripts that make several changes in quick succession.

What Must Be In Place Before the Scheduler Starts

The scheduler reads cifile before it reads the saved job file, and it checks every job it loads against the list. A job whose interpreter name is absent at that moment is silently reassigned to the first entry, keeping its own load level, and the reassignment is saved with the job. Nothing is written to any log.

If cifile is missing, or its length is not a whole number of records, the scheduler recreates it with the single sh entry described above - and then reassigns every job to it.

So on a restore or a migration cifile must be back in place before btstart is run. Putting it back afterwards restores the list but not the jobs' interpreter assignments; those have to be reapplied one job at a time with btjchange -i.

The Fields of an Entry

Name

How jobs refer to the interpreter, and how it is identified on the command line.

Up to 15 characters, and unique within the file. A longer name given to btcichange -n is truncated without a message.

Program

The full path of the program to run, up to 75 characters. It must begin with a slash: the scheduler runs it directly, no search path applies, and no environment variable or ~ construct is expanded.

The job's script arrives on the program's standard input.

Arguments

A predefined argument list, up to 27 characters in total, placed before any arguments the job itself supplies.

Arguments are separated by white space, and a backslash escapes the next character, so a space or a quote can be carried inside one argument.

-s is the usual value for a shell. It stops the shell treating the job's first argument as the name of a script file.

Load level

The load level given to a job created under this interpreter, in the range 1 to 65535. A user without the special create privilege gets this value and cannot alter it.

See the load levels article for what the number controls.

Nice

A number from 0 to 39; 24 by default.

The scheduler runs at the most favourable priority the system will give it and adds this number, so the value behaves as a position on the traditional 0-to-39 scale on which 20 is an ordinary interactive process. 24 is therefore four steps below interactive, and lower numbers mean higher priority.

The same value is applied to any process the job starts through an input or output redirection.

Argument 0

Off, the default: argument 0 of the interpreter is the interpreter's name, which is what ps shows for the job.

On: argument 0 is the job's title instead, when the job has one. Some programs are confused when argument 0 differs from their own name, which is why this is optional.

Expand args

Off, the default: the job's arguments are passed to the interpreter on its command line, and the interpreter's own rules for $1, $2 and so on apply inside the script.

On: Xi-Batch substitutes $1 to $n, $* and $@ in the text of the script as it feeds the script to the interpreter, and the arguments are not placed on the command line at all. This exists for interpreters that have no positional parameters of their own.

Turn it on only for an interpreter that needs it. With it on the substitution is performed by the scheduler rather than by the interpreter, before the job's own identity is applied, so leaving it off wherever it is not required is the safer setting - and for a shell it is also the correct one, because a shell identifies syntax before it expands variables and pre-substituted quotes and special characters will not mean what they appear to mean.

Listing the Interpreters

btcilist prints the list. It takes no arguments and needs no privilege.

btcilist

There is no heading line. The columns are name, program, load level, nice value and predefined arguments, in that order; the load level and nice value are printed in adjacent fixed-width fields. An interpreter with expand args set is marked $N and one with argument 0 set is marked Set a0.

sh   /bin/sh        1000 24 -s
ksh  /bin/ksh       1000 24 -s
perl /usr/bin/perl  1000 24 -
awk  /usr/bin/awk   1000 24 -f - $N

To see the list held by another host:

btcilist -Q otherhost

In btq, press X from the jobs screen for the same list in editable form, and q to return.

Adding, Changing and Deleting from the Command Line

btcichange creates, alters and deletes entries. The interpreter's name is always the last argument, and only one interpreter can be operated on at a time. The command requires the special create privilege; without it the command exits with

You need "special create" permission to update command
interpreters - sorry.

Add a Korn shell interpreter:

btcichange -A -p /bin/ksh -a "-s" ksh

With an explicit load level and nice value:

btcichange -A -N 25 -L 1500 -p /bin/ksh -a "-s" ksh

Change an existing one:

btcichange -L 1500 sh
btcichange -N 20 sh
btcichange -a "-s --" sh

Rename one:

btcichange -n korn ksh

Delete one:

btcichange -D old_interp

Things btcichange does not check.

• It does not verify that the program path exists or is executable. A wrong path is accepted, and the failure appears only when a job tries to run.
• A path that does not start with a slash is discarded rather than rejected. On an add the command then stops with "no path name set for new command interpreter"; on an update the option is simply ignored and the existing path is left alone.
• It does not check whether any job is using an interpreter before deleting it. The btq screen does; the command line does not.
• Renaming does not find or update jobs that refer to the old name. Those jobs are then in the position described under "What happens when the interpreter is not there".
• The first entry cannot be deleted: "you cannot delete the default shell name."

If -L is omitted on an add, the new interpreter takes the load level from the special create load level in the profile of the user running the command, which btulist shows as Specll. If -N is omitted it takes 24.

Editing the List in btq

Press X from the jobs screen. The keys on that screen are:

• A - add a new interpreter (it prompts for the name, then the path)
• D - delete the one under the cursor
• N - change the name
• P - change the program path
• a - change the predefined arguments
• L - change the load level
• n - change the nice value
• 0 (zero) - toggle argument 0
• E - toggle expand args
• j k - move down and up, or use the arrow keys
• ? - help, q - leave the screen

Every change is written to the file as it is made; there is no separate save step and no way to abandon a set of edits. Without the special create privilege the screen is read-only and any change key reports a permission error.

btq checks a path more thoroughly than the command line does: it must be absolute, must exist, must be a regular file, and must be executable by somebody. btq also refuses to delete an interpreter that a queued job is using.

How a Job Chooses an Interpreter

At submission:

btr -i ksh script.ksh

On a job already in the queue:

btjchange -i ksh <job_number>

Or in btq, select the job on the jobs screen and press x.

A job submitted without -i gets the first entry in the list and that entry's load level.

Order matters on the command line. -i sets the job's load level from the interpreter, so a -l given before -i is overwritten by it. Where a job needs both, put -l after -i:

btr -i heavy -l 5000 process-warehouse.sh

An unknown name is refused when the job is submitted or changed:

btr: Unknown command interp `nosuch'

What Happens When the Interpreter Is Not There

Four different things, depending on when the name is resolved.

At submission or change

The command is refused with the message above and the job is unchanged.

When the scheduler starts

Every job naming an interpreter that is not in cifile is silently reassigned to the first entry, keeping its own load level.

When a local job runs

The job is selected normally, and the execution process cannot resolve the name. It exits with status 248 before the job's own output and error files are created, so nothing is written anywhere. The job is recorded as an error - progress Err - and, if it is a repeating job, it stays in that state and never runs again until btjchange -N resets its progress code.

When a job owned by another host is considered

The scheduler checks the interpreter name against its own list on every pass and skips the job if it is absent. Nothing is displayed and nothing is logged; the job simply never starts here.

This is the case for deleting an interpreter only after checking what still refers to it:

btjlist -N -F "%N|%I" | grep '|old_interp$'

Common Shell Setups

The -s argument stops a shell treating the job's first argument as a script file name; it is wanted on all three.

btcichange -A -p /bin/sh  -a "-s" sh_alt
btcichange -A -p /bin/ksh -a "-s" ksh
btcichange -A -p /bin/csh -a "-s" csh

The program path differs between platforms - /usr/bin/ksh on some systems, /bin/ksh on others - so check the path on the machine before adding the entry, since btcichange will not.

Interpreters That Are Not Shells

Perl reads its program from standard input when the file argument is a single minus sign, and passes anything after that to the script:

btcichange -A -p /usr/bin/perl -a "-" perl
btr -i perl process.pl

awk reads its program from standard input with -f -, leaving the job's arguments as its data files:

btcichange -A -p /usr/bin/awk -a "-f -" awk
btr -i awk -a datafile getxi.awk

Any other program that reads instructions from standard input can be set up the same way. The one thing to establish first is how the program is told to read standard input, because that is what goes in the predefined arguments.

Several Entries for One Program

The same program may appear in more than one entry, with different nice values or different load levels. This is the mechanism for classifying work, because a user without the special create privilege takes the interpreter's load level and cannot override it.

btcichange -A -p /bin/sh -a "-s" -N 30 -L 1000 sh_low
btcichange -A -p /bin/sh -a "-s" -N 10 -L 1000 sh_high
btcichange -A -p /bin/sh -a "-s" -L 500  light
btcichange -A -p /bin/sh -a "-s" -L 5000 heavy

btr -i sh_low  cleanup.sh
btr -i sh_high urgent.sh
btr -i light   generate-report.sh
btr -i heavy   process-warehouse.sh

What the load level numbers should be is a question about the system as a whole rather than about interpreters; the load levels article covers it. The numbers above are placeholders, not recommendations.

Wrapper Interpreters

An interpreter does not have to be a language. A shell script that reads the job on its standard input, does something with it and hands it on is a legitimate entry, and is how sites add logging or a dry-run mode.

An interpreter is a program the scheduler executes on a job's behalf, under the identity of the job's owner, for every job that names it. A wrapper therefore has the same standing as any other program run on a schedule, and the same rules apply:

• Put it in a directory that only administrators can write to - the same standing as /usr/local/bin, not a shared or group-writable working directory, and never a directory writable by all users.
• Own it as root or as the batch account and leave it not writable by group or other. Anyone who can rewrite the file controls what every job using that interpreter runs.
• Give the entry a path under that directory rather than a path that resolves through a symbolic link somebody else can replace.
• Have it pass the job's exit status through unchanged, or every job under it will be recorded as having succeeded.

The privilege to add or change an interpreter is special create, and the same privilege lets a user set a job's load level directly. Grant it to administrators only, and review the list periodically - btcilist shows every entry and its program path in one screen.

Backing Up and Restoring the List

cifile is a binary file and is not portable between machines. The supported way to capture it is xb-ciconv, which reads the file and writes a shell script of btcichange commands that recreates the entries:

xb-ciconv -D /var/spool/xi/batch cifile cifile.sh

The script it produces needs the special create privilege to run, and because it works through btcichange it can be run against a scheduler that is already up. That is the route to prefer over copying the binary file back, for the reason given under "When a change takes effect".

Good Practice

Use names that say what the entry is for. The name is the only thing a job records, and it is what appears in a listing: sh, ksh, perl, reports, updates. Avoid interp1 and test.

Add an entry only when it does something the existing ones do not - a different program, a different nice value, or a different load level. Each entry is another name that has to exist on every machine a job might run on.

Check the path before adding the entry, since neither btcichange nor the scheduler will tell you it is wrong until a job fails.

Test a new interpreter with a job of its own before pointing production work at it:

btr -i new_interp test.sh

Keep the same names on every machine in a network. A job owned by another host is skipped, silently, if the interpreter it names does not exist here.

Record the list outside the spool directory. An xb-ciconv script kept with the rest of the system configuration is both the backup and the documentation.