     [file version.texi not included]

This is the reference manual for the lyskomd LysKOM server version VERSION.

Copyright © 1995-2005 Lysator ACS.

Permission is granted to make and distribute verbatim copies of this specification provided the copyright notice and this permission notice are preserved on all copies.

lyskomd
*******

lyskomd is a server for the LysKOM conferencing system. This info file documents version VERSION of lyskomd.

Copying
*******

lyskomd is free software. It is distributed under the Gnu General Public License version 2. The file COPYING in the top level of the distribution contains the text of the license.

Overview
********

LysKOM is a conferencing system (Or in modern terms, enabling technology for Computer-Supported Cooperative Work (CSCW).). Similar systems were QZ-KOM and PortaCOM (Also known as ``PottaKOM'' and ``BortaKOM''.). The LysKOM system is copyrighted by Lysator Academic Computing Society and distributed under conditions of the GNU General Public License version 2. LysKOM and its documentation is provided ``as is'' without warranty of any kind.

This reference manual documents version VERSION of the lyskomd LysKOM server. The lyskomd server is the work of several people. The main contributors have been Per Cederqvist ceder@lysator.liu.se, Inge Wallin inge@lysator.liu.se, Thomas Bellman bellman@lysator.liu.se, David Byers byers@lysator.liu.se and Peter Eriksson pen@lysator.liu.se.

History
=======

In 1990, Per Cederqvist ceder@lysator.liu.se and Peter Eriksson pen@lysator.liu.se and a few other persons started to write the server. It was operational in the summer of 1990, even though the members of Lysator discovered a thing called MUD. We started using RCS on 20 May 1991. The first release was made on 16 Sept 1991. Around that time we switched from RCS to CVS to handle our source code.

Installation
************

Instructions for compiling and installing lyskomd are in the files README and INSTALL, located in the top level of the lyskomd distribution.  Installation should be straightforward on most platforms.

Configuration
*************

There are two configuration files for lyskomd. One defines the server options and the other defines aux-item types The Aux-Item List.

Server Configuration File
=========================

The server reads its configuration from a configuration file. The default configuration file is /usr/lyskom/etc/config. The location of the configuration file can be changed at run-time by supplying an argument to lyskomd.

The configuration file is line oriented. Each line consists of a parameter name followed by a colon, and the value of the parameter. Empty lines and lines whose first non-blank character is a # are ignored.

Parameter Types
---------------

Every parameter has a type. The standard types are:

bool
     The parameter can be true or false. Legal values are on, true, yes and 1 for true and off, false, no and 0 for false.

locale-name
     The parameter is a locale name. The value must be a legal locale name of the system where lyskomd is running.

path
     The parameter is a path name. The value must be a legal path on the system where lyskomd is running. Most paths you can specify can be either absolute paths (if they begin with a /) or paths relative to the installation prefix which is specified at compile time and can be overridden by the Prefix: parameter in the configuration file.

portname
     The parameter is a TCP/IP port. It can be a symbolic port name (traditionally looked up in /etc/services) or a port number.

int
     The parameter is a number of some sort. It can be a conference number, text number or perhaps something else.

double
     The parameter is a floating point number.  Any syntax that the C function strtod accepts is OK.  Examples of truly portable values: 1 or 1.3.

timeval
     The parameter is a time period.  It consists of a floating point number (in the same format as for parameters of type double), optionally followed by optional whitespace and a suffix.  If no suffix is specified, it defaults to the suffix mentioned in the description of the parameter.

     Valid suffixes includes:

       * seconds

       * second

       * sec

       * s

       * minutes

       * minute

       * min

       * hours

       * hour

       * h

       * days

       * day

       * d

       * milliseconds

       * millisecond

       * m

       * microseconds

       * microsecond

       * u

A few parameters have ad-hoc types, that are used for a single parameter.  They are documented in the description of that parameter.

Parameters
----------

Max conferences: INT
     The maximum number of conferences possible in the server. This number must be larger than the number of conferences in the database. This parameter is required. There is no default.

Max texts: INT
     The maximum number of texts possible in the server. This number must be larger than the number of texts in the database. This parameter is required. There is no default.

Locale: STRING
     Use STRING as the locale to run in. This parameter is only available om systems which support the setlocale call. If this parameter is not set, no call to setlocale will be made. The default is unset.

Force ISO 8859-1: BOOL
     This option is provided for those with dysfunctional computers that cannot handle setlocale properly. If this is set, lyskomd will handle texts according to the ISO 8859-1 (latin1) alphabet. Default is off.

Prefix: PATH
     Specify the installation prefix.  All relative filenames that the server uses are interpreted relative to this directory.  The default value of this parameter is set at compile time.  The default is /usr/lyskom, but it can be changed by the --prefix argument of configure at compile time.

Send async: BOOL
     Do not send any non-requested messages. This disables the sending of messages about events in the server to all connections. Use of this parameter is not recommended. Default is on.

Listen: PORTNAME
Listen: HOSTNAME:PORTNAME

     Specify which IP number and port the server should use when listening for new clients.  HOSTNAME may be a FQDN (such as kom.lysator.liu.se), an IPv4 address (such as 10.0.0.1), or an IPv6 address enclosed in brackets (such as [2001:db8::a00:20ff:fea7:ccea].  If a FQDN resolves to several IP addresses, it is undefined if the server will listen to all of them or just pick one of them, so avoid that situation.  If the PORTNAME is the empty string, the default value of 4894 will be used.

     The default is 4894 (listen to port 4894 on all interfaces).

     These parameters can be specified several times to cause the server to listen to many ports and/or interfaces.  The default is removed if this parameter is specified at least once.

Presentation of conferences: INT
     The number of the conference where presentations should be sent. Defaults to 1. This option is ignored in lyskomd 1.9 and later. Set this using dbck or the set-info.

Presentation of persons: INT
     The number of the conference where presentations should be sent. Defaults to 2. This option is ignored in lyskomd 1.9 and later. Set this using dbck or the set-info.

Motd-conference: INT
     The number of the conference where "message-of-the-day" messages should be sent. Defaults to 3. This option is ignored in lyskomd 1.9 and later. Set this using dbck or the set-info.

News-conference: INT
     The number of the conference where news of interest to the readers of this LysKOM server should be written. This is typically a conference with very low traffic which everyone shoule be a member of. Clients should offer new users to join it. Defaults to 4. This option is ignored in lyskomd 1.9 and later. Set this using dbck or the set-info.

Message of the day: INT
     Default message-of-the-day of this server. The text will be shown automatically by conforming LysKOM clients when a user logs on. This option is ignored in lyskomd 1.9 and later. Set this using dbck or the set-info.

Garb: BOOL
     Should the database be automatically purged of old texts?  The default is on.

Never save: BOOL
     Completely disables saving the database. Do not set this to true unless you really know what you're doing. The default is false.

Log accesses: PATH
     This parameter can only be set if the server has been compiled with LOGACCESSES defined. It will save a trace of all activity in the database to a file, for later use in simulations et c. Compiling with LOGACCESSES slows the server down quite a lot, so it is normally not defined.

Data file: PATH
     The path relative to the installation prefix (The installation prefix can be specified at compile time, and overridden by the Prefix: parameter.) where part of the database is kept. The default is var/lyskomd/db/lyskomd-data.

Backup file: PATH
     The path relative to the installation prefix where a backup of the database is kept. This file will always contain a complete database, but it may be a little out-of-date. Default is var/lyskomd/db/lyskomd-backup.

Backup file 2: PATH
     The path relative to the installation prefix where a previous generation of the backup of the database is kept. This file may be needed if an error in the backup file is detected during the creation of the data file. Default is var/lyskomd/db/lyskomd-backup-prev.

Lock file: PATH
     Name of the lock file that ensures that dbck and lyskomd never attempt to modify the database at the same time.  It should always reside in the same directory as the Data file.  Default is var/lyskomd/db/lyskomd-lock.

Text file: PATH
     The path relative to the installation prefix where the actual texts in the database are kept. Default is var/lyskomd/db/lyskomd-texts.

Text backup file: PATH
     When dbck is run with the -g option (Invoking dbck, it will store the previous contents of the text file in the file specified by this option. The path is relative to the installation prefix. This file is never used by lyskomd itself. Default is var/lyskomd/db/lyskomd-texts-backup.

Backup export directory: PATH
     When splitkomdb is run, it will create a copy of the database in this directory.  The copy will be split in a way that helps to keep incremental backups of that directory small.  See splitkomdb.  The path is relative to the directory specified by Prefix:.  This directory is never used by lyskomd itself.  Default is var/lyskomd/exportdb.

Number file: PATH
Number temp file: PATH
     Name of the file where the first unused conference and text numbers are stored.  This file contains a single line.  It is rewritten each time a new conference or text is created, to ensure that numbers are never reused even if the server later crashes before it has time to save the database.  The information is first written to Number temp file:, and then renamed to Number file:.  The path is relative to the installation prefix.  Default is var/lyskomd/db/number.txt and var/lyskomd/db/number.tmp, respectively.  Both files must reside on the same partition.

Log file: PATH
     The path relative to the installation prefix where log messages from lyskomd are written. Default is var/lyskomd.log.

Log statistics: PATH
     Whenever lyskomd receives a SIGUSR1 it will append a timestamp and a count of how many different atomic calls have been made in this file. The path is relative to the installation prefix. Default is var/lyskomd.stats.

Pid file: PATH
     When lyskomd is up and running it will write its pid in this file. The path is relative to the installation prefix. This file is used so the updateLysKOM script can easily find out what pid the LysKOM server has. Default is var/run/lyskomd.pid.

     This file should be removed when the computer reboots, before komrunning or updateLysKOM is run.

Memory usage file: PATH
     When lyskomd exits normally it appends some info on its usage of memory to this file. The path is relative to the installation prefix. Almost any memory leak bugs should be detectable by looking in this file. Default is var/lyskomd.memory.

Aux-item definition file: PATH
     This file defines which aux-items the server should support and how it should handle them. You will find the details in see Aux-Item Definition File. The path is relative to the installation prefix. Default is etc/aux-items.conf. This file is re-read if a SIGWINCH singal is sent to the server. Warning: If the aux-item definition file contains an error so that it cannot be parsed, the server will call restart_kom(), which will cause the server to abort without saving the database.  Always test the file on a standby system first!

Core directory: PATH
     The Directory where core dumps are written. This path is relative to the installation prefix. Default is var/lyskomd.cores.

Connection status file: PATH
Connection status temp file: PATH
     Where to store a status file that contains information about all connections.  The status is written to the temp file and atomically renamed to the status file.

     The path is relative to the installation prefix.  Defaults are var/lyskomd.clients and var/lyskomd.clnt.tmp.  Both files must reside on the same file system.  See Files, for information about the file format.

Status file: PATH
     This file is created by komrunning to indicate that lyskomd should currently not be running.  When this file exists updateLysKOM will send it a SIGTERM signal, so that it saves the database and dies.  Default is var/lyskomd/db/status.

Nologin file: PATH
     If this file exists, the server will not allow any connections at all. Default is /etc/nologin.

Garb busy postponement: TIMEVAL
     How often should the garb run when the server is busy serving clients? Default is once every 50 milliseconds.

Garb timeout: TIMEVAL
     How long to sleep when the server is garbage-collecting texts, and has nothing else important to do.  Default is 0 milliseconds.

Sync timeout: TIMEVAL
     How long to sleep when lyskomd is saving its database. Defaults to 0 milliseconds.

Permissive sync: BOOL
     Turning this option on lets any session sync the LysKOM database. Turning it off restricts the operation to LysKOM administrators. Default is off.

Garb interval: TIMEVAL
     How long to wait between each garb sweep.  Defaults to 1440 minutes, which means that 24 hours will pass between each garb sweep.

Sync interval: TIMEVAL
     How long to wait between syncs. The current version of lyskomd keeps changes to the database in memory until they are synced to disk. This parameter specifies how long the server waits before attempting to dump the database. The default is 5 minutes.

Sync retry interval: TIMEVAL
     If anything goes wrong while trying to dump the data base (such as if the disk is full), lyskomd will wait for this long before trying again. Default is 1 minute.

Saved items per call: INT
     When the server is saving the database, it does so in the background. It serves one call from a client, saves a few items to the new database file, serves another call, et c.  This parameter sets the number of items (texts, conferences, persons) that are saved after each call. Default is 5.

Penalty per call: INT
     Penalty points given to a client once a call is completed.  This affects the scheduling.  Default is 10.

Penalty per read: INT
     Penalty points given to a client each time a read(2) is performed on the socket connected to the client.  This affects the scheduling.  Default is 1.

Max penalty: INT
     Once a client receives this many penalty points, the server will stop reading from the socket connected to the client.  (Once the server becomes idle, all penalty points will be aged, so the server will soon start reading from it again.)  Default is 100.

Low penalty: INT
     Once the penalty points for a client is reduced below this setting, the server will start reading from the client again.  This should be lower than Max penalty.  Default is 20.

Default priority: INT
Max priority: INT
     The default and max scheduling priority of a client.  Both values must currently be set to 0, which is the default.

Default weight: INT
Max weight: INT
     The default and max scheduling weight for a client.  Defaults to 20 and 100.

Connect timeout: TIMEVAL
     If the client doesn't send the initial handshake (such as A27Hceder@stanly.lysator.liu.se) within this time period, the client will be disconnected. Default is 30 seconds.

Login timeout: TIMEVAL
Active timeout: TIMEVAL
     If nothing is sent to the client for this long, the client will be disconnected.  Both asynchronous messages and replies to requests from the clients will reset the timer.  The Login timout: value is used while nobody is logged in on the session. Default is 30 minutes and 11.5 days, respectively.

Max client data length: INT
     The maxiumum allowed length for client name and version data. The default is 60.

Max conference name length: INT
     The maximum length of conference names. The default is 60.

Max password length: INT
     Only the first eight characters of the password are currently significant, even if this number is much larger. The default is 128.

Max what am I doing length: INT
     The maximum length of the string permitted in the protocol A call change-what-i-am-doing. The default is 60.

Max username length: INT
     The maximum length permitted for user names. Default is 128.

Max text length: INT
     The maximum length allowed for a text. The default is 131072 characters.

Max aux_item length: INT
     The maximum length allowed for a single aux-item. The default is 16384 characters.

Max broadcast length: INT
     The maximum length allowed for broadcast messges. The default is 1024 characters.

Max regexp length: INT
     The maximum length allowed for regexps in various calls. The default is 1024 characters.

Statistic name length: INT
     The maximum lenght allowed for the name of a measured statistics.  The default is 64 characters.

Max marks per person: INT
     The maximum number of marks a person is allowed to have. The default is 2048.

Max marks per text: INT
     The maximum number of marks a text can have. The default is 1024.

Max recipients per text: INT
     The maximum number of recipients of a text. The default is 512.

Max comments per text: INT
     The maximum number of comments a text can have. The default is 128.

Max footnotes per text: INT
     The maximum number of footnotes a text can have. The default is 32.

Max links per text: INT
     The maximum number of misc info items that can be added to a text.

Max mark_as_read chunks: INT

Max super_conf loop: INT

Max accept_async len: INT
     Maximum length of list accepted in the accept_async call. Default is 128.

Max aux_items deleted per call: INT
     Maximum number of aux_items that can be deleted in one call. Default is 128.

Max aux_items added per call: INT
     Maximum number of aux_items that can be added at once. Default is 128.

Max read_ranges per call: INT
     Maximum number of read_ranges that can sent in a single request. Default is 512.

Default garb nice: INT
     Each conference has a lifetime for texts written in it. The lifetime is counted in days, and can be set for each conference by the administrator of the conference. This is the default value assigned to new conferences. Default is 77 days.

Default keep commented nice: INT
     A text will not be removed if it has comments newer than a certain number of days. This number can be set for each conference. This parameter specifies the default value for that number of days. The default is 77.

Max client message size
     The maximum number of bytes that is read or written in a single system call.  Defaults to 8176.  (Attempts to set it to a larger value will currently only affect the input.)

Max client transmit queue messages: INT
Max client transmit queue bytes: INT
     Max number of pending data blocks (or total number of bytes) in the reply queue to a client. If there is ever more than this many data blocks in the queue the client will be disconnected. Each atomic question typically generates two data blocks. Default is 50 and 100000, respectively.

Stale timeout: TIMEVAL
     If the transmit queue of a client is full for this long, without the server being able to send anything to the client, the client will be disconnected.  Default is 60 minutes.

Max simultaneous client replies: INT
     This is a performance tuning parameter of little real interest. Default is 10.

Open files: INT
     Try to persuade the operating system to allow lyskomd to have this many open file descriptors simultaneously. Each client that is connected to the server occupies one file descriptor, and lyskomd needs several file descriptors for internal purposes. Default is to not use this parameter.

Use DNS: BOOL
     The IP address of a client is looked up using DNS when it connects. Unfortunately, this lookup blocks the entire server, and it can take several minutes.  You can disable DNS lookup with this parameter. Default is on.

DNS log threshold: DOUBLE
     If the Use DNS: parameter is true, the server will measure the time each DNS lookup takes.  If the time exceeds the specified threshold, an entry will be made in the log.  The value is specified in seconds.  The default value is 1.5 seconds.  If your libc supports it, you can enter +inf to disable logging.

Anyone can create new persons: BOOL
     If this is set, anyone can create a new person, even if he lacks special bits for doing so. Default is on.

Anyone can create new conferences: BOOL
     If this is set, anyone can create a new conferences, even if he lacks special bits for doing so. Default is on.

Allow creation of persons before login: BOOL
     If this is set, persons can connect the the server and create a new person without logging in.  This is how new users register in open environments. If this option is off, then new persons can only be created by existing users. The default is on.

Default change name capability: BOOL
     If this is set, new users are created with the ability to change their own name. Default is on.

Ident-authentication: POLICY
     Decide how strictly the server should use the IDENT protocol. The policy can take any of three values:

     off or never
          Do not use the IDENT protocol.

     on or try
          Use it, but allow logins even if the lookup fails.

     require or required
          Disallow connections if the server cannot find a IDENT login name.

Log login: BOOL
     Should logins be logged to the log file?  Default value is off.

Cache conference limit: INT
     How many conference statuses the server cache should hold in main memory. Default is 20. This parameter should be set to at least the number of expected simultaneous logins.

Cache person limit: INT
     How many person statuses the server cache should hold in main memory. Default is 20. This parameter should be set to at least the number of expected simultaneous logins.

Cache text_stat limit: INT
     How many text statuses the server cache should hold in main memory. The default is 20. This parameter should be increased on busy servers.

Echo: STRING
     Write STRING in the log when the config file is read.

Jubel: PERS_NO TEXT_NO
Jubel: public PERS_NO TEXT_NO
     States that PERS_NO is not allowed to create text number TEXT_NO. Default is unset. This parameter may be used multiple times. The form with the string public means that the text must have a public conference as recipient.

Jubel: PERS_NO DIVIDEND REMAINDER

Jubel: public PERS_NO DIVIDEND REMAINDER
     States that PERS_NO is not allowed to create any text number T which meets the condition T % DIVIDEND == REMAINDER. Default is unset. This parameter may be used multiple times. The form with the string public means that the text must have a public conference as recipient.

Add members by invitation: BOOL
     If this is set, then adding others as members to a conference sets the invitation bit of the membership. If this is off, the membership bit is set to whatever the caller specifies. The default is on.

Allow secret memberships: BOOL
     If this is set, then memberships may be secret. Otherwise any attempt to create a secret membership or change an existing membership to a secret membership will fail. The default is on.

Allow reinvitations: BOOL
     If this is set, then it is possible to set the invitation bit of a membership even after it has been cleared. If it is not set, then the invitation bit of a conference type can only be set when the membership is created. It can be cleared at any time. The default is off.

lyskomd path: PATH
     Path to the lyskomd binary.  This is used by updateLysKOM to find the right program to run.  Defaults to sbin/lyskomd.

savecore path: PATH
     Path to the savecore-lyskom program.  If a file named core exists in the directory specified with Core directory when updateLysKOM is about to start lyskomd, this program will be called first.  It could, for instance, move the core file so that it is available for later debugging.  The script supplied by the distribution does nothing. Defaults to sbin/savecore-lyskom.

Normal shutdown time: INT
     In a normal setup, updateLysKOM will be run from cron once every ten minutes or so.  If it detects that it has taken lyskomd more than INT minutes to shut down it will print a warning message.

Mail after downtime: INT
Mail until downtime: INT
     If lyskomd has been down for X minutes, where Mail after downtime <= X < Mail until downtime, updateLysKOM will send a mail message to the mail address found on the first line of the status file.  Actually, it is the age of the status file (named with Status file) that is measured. The defaults are 60 and 120, respectively.

sendmail path: PATH
     Path to the sendmail-compatible program that updateLysKOM should use to send mail.  This program will be invoked with a -t option via a popen() call.  It should accept an email header, a blank line, an email body, and a terminating line consisting of a single . on standard input. The default is found at configure time.  The special value : means that no mail will ever be sent.

Free space warning level: DOUBLE
Free space warning percent: DOUBLE
Free inodes warning level: DOUBLE
Free inodes warning percent: DOUBLE
Free space critical level: DOUBLE
Free space critical percent: DOUBLE
Free inodes critical level: DOUBLE
Free inodes critical percent: DOUBLE
     These parameters determine when the checkkomspace progam should consider the space to be sufficiently large.  All levels are given in bytes.  Specify 0 if you don't want checkkomspace to check a particular limit.

Aux-Item Definition File
========================

The default aux-item definition file should not be changed unless it is really necessary. The need to change the definitions will probably only arise at installations used for client or server development.

The location of the aux-item definition file is specified by the Aux-item definition file option in the server configuration file. The default location is /usr/lyskom/etc/aux-items.conf.

Syntax of the Aux-Item Definition File
--------------------------------------

The aux-item definition file contains a sequence of aux-item definitions. Each definition specifies one type of predefined aux-item: its number, name, and properties. Empty lines and all characters from a # character to the end of the line are ignored.

Each entry has the following format:

             tag : name (target, target, ... )
             {
                 field = value;
                 field = value;
                 ...
             }

TAG is an integer, the aux-item's tag. If a tag is defined more than once, the last definition is used.

The TARGETs specify what kind of objects aux-items with tag TAG can be added to. Valid targets are:

any
     Aux-items with the specified tag can be added to any object in the database. This is shorthand for text,conference,letterbox,server.

text
     Aux-items with the specified tag can be added to texts.

conference
     Aux-items with the specified tag can be added to conferences that are not letterboxes.

letterbox
     Aux-items with the specified tag can be added to conferences that are letterboxes.

server
     Aux-items with the specified tag can be added to the server itself.

It is legal to add one of the keywords create or modify before any target except server. If create is specified, aux-items with the specified tag can only be added when an object is being created. They cannot be added later. If modify is specified, aux-items with the specified tag can only be added after an object has been created. They cannot be added when the object is being created.

Each FIELD/VALUE pair specifies a property of aux-items with the specified tag. Most values are boolean or trillian. Legal values for either type are true and false. Boolean values have reasonable defaults; trillian values can be unset.

author-only
     Boolean, default false. When true, restricts creation of items with this tag to specific users. Items with this tag may be attached to texts only by the author of the text and supervisors of the author. Items with this tag may be attached to conferences only by supervisors of the conference. For conferences with the letterbox flag set, the person with the same number as the conference may also attach items with this tag. This flag has no effect on items being attached to the server.

supervisor-only
     Boolean, default false. When true, restricts creation of items with this tag to specific users. Items with this tag may be attached to texts only by supervisors of the author (which does not necessarily include the author). Items with this tag may be attached to conferences (with or without the letterbox flag) only by supervisors of the conference. This flag has no effect on items being attached to the server. Aut-item types with the supervisor-only flag are primarily useful in servers where users are not their own supervisors, to add information to letterbox conferences that users themselves should be unable to alter.

system-only
     Boolean, default false. When true, only the server can initiate creation of items with this tag. This is normally used for items that are created automatically in response to events in the system.

permanent
     Boolean, default false. When true, aux-items with this tag cannot be deleted once they have been created.  (They will be deleted automatically when the object they are assigned to is deleted.)

unique
     Boolean, default false. When true, there can only be one non-deleted item with this tag per creator.

unique-data
     Boolean, default false. When true, there can only be one non-deleted item with this tag that contains the same data (regardless of who creates the item).

owner-delete
     Boolean, default false. When true, the owner of the object that this aux-item is attached to can always delete the aux-item.  For a text, the owner is defined as the supervisor(s) of the author of the text. For a conference, the owner is defined as the supervisor(s) of the conference.

inherit-limit
     Integer, default 0. The maximum number of times items with this tag can be inherited, plus one. Zero means an unlimited number of times, one means no times, 2 means once and so forth. This number overrides the inherit-limit set by the client only if that number is higher than this one.

inherit
     Trillian. When set, the inherit bit on new items with this tag is forced to the specified value.

secret
     Trillian. When set, the secret bit on new items with this tag is forced to the specified value.

hide-creator
     Trillian. When set, the hide-creator bit on new items with this tag is forced to the specified value.

dont-garb
     Trillian. When set, the dont-garb bit on new items will be forced to the specified value.

reserved-2

reserved-3

reserved-4
     Trillian. When set, these flags force the values of the three reserved bits in the aux-item flags field. These should only be used by lyskomd developers, and then only very carefully.

validate

     String or function, default none. When set to a string, this specifies a regexp that must match the data field in newly created items with this tag. If the regexp fails to match, then the item will not be created. The syntax for strings is essentially the same as the syntax used in C files. When set to a function, this specified a built-in validation function to call.

     The following validator functions are currently implemented:

     existing-readable-text
          Creation is only allowed if the item contains the number of an existing text that the item creator has permission to read.

There are a few fields which specify actions the server is to take when something happens to aux-items with the specified tag. Each of these values is a function specification, the name of a trigger function defined in lyskomd. The syntax for functions is the name followed by an empty pair of parens. It is not possible to pass arguments to the functions yet.

add-trigger
     Function to call when an item with the specified tag is added to an object.

delete-trigger
     Function to call when an item with the specified tag is scheduled for deletion.

undelete-trigger
     Function to call when an item with the specified tag scheduled for deletion is unscheduled. It should undo the effects of the delete trigger.

The following trigger functions are currently defined:

mark-text
     Increase the mark count for the text the item refers to. The item must contain the number of a text. This trigger should be combined with the existing-readable-text validation function.

unmark-text
     Decrease the mark count for the text the item refers to. The item must contain the number of a text. This trigger should be combined with the existing-readable-text validation function.

link-faq
     Create a faq-for-conf item linked to a faq-text item. This trigger is used exclusively for faq-text items. The item must contain the number of a text. This trigger must be combined with the existing-readable-text validation function.

Running lyskomd
***************

This section explains how to run lyskomd, the files it uses and how it can be controlled while running.

Invoking lyskomd
================

             lyskomd [-f] [-d] [CONFIG-FILE]

The option -d adds one to the debug level. The amount of output on stderr/to the log file is increased for each time the option is specified on the command line.

Using one -d makes the process print a `>' for every timeout, a message for every person that is connecting or disconnecting and a message for every successful or unsuccessful communication to the process.

The option -f tells lyskomd to stay in forground mode, and not run in the background as a daemon.  The output that is normally written to the log file is instead sent to stderr.

The optional CONFIG-FILE argument can be used to specify the server configuration file. See Server Configuration File.

Signals
=======

It is possible to control some aspects of lyskomd using Unix signals. The following signals have special meaning to the server:

SIGTERM
SIGHUP
SIGINT
     Logs out all sessions, saves the database and exits normally.  Use SIGTERM; the other signals currently have the same functionality, but that may be changed in the future.

SIGQUIT
     Saves the database and dump core. (This should only be used for debugging purposes.)

SIGUSR1
     Print statistics about how often different commands have been used since the process started.

SIGUSR2
     Forks a child that immediately dumps core. The main process just waits until the child is done and then continues.

SIGWINCH
     Re-read the aux-item definition file. Warning: If the aux-item definition file contains an error so that it cannot be parsed, the server will call restart_kom(), which will cause the server to abort without saving the database.  Always test the file on a standby system first!

Files Used by lyskomd
=====================

All file names can be changed in the server configuration file. See Parameters.

/usr/lyskom
     Default value of the Prefix parameter. The default of this value is set at compile time, but it can be changed in the server configuration file.  See Parameters.

PREFIX/var/lyskomd/db/lyskomd-data
     Half of the database: all status information.

PREFIX/var/lyskomd/db/lyskomd-texts
     The other half of the database: the actual texts.

PREFIX/var/lyskomd/db/lyskomd-backup
     A backup copy of lyskomd-data. Never, ever delete this file unless you know what you are doing, or you may lose the entire data base. Most of the time this is the only complete database file!

PREFIX/var/lyskomd/db/number.txt
     Information about the highest used text- and conference numbers.  In case of a crash, some objects may be lost.  This file ensures that even if that happens, their numbers will not be reused.

PREFIX/var/run/lyskomd.pid
     File with the pid of the lyskom-process.

PREFIX/var/lyskomd.memory
     On normal exit, lyskomd will append some statistics to this file. It can be used for detecting memory leaks.

PREFIX/etc/aux-items.conf
     This file contains definitions of the aux-items that the server should support.  It is read by lyskomd at startup.

PREFIX/var/lyskomd.clients
     A list of all currently connected clients, maintained by the server. The data about each client is collected on a single line:

       * The file descriptor

       * The session number

       * 1 if the handshake is OK, the reverse DNS has completed, and the IDENT lookup has completed.  0 otherwise.

       * The IP address of the client

       * The port number of the client

     In the following example, we see that file descriptor 437 is used by session 330978, and the connection is from 130.236.254.83:3156.

          437 330978 130.236.254.83 3156

/etc/nologin
     If this file exists, lyskomd will not allow anyone to connect to the server. This path can be set with the Nologin file parameter in the server configuration file.

Invoking updateLysKOM
*********************

             updateLysKOM [-c CONFIG-FILE] [ -v ] [ -V ]

updateLysKOM determines if lyskomd should be running.  It can start or stop lyskomd if needed.  It uses the same configuration file as lyskomd (see Server Configuration File).  You can use -c CONFIG-FILE to override the compiled-in default.  Note, however, that this option is not passed along to lyskomd if updateLysKOM starts it, so the option should be used with extreme caution.

-v and -V causes updateLysKOM to report its version number and exit.

updateLysKOM is normally run from cron; see Administration.

Invoking komrunning
*******************

             komrunning [-c config-file] [start | stop]
             komrunning -v | -V

komrunning, when invoked with no arguments, reports whether lyskomd is currently running or not, and whether it should be running or not.  komrunning start attempts to start lyskomd.  komrunning stop attempts to stop lyskomd, and it will not return until the server has saved its database and exited.

komrunning uses the same configuration file as lyskomd (see Server Configuration File).  You can use -c CONFIG-FILE to override the compiled-in default.  Note, however, that this option is not passed along to updateLysKOM if komrunning invokes it, so the option should be used with extreme caution.

The komrunning can be installed in /etc/init.d/.  Be careful, however, to ensure that the pid file is removed earlier during the boot sequence.

-v and -V causes komrunning to report its version number and exit.

Invoking checkkomspace
**********************

             checkkomspace [-d] [config-file]
             checkkomspace -h

checkkomspace checks if the partition where the database files are is large enough.  It is intended to be run at regular intervals from a management system such as Nagios.  The exit status will be 0 if all is OK, 1 if a warning condition exists, and 2 if an error exists.

This program knows the disk usage patterns that lyskomd has, so it calculates the free space based on what lyskomd actually needs.  This might be several hundred megabytes more than what a call to df would report.  This also means that the result will not fluctuate as wildly as a df report would.

checkkomspace uses the same configuration file as lyskomd (see Server Configuration File).  You can specify a different config file via the CONFIG-FILE argument.

-h checkkomspace to report its version number and exit.

-d is only intended to be used by the test suite.

Administration
**************

The first thing you will have to do is to follow the instructions in the files INSTALL and README. This will set up the LysKOM system with a database containing a few necessary conferences and one person - the administrator.

Once the LysKOM system is running, there is not much you will have to do to keep it that way. One thing to remember is that the current release of the server has an incomplete handling of garbage collection of the database. The database is split into two files, the information file and the text file. Newly written texts are concatenated to the text file and old texts are never removed. The information file contains information about conferences, users and where in the text file the texts are. This file is properly garbage collected, but not the text file.

There is a program called dbck (Data Base Check) which is used to check the consistency of the LysKOM database. This program can also be used to shrink the text file. To do this, just type dbck -g. See DBCK Reference. When dbck is to be run on the database, the LysKOM server must be stopped, or unrepairable damage may result. See below for a description on how to stop the server.

There is a program called updateLysKOM which is used to ensure continuous operation. This program should be run with certain intervals, for instance from cron. If the LysKOM server has died for some reason, updateLysKOM restarts it. If the server is still running properly, updateLysKOM sends a signal (SIGUSR1) to it, which causes the server to write some statistics to a file named var/lyskomd.stats in the lyskom directory.

Taking the server down cleanly can be done in two ways: through the use of the LysKOM protocol on a socket, preferably through the use of a suitable client, or by sending the signal SIGTERM to it. This will cause the server to save the database and close all client connections. It will also create a file named var/lyskomd.memory in which the memory usage of the server is reported.

To prevent updateLysKOM from restarting a server, create a file named /usr/lyskom/var/lyskomd/db/status. The file should contain a valid mail address on the first line. updateLysKOM will not restart the server as long as that file exists. In addition, if the file is between 1 and 2 hours old (configurable) an email will be sent to the mail address found in the file. If the file is older than that, an error message will be printed on stderr and updateLysKOM will exit with a non-zero exit status. cron is expected to deliver the error message to an operator.

The shell script komrunning can be used to start and stop the LysKOM server. With no arguments, it will report the status.

             komrunning stop

will (attempt to) shut down the server, creating the file /usr/lyskom/var/lyskomd/db/status.  If the user running komrunning doesn't have permission to send signals to lyskomd the actual shutdown will be delayed until the next time that updateLysKOM is run.

             komrunning start

will restart the server.  The actual starting of the server will be done by updateLysKOM the next time it is run.  komrunning only removes the  /usr/lyskom/var/lyskomd/db/status file.

Known Bugs
**********

  * lyskomd should re-read the config file when a SIGHUP is received.

  * The security policy is vague and the implementation is frayed at the edges.

  * The choice of asynchronous messages is not very good.

  * The server uses too much memory.

DBCK Reference
**************

dbck is a program that can is used for minor database maintenance tasks and for repairing a corrupt lyskomd database.

Overview
========

The dbck program is used for minor maintenance of the LysKOM database and for repairing corrupt databases. In brief it performs the following functions:

  * Compact the text file to remove deleted texts.

  * Repair inconsistent membership information.

  * Repair invalid recipients

  * Repair inconsistent comment links

  * Correct invalid local text numbers

  * Correct invalid text maps

  * Set special conferences

  * Convert between database formats

Invoking dbck
=============

The functionality of dbck is controlled through command-line switches. These are documented below.

If dbck is invoked without any options it will read the database and report on its integrity.  No files will be modified.

General Options
---------------

These options control the general behavior of lyskomd.

-h or --help
     Give a usage message (which includes the version number and the compiled-in default location of the config file) and exit immediately.

-v or --verbose
     Verbose mode. Report not only errors but the status of the database.

-F or --force-output
     This option forces dbck to write the database file. Normally dbck will only write a new database file if changes have been made for some other reason. If you want to simply convert a database from one version to another, you will probably have to give this option.

Database Repair Options
-----------------------

The following options control database repair.

-i or --interactive
     Run interactively. If any inconsistency is found, a remedial cure will be suggested and the user must confirm the correction.

-r or --auto-repair
     Repair simple errors without asking.

-c or --set-change-name
     Consider it an error if the change-name capability of a person is not set. Due to a bug that capability was never set for newly created persons in release 1.6.1 of lyskomd. This option can be used to repair the damage.

Format Conversion Options
-------------------------

dbck can be used to conver the LysKOM database from one storage format to another. This is necessary only when moving the database to a new server version.

-F or --force-output
     This option forces dbck to write the database file. Normally dbck will only write a new database file if changes have been made for some other reason. If you want to simply convert a database from one version to another, you will probably have to give this option.

-o or --output-version
     This option is used to set the output version of the database. This option will normally be used in conjunction with the -F option.

     Version 1.9 of lyskomd requires database version 1; version 2.0 requires database version 2. Versions of lyskomd prior to 1.9 requires database version 0. Note that information is irrevocably lost when converting from a higher to a lower database version. This options requires an argument: the output format version.

Database Update Options
-----------------------

dbck can be used to update certain aspects of the database that either were impossible to update in early versions of protocol A or that are inconvenient in all protocol versions.

-g or --compact-text-mass
     Do garbage collection on the texts part of the database. This removes all unreferenced texts from the database.

-P or --clear-password
     Clear the password of a specified user. This option is silently ignored if the user does not exist. This option requires an argument: the ID of the person whose password is to be cleared.

-G or --grant-all
     Grant all privileges to the specified user. This option is silently ignored if the user does not exist. This option requires an argument: the ID of the person who is to be granted all privileges.

--pers-pres-conf
     Set the person presentation conference of the server to the specified conference. Since version 1.9 of lyskomd the set-info call can be used to do this.

--conf-pres-conf
     Set the conference presentation conference of the server to the specified conference. Since version 1.9 of lyskomd the set-info call can be used to do this.

--motd-conf
     Set the message-of-the-day conference of the server to the specified conference. Since version 1.9 of lyskomd the set-info call can be used to do this.

--kom-news-conf
     Set the news-about-lyskom conference of the server to the specified conference. Since version 1.9 of lyskomd the set-info call can be used to do this.

--motd-of-kom
     Set the message of the day of the server to the specified text. Since version 1.9 of lyskomd the set-info call can be used to do this.

Reporting Options
-----------------

These options control reporting of information about the database.

-s or --print-statistics
     Gather statistics about the lengths of texts. A table containing the frequence of all lengths that are currently present is printed.

-t or --list-text-no
     Print ``Checking TEXT-NO'' for every text that examined. Warning: This produces lots of output.

Notes for DBCK
==============

The messages ``Conference CONF-NO has a bad Text-list. Starts with 0'' and ``Person PERS-NO has created NUM conferences, not NUM (as said in person-stat).'' are normal. If you get them when you specify the -g option, let dbck repair them and run dbck -g again.

Files Used by dbck
==================

dbck uses the same files as lyskomd (see (lyskomd).)

All file names can be changed in the server configuration file. See (lyskomd)Parameters.

/usr/lyskom
     Default value of Prefix:. The default of this value is set at compile time, but it can be changed in the server configuration file. See (lyskomd)Parameters.

PREFIX/var/lyskomd/db/lyskomd-data
     Half of the database: all status information.

PREFIX/var/lyskomd/db/lyskomd-texts
     The other half of the database: the actual texts.

PREFIX/var/lyskomd/db/lyskomd-backup
     A backup copy of lyskomd-data. Never, ever delete this file unless you know what you are doing, or you may lose the entire data base. Most of the time this is the only complete database file!

Known Bugs
==========

  * Should have an unlock database option.

  * Does not check that the data file and text file are consistent.

Backups with splitkomdb
***********************

The files created by lyskomd can become huge.  They are more or less constantly modified, so if you do incremental backups the entire lyskomd database will be saved each time you perform a backup.  The splitkomdb program improves the situation.

splitkomdb theory of operation
==============================

The splitkomdb program attempts to split the current database in several files, some of which will not be modified the next time you run splitkomdb.  An incremental backup will thus only have to save some of the files created by splitkomdb.

splitkomdb can be run while lyskomd is running.  It will use the last completely saved snapshot, and make a copy of it in the directory specified by the Backup export directory: parameter (default: /usr/lyskom/var/lyskomd/exportdb/).

There are two recommended ways to make backups of a lyskomd database:

  * Run splitkomdb before each backup, and instruct the backup program to backup /usr/lyskom/var/lyskomd/exportdb, but ignore /usr/lyskom/db.  Run splitkomdb -f before each full backup.

  * Backup all the files in /usr/lyskom/db.

The first alternative needs more disk for the copy in /usr/lyskom/var/lyskomd/exportdb and is slightly more complex to set up; the second alternative saves the entire database each time you make a backup.

Invoking splitkomdb
===================

             splitkomdb [-c config-file] [-f]
             splitkomdb -v | -V

splitkomdb creates a splitted copy of the most recent database snapshot in the directory specified by the Backup export directory: parameter.

If -v or -V is specified, splitkomdb prints its version number and exits.

The -f option is a hint to splitkomdb that the next backup will be a full backup.  When this option is used, splitkomdb might rewrite all the output files in a way so that future invocations of splitkomdb will create a little incremental data as possible.

The optional CONFIG-FILE argument can be used to specify the server configuration file. See Server Configuration File.

Typical splitkomdb setup
========================

A typical way to run splitkomdb is from cron.  A crontab entry might look like this:

     50 2 26 * * /sw/lyskom/bin/splitkomdb -f
     10 3 * * * /sw/lyskom/bin/splitkomdb

This entry assumes that backups are started at 03:30.  It gives splitkomdb 20 minutes to run.  (That should be enough for most sites, but please check what is appropriate for your site.)

Full backups are run on the 26th of each month.  On those days, at 02:50, splitkomdb -f is run.

Restoring the database from splitkomdb
======================================

The splitted format of the database is currently very simple:

var/lyskomd/exportdb/lyskomd-texts-base.backup
     This file contains the first part of Text file: (default: var/lyskomd/db/lyskomd-texts).  This file is created when the -f option is given to splitkomdb.

lyskomd-texts-tail.backup
     This file contains the rest of Text file:.  It is always recreated when splitkomdb is run.

lyskomd-data.backup
     This file contains a copy of Data file: (default: var/lyskomd/db/lyskomd-data).  If Data file: file wasn't clean when splitkomdb was run, Backup file: (default: var/lyskomd/db/lyskomd-backup) will be copied instead.

This means that it is easy to restore the database from the splitted database.  If you use the default paths, all you have to do is run these commands:

     $ cd /usr/lyskom/var/lyskomd/exportdb
     $ cat lyskomd-texts-base.backup \
     >     lyskomd-texts-tail.backup \
     >     > ../db/lyskomd-texts
     $ cp lyskomd-data.backup ../db/lyskomd-data

Future versions of splitkomdb may use a different format.  It may even evolve into something so complex that you need a program to recreate the database.  Always check the current documentation for information on how to restore the database.

Hacking lyskomd
***************

The Database
============

This section is not translated to English yet.  See a comment in the lyskomd.texi for the raw Swedish text.

Adding Configuration Parameters
===============================

Make sure that you really understand what you want to configure. Think it over again. Find a good, descriptive name for it.

Decide what values the parameter can be set to. Integers? Booleans?

Document the parameter in lyskomd.texi.

Add a field to struct kom_par in param.h.

Add it to parameters[] in server-config.c. See conf-file.h and maybe conf-file.c for information on this structure.

Make sure that the parameter is used instead of any previous hard-coded value. Make sure that dbck can cope with it.

Adding Asynchronous Messages
============================

async.h
     Add the message in enum async { }. Make sure that ay_dummy_last is one more than any other message. If the message is to be sent by default, which is not recommended, place its number into ASYNC_DEFAULT_MESSAGES.

prot-a-send-async.c
prot-a-send-async.h
     Write a function that sends the message. This function is responsible for writing the message to a particular connection and for ensuring that the message is not sent to clients who do not want it. Make sure that the second argument to async_header really is the number of elements being sent. Arrays count as two elements: the item count and the elements.

send-async.c
send-async.h
     Write a function that sends the message to appropriate clients. This function is responsible for checking that async messages should be sent at all, for each client check if it allowed to see the message and ensure that the protocol specified by the connection is appropriate. The send function should either take a struct connection * as an argument and send the message to that connection, or loop over all connections. Most send functions take a connection pointer; the looping is dealt with elsewhere.

session.c
     Add a case label for the enum in the large switch statement.

Make sure that the message is sent in appropriate places.

testsuite/lyskomd.0/03.exp
     A few tests will fail.  Fix them.

Document the message type in Protocol-A.texi.

Write test cases for the new async message.

Function Templates for prot-a-send-async.c
------------------------------------------

This is what a typical function in prot-a-send-async.c should look like. This function is responsible for checking that the client is accepting the message and writing the message itself to the connection.

     void
     prot_a_async_SOMETHING(Connection   *cptr,
                                PARAMETERS)
     {
         ASYNC_CHECK_ACCEPT(cptr, ay_SOMETHING);
         async_header(cptr, NUM_TOKENS, ay_SOMETHING);
         /* Output the body of the message */
         async_trailer(cptr);
     }

Function Templates for send-async.h
-----------------------------------

This is what a typical function in send-async.c should look like. This function is responsible for sending the message to all connections that are appropriate, not sending it if the server is not supposed to send messages at all, and for checking that the protocol specified by the client is one the server knows.

     void async_SOMETHING( PARAMETERS )
     {
         Connection *cptr;
         Session_no i = 0;

         if (!param.send_async_messages)
             return;

         while ((i = traverse_connections(i)) != 0)
         {
             cptr = get_conn_by_number(i);

             switch(cptr->protocol)
             {
             case 0:
                 /* No protocol specified yet */
                 break;
             case 'A':
                 /* Check that connection is logged on. We might
                    want to check other things here too, such as
                    if the connection is allowed to see the message */
                 if (handshake_ok(cptr, 0))
                     prot_a_async_SOMETHING(cptr, PARAMETERS);
                 break;
             default:
                 restart_kom("async_SOMETHING(): bad protocol.\n");
                 break;
             }
         }
     }

Template for a function that sends to a single connection:

     void async_SOMETHING(struct connection *cptr,
                              PARAMETERS)
     {
         if (!param.send_async_messages)
             return;

         switch(cptr->protocol)
         {
         case 0:
             /* No protocol specified yet */
             break;
         case 'A':
             /* Check that connection is logged on. We might
                want to check other things here too, such as
                if the connection is allowed to see the message */
             if (handshake_ok(cptr))
                 prot_a_async_SOMETHING(cptr, PARAMETERS);
             break;
         default:
             restart_kom("async_SOMETHING(): bad protocol.\n");
             break;
         }
     }

Adding a New Protocol Request
=============================

Before doing anything, think again. Make sure that the protocol request is needed, is in line with the rest of the protocol, behaves the way people want it to, and that everyone involved agrees that it is a good idea.

 1. Document the request in Protocol-A.texi

 2. Declare the function in include/services.h

 3. Declare the function last in server/fncdef.txt. It should be given a call number one higher than the currently existing highest contiguous call number.

 4. If the function takes an input parameter of a new type, changes need to be made in several places. See Adding New Input Types.

 5. If the function takes too many parameters of type num, string or c_string, the definition of Connection in server/connection.h has to be changed.

 6. If the function has an output parameter of a new type, changes need to be made in several plaves. See Adding New Result Types.

 7. Write the function in a suitable place in the server directory.

 8. Write tests for the new function in server/testsuite/lyskomd.0. Write one file for testing the functionality. Write tests in 01.exp (behavior when the client is not logged on) and 03.exp (normal behavior.)

 9. Run the testsuite to make sure nothing old has been broken.

Every request handler should call the CHK_CONNECTION macro before doing anything else. This ensures that the active_connection variable is non-NULL. The only time when this might not be the case is if the request handler is not called in response to a client request. This should never happen, but might if someone gets careless.

If your function should not be available before the user is logged in, call the CHK_LOGIN macro after doing CHK_CONNECTION.

Take care returning errors to the client. Previous versions of lyskomd leaked secret information through error returns. For example, the following code leaks information:

     Success service(Conf_no conf_no, Text_no text_no)
     {
         Conference *conf_stat;
         Text_stat  *text_stat;

         CHK_CONNECTION(FAILURE);
         CHK_LOGIN(FAILURE);
         GET_C_STAT(conf_stat, conf_no, FAILURE);
         GET_T_STAT(text_stat, text_no, FAILURE);
         if (!has_access(conf_no, active_connection, read_protected))
         {
             err_stat = conf_no;
             kom_errno = KOM_UNDEF_CONF;
             return FAILURE;
         }
         if (!text_read_access(active_connection, text_no, text_stat))
         {
             err_stat = text_no;
             kom_errno = KOM_NO_SUCH_TEXT;
             return FAILURE;
         }

         /* Permissions checked. Do the deed. */

         return OK;
     }

This request can be used to gain precise information on which conferences and texts exist in the system. If an unprivileged user makes a request for any conference and readable text, and the user receives a KOM_NO_SUCH_TEXT error, the user can deduce that the conference exists, but is secret. If the user makes a request for a conference known to be secret and a text known not to be readable (either secret or deleted), and the user receives a KOM_UNDEF_CONF error, the user can deduce that the text does exist.

To avoid traps like these, do permission checks for objects immediately after attempting to get them from the database.

See also:

  * Adding New Input Types

  * Adding New Result Types

  * Modifying Output Types

Adding New Input Types
======================

Changes need to be made in the following files:

Protocol-A.texi
     Document the new type.

server/call-switch.awk
     The new type has to be added to the cascaded ifs that translate the type name to code that points to the appropriate field in a Connection structure.

server/prot-a-parse-arg-c.awk
     The new type has to be added to the cascaded ifs that create the argument list parser.

server/connections.h
     The definition of Connection must be extended with a field where the parse value can be stored. Don't even think about trying to reuse an existing field. It's more trouble than it's worth.

server/connection.c
     Free the contents of the field in free_parsed.

server/prot-a.c
     Free the contents of the field in prot_a_destruct.

server/internal-connections.c
     Initialize the contents of the field in init_connection.

server/testsuite/lyskomd.0/29.exp
     Add test cases for disconnecting in the middle of the data type.  This is important both for singletons and lists.

Adding New Result Types
=======================

Changes need to be made in the following files:

Protocol-A.texi
     Document the new type.

server/prot-a.c
     Add a line in the prot_a_reply switch that calls the correct output function.

server/connections.h
     Add the type in enum res_type and union result_holder.

server/prot-a-output.c
server/prot-a-output.h
     Write a function that outputs the new type to a connection. Use the existing functions as templates.

Modifying Output Types
======================

When you modify an existing type you have to rename the old version of the type since it will still be used in existing calls. The convention has previously been to rename SOMETHING to SOMETHING_old, but the preferred method is to append an underscore and the protocol version in which the current version of the type was introduced. For example, if the type Gazonk was introduced in protocol version 11, and a new version is to be introduced in protocol version 15, the current Gazonk structure is renamed to Gazonk_11.

This is to avoid names like Something_older, Something_oldest and Something_Truly_Ancient.

Changes need to be made to the following files:

Protocol-A.texi
     Document the new type in the appropriate section. Rename the existing type in the type documentation and in all calls that return it. Be thorough!

fncdef.txt
     Rewrite all calls that use the modified type so they use the old version of the type.

prot-a.c
     Modify the current line in prot_a_reply for the existing version of the type, and add a new line for the new version of the type.

connections.h
     Modify the existing entry in enum res_type and union result_holder, if necessary. This should only be necessary if the server uses both a new and old type internally, which is not recommended. Add new entries for the new version of the type.

prot-a-output.h
prot-a-output.c
     Rename the existing output routing according to the new name of the type. Write a new output routine for the new version of the type.

memory.c
     If there are functions for the type in memory.c, make sure that your new type is initialized, cleared and copied in an appropriate manner.

If the type you modify is stored in the database, make sure it gets saved properly. See Modifying Stored Types.

Adding Aux-Item Types
=====================

 1. Document the new type in Protocol-A.texi

 2. Write a definition of the new type in run-support/aux-items.conf.

 3. Some tests in at least server/testsuite/lyskomd.0/01.exp, server/testsuite/lyskomd.0/03.exp and server/testsuite/lyskomd.0/18.exp will fail when new predefined aux-items are added.  Fix the tests.

 4. Write test cases for the new aux-item.  If the aux item can be set on a letterbox, do so in server/testsuite/lyskomd.0/03.exp where the comment containing AUXITEM says to do so.  More complex aux-items should have more tests written for them. server/testsuite/lyskomd.0/20.exp might provide some inspiration.

 5. If the new type requires add, delete or undelete triggers that do not already exist, declare the trigger functions in aux-items.c and add them to the aux_item_triggers array in the same file.

 6. If the new type is so complex that is cannot be fully defined in aux-items.conf, then add it to the compiled_aux_items array in aux-items.c. Note that this functionality has not been tested until someone actually adds one of these beasts, so watch your step.

Modifying Stored Types
======================

If you want to modify an existing type that is stored in the database, think again. Can the job be done with aux-items instead? Is it really necessary?

Be very, very careful when doing this. You have to make sure that the type as sent in old calls and async messages is not changed in any way. You have to make sure that the type can be stored to and read from the database.

 1. Document the changes in Protocol-A.texi if the change is visible in the protocol.

 2. Bump the database version number by one for the next release of the server.

 3. Write a function in ram-output.c to output the new format. Update all old functions in ram-output.c that are database version dependent so that they can deal with the new database format.

 4. Write a function in ram-parse.c to read the new format. Update all old functions in ram-parse.c that are database version dependent so that they can deal with the new database format.

 5. Set the default database format in ram-parse.c and ram-output.c. The variables to change are input_format and output_format, respectively.

 6. Don't forget to update the functions in memory.c

 7. Update dbck so that it can convert to the new format.

 8. Add as many test cases as are needed for the dbck conversion.

Template for ram-output.c
-------------------------

For types that can be output in several different formats, use the following templates for them. You have to be able to output in all formats, or dbck will be unable to convert between formats.

     static void
     foutput_SOMETHING_0(FILE *fp, SOMETHING *o)
     {
         /* Output version 0 of SOMETHING */
     }

     static void
     foutput_SOMETHING_1(FILE *fp, SOMETHING *o)
     {
         /* Output version 1 of SOMETHING */
     }

     static void
     foutput_SOMETHING_2(FILE *fp, SOMETHING *o)
     {
         /* Output version 2 of SOMETHING */
     }

     void foutput_SOMETHING(FILE *fp, SOMETHING *o)
     {
         switch(output_format)
         {
         case 0:
             foutput_SOMETHING_0(fp, info);
             break;
         case 1:
             foutput_SOMETHING_1(fp, info);
             break;
         case 2:
             foutput_SOMETHING_2(fp, info);
             break;
         default:
             restart_kom("unknown database format: %d", output_format);
             break;
         }
     }

Note that if two versions are the same, only write one function. For example, if version 0 and version 1 are the same, only write an foutput_SOMETHING_0 function and call it from both case 0 and case 1.

Template for ram-parse.c
------------------------

     static Success
     fparse_SOMETHING_0(FILE *fp, SOMETHING *o)
     {
         /* Parse version 0 */
         return OK;
     }

     static Success
     fparse_SOMETHING_1(FILE *fp, SOMETHING *o)
     {
         /* Parse verson 1 */
         return OK;
     }

     static Success
     fparse_SOMETHING_2(FILE *fp, SOMETHING *o)
     {
         /* Parse verson 2 */
         return OK;
     }

     extern Success
     fparse_SOMETHING(FILE *fp, SOMETHING *o)
     {
         if ( fparse_long_errors != 0 )
         {
             log("fparse_SOMETHING(): fparse_long_errors == %d on entry."
                 " Reset.\n",
                 fparse_long_errors);
             fparse_long_errors = 0;
         }

         switch (input_format)
         {
         case 0:
             return fparse_SOMETHING_0(fp, o);
             break;
         case 1:
             return fparse_SOMETHING_1(fp, o);
             break;
         case 2:
             return fparse_SOMETHING_2(fp, o);
             break;
         default:
             restart_kom("unknown input format: %d\n", input_format);
             return FAILURE;
             break;
         }
     }

Note that if two versions are the same, only write one function. For example, if version 0 and version 1 are the same, only write an fparse_SOMETHING_0 function and call it from both case 0 and case 1.

Hacking Notes
=============

Parsing Bit Fields
------------------

The parser for a bit field parameter type should be very tolerant of the length of the token. Anything from a single bit and up should be permitted. The parser should use default values for bits that are not provided and ignore extra bits.

Here is a model function:

     void
     prot_a_parse_bitfield(Connection *client,
                           Bitfield   *res)
     {
         String          token;
         String_size     len;

         token = prot_a_get_token(client);
         len = s_strlen(token);
         if (len <= 0)
             longjmp(parse_env, ISC_PROTOCOL_ERR);

         init_bitfield(res);
         switch (len = s_strlen(token))
         {
         default:
         case 8: res->bit_8 = token.string[7];
         case 7: res->bit_7 = token.string[6];
         case 6: res->bit_6 = token.string[5];
         case 5: res->bit_5 = token.string[4];
         case 4: res->bit_4 = token.string[3];
         case 3: res->bit_3 = token.string[2];
         case 2: res->bit_2 = token.string[1];
         case 1: res->bit_1 = token.string[0];
         }
     }

The function gets the token, checks the sanity of the length, then initialized the result to its default values. Then it does a switch on all token lengths that are equal to or smaller than the number of bits the server knows about. The fall-through ensures that all bits in the token are read.

Membership Notes
----------------

The position field in the membership is not stored. It has to be set every time a membership is requested for transmission to the client.

Linking Pairs of Aux Items
--------------------------

Sometimes two aux items need to work in tandem. The first instance of this was the FAQ and FAQ-for-conference items. The FAQ item contains the text number of a text that is a FAQ for a conference. The FAQ-for-conference item contains the conference for which a text is a FAQ. This is needed so that deletion of the text properly removes the aux-item on the conf (plus, it's nice to be able to see that a text is a FAQ.)

The linked_item field in the Aux_item structure is for linking items. The linking must be managed through the use of triggers. This field is not visible in the protocol. It is saved in the database. It is not possible to have more than one link per item.

Please remember the following points.

  * The target of a link should have a link back. All links need to go both ways.

  * In the add trigger for one end, create the other end of the link and set the linked_item field in both items. Don't forget to mark the objects at both ends as changed.

  * Deletion and undeletion of the other side of the link will be managed automatically. You don't need delete and undelete triggers simply to destroy the other side of a link.

  * Don't kill the server because one end is missing. It is possible for the administrator to remove an item manually. Log a message and continue working.

Notes for fncdef.txt
--------------------

The fncdef.txt file is used to define the RPC functions. Each line consists of the call number, the return type of the call, the parameters and the output types of the call.

Some examples:

         10 number  create_conf_old c_string (param.conf_name_len) conf_type
         12 success lookup_name c_string (param.conf_name_len) : conf_list

The first line defines a call named create_conf_old that takes two arguments, a string that can only be as long as param.conf_name_len and a conf_type. It returns a number to the client. If the service call returns -1, the server will return an error. The create_conf_old call has RPC number 10.

The second line defines a call named lookup_name that takes a string argument that can be no longer than param.conf_name_len, and returns a conf_list. The service call returns a Success. If it does not return OK, the server will return an error. The lookup_name call has RPC number 12.

Scripts That Use fncdef.txt
...........................

The following scripts operate on fncdef.txt. If you make modifications to the format of fncdef.txt, you have to update these scripts.

call-switch.awk
     Generates call-switch.incl, which is included by connections.c

com-h.awk
     Generates com.h, which is included by several files.

fnc-def-init.awk
     Generates fnc-def-init.incl, which is included by connections.c.

prot-a-is-legal-fnc.awk
     Generates prot-a-is-legal-fnc.incl, which is included by prot-a.c

prot-a-parse-arg-c.awk
prot-a-parse-arg-h.awk
     Generates prot-a-parse-arg.c and prot-a-parse-arg.h.

Traversing Connections
----------------------

Since session 0 is interpreted as the currently active session by get_conn_by_number it is important to be careful when traversing sessions. Code like this does not work since it will do one iteration through the loop with sess set to zero. This formerly caused get_conn_by_number to return NULL, but now causes it to return the session pointer for the current session.

             for (sess = 0; (sess = traverse_connections(sess)) != 0; )
             {
                 cptr = get_conn_by_number(sess);
                 ...
             }

The canonical traversal code looks like this:

             Session_no      session = 0;
             while ((session = traverse_connections(session)) != 0)
             {
                 cptr = get_conn_by_number(session);
                 if (handshake_ok(cptr, 0)) /* can sometimes be skipped */
                 {
                    ...
                 }
             }

This code has session set to a session number before ever entering the loop.

Debugging and Testing
=====================

We're slowly adding support for debugging and testing lyskomd properly.

The Test Suite
--------------

The lyskomd test suite is in src/server/testsuite. Please extend this with additional test cases every time you make modifications to the server. Run the test suite often to make sure that your changes did not break anything.

The file config/prot-a.exp contains some support for protocol A. Don't use these functions in test cases. Use them to set up the inital database and for things that have to be done, such as logins and enabling privileges, but that don't need to be tested. Also, don't count on all the code in prot-a.exp to be fully functional. Add new functions to this file as you see fit.

The basic structure of a test case is the following:

     source "config/prot-a.exp"
     read_versions

     lyskomd_start
     client_start 0
     talk_to client 0
     kom_connect "DejaGnu test suite"

     The test cases

     talk_to client 0
     kom_logout
     kom_login 5 [holl "gazonk"] 0
     kom_enable 255
     send "9999 44 0\n"
     simple_expect "=9999"

     client_death 0
     lyskomd_death

Use the existing test cases as templates.

Configuration Options
---------------------

There are several testing and debugging-related configuration options for lyskomd. Some of them also apply to libisc.

--with-purify
     Build lyskomd with Purify. This currently does not work.

--with-efence
     Build lyskomd with Electric Fence for checking buffer overruns. This option does work.

--with-checker
     Build lyskomd with Gnu Checker for checking memory accesses, leaks, file descriptors and all kinds of stuff. As of Checker version 0.99.6, Gnu Checker cannot deal with lyskomd. Once built, and this requires modifications to Checker (at least on Linux) it reports spurious errors. Still, the option is here for those who want to try it out.

--with-gcov
     Build lyskomd with instrumentation for gcov. You have to use this option if you want to run gcov on lyskomd. For gcov to be effective, you should turn off optimization as well.

--with-traced-allocations
     There is some builtin support for detecting memory leaks in lyskomd. Whenever the server exits normally it reports how much memory it still uses to var/lyskomd.memory.  The count should always be 0.

     If there is a leak you can use this option to trace it down.  See src/server/ram-smalloc.c for more information.  You need gdb and a lot of time to use this option.

--with-optimization=VALUE
     Build lyskomd with the specified level of optimization. Use either numeric values to select the level of optimization, or say --with-optimization=no or --without-optimization to turn optimization off.

Coverage Testing
----------------

When you write new code, make sure that it is completely covered by the test suite. Run the lyskomd configure script with the --with-gcov, --with-debug-calls and --without-optimization flags to instrument the server for coverage testing with gcov.

If you run configure without the --without-optimization option, the server will be compiled with optimizations on. This is fine, but the coverage data from gcov isn't completely reliable since parts of the program may have been optimized out of existance.

Recompile everything, then run the test suite. Next do gcov -f FILENAME to compute coverage information for the file FILENAME. The resulting file FILENAME.gcov shows which lines have been executed, and which haven't been run. Try to get 100% coverage.

Debug Calls
-----------

Run the configure script with --with-debug-calls to compile in support for debugging calls in the server. These calls are strictly for making testing easier (or possible.) They are not official, and they may change at any time.

memory-info (DEBUG) Experimential
.................................

             memory-info [1000] ( )
                     -> (( arena     :       INT32;
                           ordblks   :       INT32;
                           smblks    :       INT32;
                           hblks     :       INT32;
                           hblkhd    :       INT32;
                           usmblks   :       INT32;
                           fsmblks   :       INT32;
                           uordblks  :       INT32;
                           fordblks  :       INT32;
                           keepcost  :       INT32; ));

This call returns the data returned by mallinfo in the server. See the man page for mallinfo for explanations of the fields.

set-marks (DEBUG) Experimental
..............................

             set-marks [1001] (( text-no     :   Text_no;
                                 no-of-marks :   INT32; ))
                     -> ( );

Set the number of marks on text text-no to no-of-marks, regardless of how many marks it really has. This call is useful for forcing the database into a state where the number of marks is incorrect in some way.

backdate-text (DEBUG) Experimental
..................................

             backdate-text [1002] (( text-no :   Text_no;
                                     seconds :   INT32; ))
                     -> ( );

Backdate a text in the server. Change the creation date of text text-no so it appears to have been created seconds earlier than it was actually created. This can be used to test the garbage collector.

The local-to-global structure
=============================

The data structure that stores the mapping from local to global text numbers is currently one of the more advanced structures used by lyskomd.

This section is not translated to English yet.  See a comment in the lyskomd.texi for the raw Swedish text.

Coding conventions
==================

When I write this chapter, lyskomd is over 13 years old.  It shows. The code looks differently.  Here are a few notes on the coding style that is preferred.  When you make a substantial change in a function, please update it to this style as well.

  * Keep lines to at most 79 characters.  (Exception: the DejaGNU test scripts can be as long as you like.  Proper TCL quoting is a bigger nuisance than overly long lines.)

  * Don't make invisible whitespace changes: don't change tab to space or vice versa, don't add or remove trailing whitespace.  It makes the output of cvs diff harder to read.

  * Document the public API of a function in the .h file, not the .c file.

  * Don't use Bool to return a failure indication.  Use Success instead.  That type has two values: OK and FAILURE.  They cannot be mistaken, but it isn't obvious if TRUE means that some operation failed or if it succeeded.

  * If a function returns Bool, its name should make it clear what TRUE means.  Don't call the function validate_existing_text; call it validate_text_exists. (But it might be better to use Success.)

  * Follow this indentation example:

          int
          some_fun(int x,
                   int y)
          {
              if (x > y)
                  return x;
              else
              {
                  if (y == 0
                      && x < y)
                  {
                      return x;
                  }
              }

              return 0;
          }

  * Don't use redundant braces (as in the first if statement above), unless they help readability (as they do in the second if statement above).

  * If you break a line near a && or || (or any other operator), put the operator at the beginning of the next line.

  * A suitable style for GNU Emacs can be found in doc/kom-style.el.

lyskomd Database Specification
******************************

This document specifies the format of the lyskomd database files. The specification is currently incomplete. Only the structure, not the actual data records are documented.

Data File Version 0
===================

Version 0 was used by lyskomd versions up to 1.8.

     database        :       header /NL/
                             next-free-num /NL/
                             confs
                             persons
                             next-text-num /NL/
                             texts
                     ;
     header          :       'CLEAN'
                     |       'DIRTY'
                     ;
     next-free-num   :       /INTEGER/
                     ;
     confs           :       confs conf
                     |       /empty/
                     ;
     conf            :       empty-record
                     |       '+' conf-record /NL/
                     ;
     persons         :       persons person
                     |       /empty/
                     ;
     person          :       empty-record
                     |       '+' person-record /NL/
                     ;
     next-text-num   :       /INTEGER/
                     ;
     texts           :       texts text
                     |       /empty/
                     ;
     text            :       empty-record
                     |       '+' text-record 'NL'
                     ;
     empty-record    :       '@' 'NL'
                     ;

The number of person and conference records is exactly one less than /next-free-num/. The number of text records is exactly one less than /next-text-num/.

Records are stored sequentially. Conference number 18 is the 18th conference record in the file. This implies that deleted records must be stored as /empty-record/ records.

/next-text-num/ is the number of the highest text. There are exactly one less than this number of text records in the database.

/next-free-num/ is the number of the highest conference. There are exactly one less than this number of both person and conference records. This implies that if conference N is not a letterbox, then person record N will be an /empty-record/.

If the header says ``CLEAN'', the database file is complete. If the header says ``DIRTY'', the server has not finished writing it completely.

Data File Version 1
===================

Version 1 was used by lyskomd version 1.9.

     database        :       header 'NL' 'records'
                     ;
     header          :       'CLEAN:00001'
                     |       'DIRTY:00001'
                     ;
     records         :       records record
                     |       /empty/
                     ;
     record          :       next-free-num
                     |       next-text-num
                     |       conference
                     |       person
                     |       info
                     |       text
                     |       deleted
                     ;
     next-free-num   :       '#C' /INTEGER/
                     ;
     next-text-num   :       '#P' /INTEGER/
                     ;
     conference      :       'C' /integer/ conf-record 'NL'
                     |       'P' /integer/ person-record 'NL'
                     |       'T' /integer/ text-record 'NL'
                     |       'I' info-record 'NL'
                     ;
     deleted         :       '-C' /integer/ 'NL'
                     |       '-P' /integer/ 'NL'
                     |       '-T' /integer/ 'NL'
                     ;

The integer in the conference, text and person records is the ID of the record. This implies that records can be in any order.

The /next-free-num/ record is used to store the next available ID for conferences in the system. There may be several of these records in the database.

The /next-text-num/ record is used to store the next available ID for texts in the system. There may be several of these records in the database.

A conference or text must have a number lower than the closest /next-free-num/ or /next-text-num/ preceding it.

The deletion records are used to indicate that an object found earlier in the database has been deleted. The implementation of these in lyskomd 1.9 does not work, and they are not used. The /-C/ record indicates deletion of a conference. The /-P/ record indicates deletion of a person. The /-T/ record indicates deletion of a text. The integer in the deletion record is the ID of the object being deleted.

Data File Version 2
===================

Version 2 is used by lyskomd version 2.0.

The structure of the data file is similar to version 1. The header has been extended with a timestamp contaning the time when the database file was saved. This timestamp consists of twenty characters, the number of seconds since 00:00:00 GMT January 1, 1970 (a Unix time_t.)

     database        :       header 'NL' 'records'
                     ;
     header          :       'CLEAN:00001' 'NL' timestamp 'NL'
                     |       'DIRTY:00001' 'NL' timestamp 'NL'
                     ;
     timestamp       :       digit digit digit digit digit
                             digit digit digit digit digit
                             digit digit digit digit digit
                             digit digit digit digit digit
                     ;
                     ;
     digit           :       '0' | '1' | '2' | '3' | '4'
                             | '5' | '6' | '7' | '8' | '9'
                     ;

Furthermore several data types have been changed to accommodate additions introduced in version 10 of protocol A. The /server-info/, /conf-record/ and /text-record/ include information about aux-items (highest-aux-no and aux-item-list.) The /conf-record/ contains the expire field added to the conf-stat structure. The /conf-record/ and /person-record/ records use the new local-to-global structure for storing maps.