Roundup
Roundup jump to Administration Guide

Administration Guide

What does Roundup install?

There’s two “installations” that we talk about when using Roundup:

  1. The installation of the software and its support files. This uses the standard Python mechanism called “setuptools” and thus Roundup’s core code, executable scripts and support data files are installed in Python’s directories. On Windows, this is typically:

    Scripts

    <python dir>\scripts\...

    Core code

    <python dir>\lib\site-packages\roundup\...

    Support files

    <python dir>\share\roundup\...

    and on Unix-like systems (eg. Linux):

    Scripts

    <python root>/bin/...

    Core code

    <python root>/lib-<python version>/site-packages/roundup/...

    Support files

    <python root>/share/roundup/...

  2. The installation of a specific tracker. When invoking the roundup-admin “inst” (and “init”) commands, you’re creating a new Roundup tracker. This installs configuration files, HTML templates, detector code and a new database. You have complete control over where this stuff goes through both choosing your “tracker home” and the main -> database variable in the tracker’s config.ini.

Configuring Roundup’s Logging of Messages For Sysadmins

You may configure where Roundup logs messages in your tracker’s config.ini file. Roundup will use the standard Python (2.3+) logging implementation.

Configuration for standard “logging” module:
  • tracker configuration file specifies the location of a logging configration file as logging -> config
  • roundup-server specifies the location of a logging configuration file on the command line
Configuration for “BasicLogging” implementation:
  • tracker configuration file specifies the location of a log file logging -> filename
  • tracker configuration file specifies the level to log to as logging -> level
  • roundup-server specifies the location of a log file on the command line
  • roundup-server specifies the level to log to on the command line

(roundup-mailgw always logs to the tracker’s log file)

In both cases, if no logfile is specified then logging will simply be sent to sys.stderr with only logging of ERROR messages.

Configuring roundup-server

The basic configuration file is as follows (taken from the roundup-server.ini.example file in the “doc” directory):

[main]

# Host name of the Roundup web server instance.
# If left unconfigured (no 'host' setting) the default
# will be used.
# If empty, listen on all network interfaces.
# If you want to explicitly listen on all
# network interfaces, the address 0.0.0.0 is a more
# explicit way to achieve this, the use of an empty
# string for this purpose is deprecated and will go away
# in a future release.
# Default: localhost
host = localhost

# Port to listen on.
# Default: 8080
port = 8017

# Path to favicon.ico image file.  If unset, built-in favicon.ico is used.
# The path may be either absolute or relative
# to the directory containing this config file.
# Default: favicon.ico
favicon = favicon.ico

# User ID as which the server will answer requests.
# In order to use this option, the server must be run initially as root.
# Availability: Unix.
# Default:
user = roundup

# Group ID as which the server will answer requests.
# In order to use this option, the server must be run initially as root.
# Availability: Unix.
# Default:
group =

# Maximum number of children to spawn using fork multiprocess mode.
# Default: 40
max_children = 40

# don't fork (this overrides the pidfile mechanism)'
# Allowed values: yes, no
# Default: no
nodaemon = no

# Log client machine names instead of IP addresses (much slower)
# Allowed values: yes, no
# Default: no
log_hostnames = no

# Have http(s) request logging done via python logger module.
# If set to yes the python logging module is used with qualname
# 'roundup.http'. Otherwise logging is done to stderr or the file
# specified using the -l/logfile option.
# Allowed values: yes, no
# Default: no
loghttpvialogger = no

# File to which the server records the process id of the daemon.
# If this option is not set, the server will run in foreground
#
# The path may be either absolute or relative
# to the directory containing this config file.
# Default:
pidfile =

# Log file path.  If unset, log to stderr.
# The path may be either absolute or relative
# to the directory containing this config file.
# Default:
logfile =

# Set processing of each request in separate subprocess.
# Allowed values: debug, none, thread, fork.
# Default: fork
multiprocess = fork

# Tracker index template. If unset, built-in will be used.
# The path may be either absolute or relative
# to the directory containing this config file.
# Default:
template =

# Enable SSL support (requires pyopenssl)
# Allowed values: yes, no
# Default: no
ssl = no

# PEM file used for SSL. A temporary self-signed certificate
# will be used if left blank.
# The path may be either absolute or relative
# to the directory containing this config file.
# Default:
pem =

# Comma separated list of extra headers that should
# be copied into the CGI environment.
# E.G. if you want to access the REMOTE_USER and
# X-Proxy-User headers in the back end,
# set to the value REMOTE_USER,X-Proxy-User.
# Allowed values: comma-separated list of words
# Default:
include_headers =

# Change to HTTP/1.0 if needed. This disables keepalive.
# Default: HTTP/1.1
http_version = HTTP/1.1

# Roundup trackers to serve.
# Each option in this section defines single Roundup tracker.
# Option name identifies the tracker and will appear in the URL.
# Option value is tracker home directory path.
# The path may be either absolute or relative
# to the directory containing this config file.
[trackers]

demo = /trackers/demo
sysadmin = /trackers/sysadmin

Additional notes for each keyword:

template
Specifies a template used for displaying the tracker index when multiple trackers are being used. It is processed by TAL and the variable “trackers” is available to the template and is a dict of all configured trackers.
ssl
Enables use of SSL to secure the connection to the roundup-server. In most cases, you will want to run a real web server (Apache, Nginx) as a proxy to roundup-server running without SSL. The real web server can filter/rate limit/firewall requests to roundup-server. If you enable this, ensure that your tracker’s config.ini specifies an https URL. See roundup-server.1 man page for additional information.
pem
If specified, the SSL PEM file containing the private key and certificate. The file must include both the private key and certificate with appropriate headers (e.g. -----BEGIN PRIVATE KEY-----, -----END PRIVATE KEY----- and -----BEGIN CERTIFICATE-----, -----END CERTIFICATE-----. If not specified, roundup will generate a temporary, self-signed certificate for use.
trackers section
Each line denotes a mapping from a URL component to a tracker home. Make sure the name part doesn’t include any url-unsafe characters like spaces. Stick to alphanumeric characters and you’ll be ok.

To generate a config.ini in the current directory from the roundup-server command line use:

roundup_server -p 8017  -u roundup --save-config  demo=/trackers/demo \
   sysadmin=/trackers/sysadmin

Note it will save an old config.ini file to config.bak and create a new config.ini. The file is recreated from scratch ignoring the contents of the current config.ini. You may need to merge the backup and config files. save-config doesn’t attempt to load or verify an existing config.ini. Running this in a tracker home directory will move the exsiting config.ini to config.bak and replace it with the roundup-server’s config.ini. This will make the tracker in the directory fail to start util the original config.ini is restored.

Configuring Compression

Roundup will compress HTTP responses to clients on the fly. Dynamic, on the fly, compression is enabled by default, to disable it set:

[web]
...
dynamic_compression = No

in the tracker’s config.ini. You should disable compression if your proxy (e.g. nginx or apache) or wsgi server (uwsgi) is configured to compress responses on the fly. The python standard library includes gzip support. For brotli or zstd you will need to install packages. See the installation documentation for details.

Some assets will not be compressed on the fly. Assets with mime types of “image/png” or “image/jpeg” will not be compressed. You can add mime types to the list by using interfaces.py as discussed in the customisation documentation. As an example adding:

from roundup.cgi.client import Client

Client.precompressed_mime_types.append('application/zip`)

to interfaces.py will prevent zip files from being compressed.

Any content less than 100 bytes in size will not be compressed (e.g errors messages, short json responses).

Zstd will be used if the client can understand it, followed by brotli then gzip encoding. Currently the preference order is hard coded into the server and not parsed using q values from the client’s Accept-Encoding header. This is an area for improvement.

In addition to dynamic compression, static files/assets accessed using @@file can be pre-compressed. This reduces CPU load on the server and reduces the time required to respond to the client. By default searching for pre-compressed files is disabled. To enable it set:

[web]
...
use_precompressed_files = Yes

in the tracker’s config.ini file. Then you can create a precompressed file and it will be served if the client is able to accept it. For a file .../@@file/library.js you can create:

tracker_home/html/library.js.gzip
tracker_home/html/library.js.br
tracker_home/html/library.js.zstd

which should be created by using (respectively):

gzip --keep --suffix .gzip library.js
brotli library.js
zstd library.js && mv library.js.zst library.js.zstd

see the man pages for options that control compression level. Note that some levels require additional memory on the client side, so you may not always want to use the highest compression available.

A pre-compressed file will not be used if its modified date is earlier than the uncompressed file. For example, if library.js.gzip is older (has earlier modification date) than library.js, library.js.gzip will be ignored. library.js will be served instead. library.js will be dynamically compressed on the fly and a warning message will be logged.

Precompressed files override dynamic compression. For example, assume the client can accept brotli and gzip. If there are no precompressed files, the data will be compressed dynamically (on the fly) using brotli. If there is a precompressed gzip file present the client will get the gzip version and not a brotli compressed version. This mechanism allows the admin to allow use of brotli and zstd for dynamic content, but not for static content.

Adding a Web Content Security Policy (CSP)

A Content Security Policy (CSP) adds a layer of security to Roundup’s web interface. It makes it more difficult for an attacker to compromise Roundup. By default Roundup does not add a CSP. If you need to implement a CSP, this section will help you understand how to add one and document the current level of support for CSP in Roundup.

Roundup’s web interface has remained mostly unchanged since it was created over a decade ago. Current releases have been slowly modernizing the HTML to improve security. There are still some improvements that need to happen before the tightest CSP configurations can be used.

Writing a CSP is complex. This section just touches on how to create and install a CSP to improve security. Some of it might break functionality.

There are two ways to add a CSP:

  1. a fixed CSP added by a server
  2. a dynamic CSP added by Roundup

Fixed CSP

If you are using a web server (Apache, Nginx) to run Roundup, you can add a Content-Security-Policy header using that server. WSGI servers like uWSGI can also be configured to add headers. An example header would look like:

Content-Security-Policy: default-src 'self' 'unsafe-inline' 'strict-dynamic';

One thing that may need to be included is the unsafe-inline. The default templates use onload, onchange, onsubmit, and onclick JavaScript handlers. Without unsafe-inline these won’t work and popup helpers will not work. Sadly the use of unsafe-inline is a pretty big hole in this CSP. You can set the hashes for all the JavaScript handlers in the CSP. Then replace unsafe-inline with unsafe-hashes to help close this hole, but has its own issues. See remediating unsafe-inline for another way to mitigate this.

The inclusion of strict-dynamic allows trusted JavaScript files that are downloaded from Roundup to make changes to the web interface. These changes are also trusted code that will be run when invoked.

More secure CSPs can also be created. However because of the ability to customise the web interface, it is difficult to provide guidance.

Dynamic CSP

Roundup creates a cryptographic nonce for every client request. The nonce is the value of the client.client_nonce property.

By changing the templates to use the nonce, we can better secure the Roundup instance. However the nonce has to be set in the CSP returned by Roundup.

One way to do this is to add a templating utility to the extensions directory that generates the CSP on the fly. For example:

default_security_headers = {
    'Content-Security-Policy': (
        "default-src 'self'; "
        "base-uri 'self'; "
        "script-src https: 'nonce-{nonce}' 'strict-dynamic'; "
        "style-src 'self' 'nonce-{nonce}'; "
        "img-src 'self' data:; "
        "frame-ancestors 'self'; "
        "object-src 'self' 'nonce-{nonce}'; "
    ),
}


def AddHtmlHeaders(client, header_dict=None):
    ''' Generate https headers from dict use default security headers

        Setting the header with a value of None will not inject the
        header and can override the default set.

        Header values will be formatted with a dictionary including a
        nonce. Use to set a nonce for inline scripts.
    '''
    try:
        if client.client_nonce is None:
            # logger.warning("client_nonce is None")
            client.client_nonce = client.session_api._gen_sid()
    except AttributeError:
        # client.client_nonce doesn't exist, create it
        # logger.warning("client_nonce does not exist, creating")
        client.client_nonce = client.session_api._gen_sid()

    headers = default_security_headers.copy()
    if isinstance(header_dict, dict):
        headers.update(header_dict)

    client_headers = client.additional_headers

    for header, value in list(headers.items()):
        if value is None:
            continue
        client_headers[header] = value.format(
            nonce=client.client_nonce)

def init(instance):
    instance.registerUtil('AddHtmlHeaders', AddHtmlHeaders)

Adding the following to page.html right after the opening <html....`> tag:

<tal:code tal:content="python:utils.AddHtmlHeaders(request.client)" />

will invoke AddHtmlHeaders() to add the CSP header with the nonce.

With this set of CSP headers, all style, script and object tags will need a nonce attribute. This can be added by changing:

<script src="javascript.js"></script>

to:

<script
    tal:attributes="nonce request/client/client_nonce"
    src="javascript.js"></script>

for each script, object or style tag.

Remediating unsafe-inline

Using a trusted script to set event handlers to replace the onX handlers allows removal of the unsafe-inline handlers. If you remove unsafe-inline the onX handlers will not run. However you can use the label provided by the onX attribute to securely enable a callback function.

This method is a work in progress. As an example proof of concept, adding this “decorator” script at the end of page.html:

<script tal:attributes="nonce request/client/client_nonce">
 /* set submit event listener on forms that have an
 onsubmit (case insensitive) attribute */
 forms = document.querySelectorAll(form[onsubmit])
 for (let form of f) {
     form.addEventListener('submit',
                           () => submit_once());
  };
</script>

will set callback for the submit even on any form that has an onsubmit attribute to submit_once(). submit_once is defined in Roundup’s base_javascript and is generated with a proper nonce.

By including the nonce in the dynamic CSP, we can use our trusted “decorator” script to add event listeners. These listeners will call the trusted function in base_javascript to replace the ignored onX handlers.

Configuring Session Databases

The session and OTK (one time key) databases store information about the operation of Roundup. This ephemeral data:

  • web login session keys,
  • CSRF tokens,
  • email password recovery one time keys,
  • rate limiting data,

can be a performance bottleneck. It usually happens with anydbm or SQLite backends. PostgreSQL and MySQL are sufficiently powerful that they can handle the higher transaction rates.

If you are using sqlite, you can choose to use the anydbm database for session data. By default it will use additional sqlite databases for storing the session and otk data.

The following table shows which primary databases support different session database backends:

D - default if unconfigured, X - compatible choice
  session db
primary db anydbm sqlite redis mysql postgresql
anydbm D   X    
sqlite X D X    
mysql       D  
postgresql         D

The backend setting is in the tracker’s config.ini file under the sessiondb section.

Using Redis for Session Databases

Redis is an in memory key/value data structure store.

You need to install the redis-py module from pypi. Then install Redis using your package manager or by downloading it from the Redis website.

You need to secure your redis instance. The data that Roundup stores includes session cookies and other authentication tokens. At minimum you should require a password to connect to your redis database. Set requirepass in redis.conf. Then change the redis_url in config.ini to use the password.

For example:

redis://:mypassword@localhost:7200/10

will connect to the redis instance running on localhost at port 7200 using the password mypassword to open database 10. The redis_url setting can load a file to better secure the url. If you are using redis 6.0 or newer, you can specify a username/password and access control lists to improve the security of your data. Another good alternative is to talk to redis using a Unix domain socket.

If you are connecting to redis across the network rather than on localhost, you should configure ssl/tls and use the rediss scheme in the url along with the query parameters:

ssl_cert_reqs=required&ssl_ca_certs=/path/to/custom/ca-cert

where you specify the file that can be used to validate the SSL certificate. Securing Redis has more details.

Users and Security

Roundup holds its own user database which primarily contains a username, password and email address for the user. Roundup must have its own user listing, in order to maintain internal consistency of its data. It is a relatively simple exercise to update this listing on a regular basis, or on demand, so that it matches an external listing (eg. unix passwd file, LDAP, etc.)

Roundup identifies users in a number of ways:

  1. Through the web, users may be identified by either HTTP Basic Authentication or cookie authentication. If you are running the web server (roundup-server) through another HTTP server (eg. apache or IIS) then that server may require HTTP Basic Authentication, and it will pass the REMOTE_USER variable (or variable defined using http_auth_header) through to Roundup. If this variable is not present, then Roundup defaults to using its own cookie-based login mechanism.
  2. In email messages handled by roundup-mailgw, users are identified by the From address in the message.

In both cases, Roundup’s behaviour when dealing with unknown users is controlled by Permissions defined in the “SECURITY SETTINGS” section of the tracker’s schema.py module:

Web Access and Register
If granted to the Anonymous Role, then anonymous users will be able to register through the web.
Email Access and Register
If granted to the Anonymous Role, then email messages from unknown users will result in those users being registered with the tracker.

More information about how to customise your tracker’s security settings may be found in the reference documentation.

Configuring Authentication Header/Variable

The front end server running Roundup can perform the user authentication. It pass the authenticated username to the backend in a variable. By default roundup looks for the REMOTE_USER variable This can be changed by setting the parameter http_auth_header in the [web] section of the tracker’s config.ini file. If the value is unset (the default) the REMOTE_USER variable is used.

If you are running roundup using roundup-server behind a proxy that authenticates the user you need to configure roundup-server to pass the proper header to the tracker. By default roundup-server looks for the REMOTE_USER header for the authenticated user. You can copy an arbitrary header variable to the tracker using the -I option to roundup-server (or the equivalent option in the roundup-server config file).

For example to use the uid_variable header, two configuration changes are needed: First configure roundup-server to pass the header to the tracker using:

roundup-server -I uid_variable ....

note that the header is passed exactly as supplied by the upstream server. It is not prefixed with HTTP_ like other headers since you are explicitly whitelisting the header. Multiple comma separated headers can be passed to the -I option. These could be used in a detector or other tracker extensions, but only one header can be used by the tracker as an authentication header.

To make the tracker honor the new variable changing the tracker config.ini to read:

[web]
...
http_auth_header = uid_variable

At the time this is written, support is experimental. If you use it you should notify the roundup maintainers using the roundup-users mailing list.

Securing Secrets

Roundup can read secrets from a file that is referenced from any of the config.ini files. If you use Docker, you can bind mount the files from a secure location, or store them in a subdirectory of the tracker home.

You can also use a secrets management tool like Docker Swarm’s secrets management. This example config.ini configuration gets the database password from a file populated by Swarm secrets:

[rdbms]
# Database user password.
# A string that starts with 'file://' is interpreted as a file
# path relative to the tracker home. Using 'file:///' defines
# an absolute path. The first line of the file will be used as
# the value. Any string that does not start with 'file://' is
# used as is. It removes any whitespace at the end of the
# line, so a newline can be put in the file.
#
# Default: roundup
password = file:///run/secrets/db_password

assuming that Docker Swarm secrets has the key db_password and the --secret db_password option is used when starting the Roundup service.

Because environment variables can be inadvertently exposed in logs or process listings, Roundup does not currently support loading secrets from environment variables.

Tasks

Maintenance of Roundup can involve one of the following:

  1. tracker backup
  2. software upgrade
  3. migrating backends
  4. moving a tracker
  5. migrating from other software
  6. adding a user from the command-line

Tracker Backup

The roundup-admin import and export commands are not recommended for performing backup.

Optionally stop the web and email frontends and to copy the contents of the tracker home directory to some other place using standard backup tools. This means using pg_dump to take a snapshot of your Postgres backend database, for example. A simple copy of the tracker home (and files storage area if you’ve configured it to be elsewhere) will then complete the backup.

Software Upgrade

Always make a backup of your tracker before upgrading software. Steps you may take:

  1. Install pytest and ensure that the unit tests run on your system (using your preferred python version):

    pip2 install pytest
    python2 -m pytest test/
    
    
    pip3 install pytest
    python3 -m pytest test/
    
  2. If you’re using an RDBMS backend, make a backup of its contents now.

  3. Make a backup of the tracker home itself.

  4. Stop the tracker web and email frontends.

  5. Install the new version of the software:

    python setup.py install
    
  6. Follow the steps in the upgrading documentation for all the versions between your original version and the new version.

    Usually you should run roundup_admin -i <tracker_home> migrate on your tracker(s) before you allow users to start accessing the tracker.

    It’s safe to run this even if it’s not required, so just get into the habit.

  7. Restart your tracker web and email frontends.

If something bad happens, you may reinstate your backup of the tracker and reinstall the older version of the sofware using the same install command:

python setup.py install

Migrating Backends

  1. Stop the existing tracker web and email frontends (preventing changes).

  2. Use the roundup-admin tool “export” command to export the contents of your tracker to disk. (If you are running on windows see issue1441336 on how to use the command line rather than interactive mode to export data.)

  3. Copy the tracker home to a new directory.

  4. Delete the “db” directory from the new directory.

  5. Set the value of the backend key under the [database] section of the tracker’s config.ini file.

  6. Use the roundup-admin “import” command to import the previous export with the new tracker home. If non-interactively:

    roundup-admin -i <tracker home> import <tracker export dir>
    

    If interactively, enter ‘commit’ before exiting.

  7. Test each of the admin tool, web interface and mail gateway using the new backend.

  8. Move the old tracker home out of the way (rename to “tracker.old”) and move the new tracker home into its place.

  9. Restart web and email frontends.

Moving a Tracker

If you’re moving the tracker to a similar machine, you should:

  1. install Roundup on the new machine and test that it works there,
  2. stop the existing tracker web and email frontends (preventing changes),
  3. copy the tracker home directory over to the new machine, and
  4. start the tracker web and email frontends on the new machine.

Most of the backends are actually portable across platforms (ie. from Unix to Windows to Mac). If this isn’t the case (ie. the tracker doesn’t work when moved using the above steps) then you’ll need to:

  1. install Roundup on the new machine and test that it works there,
  2. stop the existing tracker web and email frontends (preventing changes),
  3. use the roundup-admin tool “export” command to export the contents of the existing tracker,
  4. copy the export to the new machine,
  5. use the roundup-admin “import” command to import the tracker on the new machine, and
  6. start the tracker web and email frontends on the new machine.

Migrating From Other Software

You have a couple of choices. You can either use a CSV import into Roundup, or you can write a simple Python script which uses the Roundup API directly. The latter is almost always simpler – see the “scripts” directory in the Roundup source for some example uses of the API.

“roundup-admin import” will import data into your tracker from a directory containing files with the following format:

  • one colon-separated-values file per Class with columns for each property, named <classname>.csv

  • one colon-separated-values file per Class with journal information, named <classname>-journals.csv (this is required, even if it’s empty)

  • if the Class is a FileClass, you may have the “content” property stored in separate files from the csv files. This goes in a directory structure:

    <classname>-files/<N>/<designator>
    

    where <designator> is the item’s <classname><id> combination. The <N> value is int(<id> / 1000).

Adding A User From The Command-Line

The roundup-admin program can create any data you wish to in the database. To create a new user, use:

roundup-admin create user

To figure out what good values might be for some of the fields (eg. Roles) you can just display another user:

roundup-admin list user

(or if you know their username, and it happens to be “richard”):

roundup-admin filter user username=richard

then using the user id (e.g. 5) you get from one of the above commands, you may display the user’s details:

roundup-admin display <designator>

where designator is user5.

Running the Servers

Unix

On Unix systems, use the scripts/server-ctl script to control the roundup-server server. Copy it somewhere and edit the variables at the top to reflect your specific installation.

If you use systemd look at scripts/systemd.gunicorn. It is configured for a wsgi deployment using gunicorn, but may be a good starting point for your setup.

Windows

On Windows, the roundup-server program runs as a Windows Service, and therefore may be controlled through the Services control panel. Note that you must install the pywin32 package to allow roundup to run as a service. The roundup-server program may also control the service directly:

install the service
roundup-server -C /path/to/my/roundup-server.ini -c install
start the service
roundup-server -c start
stop the service
roundup-server -c stop

To bring up the services panel:

Windows 2000 and later
Start/Control Panel/Administrative Tools/Services
Windows NT4
Start/Control Panel/Services

You will need a server configuration file (as described in Configuring roundup-server) for specifying tracker homes and other roundup-server configuration. Specify the name of this file using the -C switch when installing the service.

Running the Mail Gateway Script

The mail gateway script should be scheduled to run regularly on your Windows server. Normally this will result in a window popping up. The solution to this is to:

  1. Create a new local account on the Roundup server
  2. Set the scheduled task to run in the context of this user instead of your normal login

Mail gateway script command line

Usage:

usage: roundup_mailgw.py [-h] [-v] [-c DEFAULT_CLASS] [-I OAUTH_CLIENT_ID]
                         [-O OAUTH_DIRECTORY] [-S SET_VALUE]
                         [-T OAUTH_TOKEN_ENDPOINT]
                         [args ...]

The roundup mail gateway may be called in one of three ways:

  • without arguments. Then the env var ROUNDUP_INSTANCE will be tried.
  • with an instance home as the only argument,
  • with both an instance home and a mail spool file, or
  • with an instance home, a mail source type and its specification.

It also supports optional -S (or --set-value) arguments that allows you to set fields for a class created by the roundup-mailgw. The format for this option is [class.]property=value where class can be omitted and defaults to msg. The -S options uses the same property=value[;property=value] notation accepted by the command line roundup command or the commands that can be given on the Subject line of an email message (if you’re using multiple properties delimited with a semicolon the class must be specified only once in the beginning).

It can let you set the type of the message on a per e-mail address basis by calling roundup-mailgw with different email addresses and other settings.

PIPE:
If there is no mail source specified, the mail gateway reads a single message from the standard input and submits the message to the roundup.mailgw module.
UNIX mailbox:

In this case, the gateway reads all messages from the UNIX mail spool file and submits each in turn to the roundup.mailgw module. The file is emptied once all messages have been successfully handled. The file is specified as:

mailbox /path/to/mailbox

In all of the following mail source types, the username and password can be stored in a ~/.netrc file. If done so, only the server name needs to be specified on the command-line. The username and/or password will be prompted for if not supplied on the command-line or in ~/.netrc.

POP:

For the mail source “pop”, the gateway reads all messages from the POP server specified and submits each in turn to the roundup.mailgw module. The server is specified as:

pop username:password@server

The username and password may be omitted:

pop username@server
pop server

are both valid.

POPS:

Connect to a POP server over tls/ssl. This supports the same notation as POP:

pops username:password@server
APOP:

Same as POP, but using Authenticated POP:

apop username:password@server
IMAP:

Connect to an IMAP server. This supports the same notation as that of POP mail:

imap username:password@server

It also allows you to specify a specific mailbox other than INBOX using this format:

imap username:password@server mailbox
IMAPS:

Connect to an IMAP server over tls/ssl. This supports the same notation as IMAP:

imaps username:password@server [mailbox]
IMAPS_CRAM:

Connect to an IMAP server over tls/ssl using CRAM-MD5 authentication. This supports the same notation as IMAP:

imaps_cram username:password@server [mailbox]
IMAPS_OAUTH:

Connect to an IMAP server over tls/ssl using OAUTH authentication. Note that this does not support a password in imaps URLs. Instead it uses only the user and server and a command-line option for the directory with the files access_token, refresh_token, client_secret, and client_id. By default this directory is oauth in your tracker home directory. The access token is tried first and, if expired, the refresh token together with the client secret is used to retrieve a new access token. Note that both token files need to be writeable, the access token is continuously replaced and some cloud providers may also renew the refresh token from time to time:

imaps_oauth username@server [mailbox]

The refresh and access tokens (the latter can be left empty), the client id and the client secret need to be retrieved via cloud provider specific protocols or websites.

You need the requests library installed to ue the IMAPS_OAUTH method.

Using roundup-admin

Part of the installation includes a man page for roundup-admin. Ypu should be able to read it using man roundup-admin. As shown above, it is a generic tool for manipulating the underlying database for you tracker.

Examples above show how to use it to:

  • install and initialize a new tracker
  • export/import tracker data for migrating between backends
  • creating a new user from the command line
  • list/find users in the tracker

The basic usage is:

Usage: roundup-admin [options] [<command> <arguments>]

Options:
 -i instance home  -- specify the issue tracker "home directory" to administer
 -u                -- the user[:password] to use for commands (default admin)
 -d                -- print full designators not just class id numbers
 -c                -- when outputting lists of data, comma-separate them.
                      Same as '-S ","'.
 -S <string>       -- when outputting lists of data, string-separate them
 -s                -- when outputting lists of data, space-separate them.
                      Same as '-S " "'.
 -V                -- be verbose when importing
 -v                -- report Roundup and Python versions (and quit)

 Only one of -s, -c or -S can be specified.

Help:
 roundup-admin -h
 roundup-admin help                       -- this help
 roundup-admin help <command>             -- command-specific help
 roundup-admin help all                   -- all available help

Commands:
 commit
 create classname property=value ...
 display designator[,designator]*
 export [[-]class[,class]] export_dir
 exporttables [[-]class[,class]] export_dir
 filter classname propname=value ...
 find classname propname=value ...
 genconfig <filename>
 get property designator[,designator]*
 help topic
 history designator [skipquiet]
 import import_dir
 importtables export_dir
 initialise [adminpw]
 install [template [backend [key=val[,key=val]]]]
 list classname [property]
 migrate
 pack period | date
 perftest [mode] [arguments]*
 pragma setting=value | 'list'
 reindex [classname|classname:#-#|designator]*
 restore designator[,designator]*
 retire designator[,designator]*
 rollback
 security [Role name]
 set items property=value [property=value ...]
 specification classname
 table classname [property[,property]*]
 templates [trace_search]
 updateconfig <filename>
Commands may be abbreviated as long as the abbreviation
matches only one command, e.g. l == li == lis == list.

One thing to note, The -u user setting does not currently operate like a user logging in via the web. The user running roundup-admin must have read access to the tracker home directory. As a result the user has access to the files and the database info contained in config.ini.

Using -u user sets the actor/user parameter in the journal. Changes that are made are attributed to that user. The password is ignored if provided. Any existing username has full access to the data just like the admin user. This is an area for further development so that roundup-admin could be used with sudo to provide secure command line access to a tracker.

In general you should forget that there is a -u parameter.

All commands (except help, genconfig, templates) require a tracker specifier. This is just the path to the roundup tracker you’re working with. A roundup tracker is where roundup keeps the database and configuration file that defines an issue tracker. It may be thought of as the issue tracker’s “home directory”. It may be specified in the environment variable TRACKER_HOME or on the command line as “-i tracker”.

A designator is a classname and an itemid concatenated, eg. bug1, user10, … Property values are represented as strings in command arguments and in the printed results:

  • Strings are, well, strings.

  • Password values will display as their encoded value.

  • Date values are printed in the full date format in the local time zone, and accepted in the full format or any of the partial formats explained below.:

    Input of...        Means...
    "2000-04-17.03:45" 2000-04-17.03:45:00
    "2000-04-17"       2000-04-17.00:00:00
    "01-25"            yyyy-01-25.00:00:00
    "08-13.22:13"      yyyy-08-13.22:13:00
    "11-07.09:32:43"   yyyy-11-07.09:32:43
    "14:25"            yyyy-mm-dd.14:25:00
    "8:47:11"          yyyy-mm-dd.08:47:11
    "2003"             2003-01-01.00:00:00
    "2003-04"          2003-04-01.00:00:00
    "."                "right now"
    
  • Link values are printed as item designators. When given as an argument, item designators and key strings are both accepted.

  • Multilink values are printed as lists of item designators joined by commas. When given as an argument, item designators and key strings are both accepted; an empty string, a single item, or a list of items joined by commas is accepted.

When multiple items are specified to the roundup get or roundup set commands, the specified properties are retrieved or set on all the listed items. When multiple results are returned by the roundup get or roundup find commands, they are printed one per line (default) or joined by commas (with the “-c” option).

Where the command changes data, a login name/password is required. The login may be specified as either “name” or “name:password”.

  • ROUNDUP_LOGIN environment variable
  • the “-u” command-line option

If either the name or password is not supplied, they are obtained from the command-line.

The -u user setting does not currently operate like a user logging in via the web. The user running roundup-admin must have read access to the tracker home directory. As a result the user has access to the files and the database info contained in config.ini.

Using -u user sets the actor/user parameter in the journal. Changes that are made are attributed to that user. The password is ignored if provided. Any existing username has full access to the data just like the admin user. This is an area for further development so that roundup-admin could be used with sudo to provide secure command line access to a tracker.

When you initialise a new tracker instance you are prompted for the admin password. If you want to initialise a tracker non-interactively you can put the initialise command and password on the command line. But this allows others on the host to see the password (using the ps command). To initialise a tracker non-interactively without exposing the password, create a file (e.g init_tracker) set to mode 600 (so only the owner can read it) with the contents:

initialise admin_password

and feed it to roundup-admin on standard input. E.G.

cat init_tracker | roundup-admin -i tracker_dir

(for more details see https://issues.roundup-tracker.org/issue2550789.)

Using with the shell

With version 0.6.0 or newer of roundup (which introduced support for multiple designators to display and the -d, -S and -s flags):

To find all messages regarding chatting issues that contain the word “spam”, for example, you could execute the following command from the directory where the database dumps its files:

shell% for issue in `roundup-admin -ds find issue status=chatting`; do
> grep -l spam `roundup-admin -ds ' ' get messages $issue`
> done
msg23
msg49
msg50
msg61
shell%

Or, using the -dc option, this can be written as a single command:

shell% grep -l spam `roundup get messages \
    \`roundup -dc find issue status=chatting\``
msg23
msg49
msg50
msg61
shell%

You can also display issue contents:

shell% roundup-admin display `roundup-admin -dc get messages \
           issue3,issue1`
files: []
inreplyto: None
recipients: []
author: 1
date: 2003-02-16.21:23:03
messageid: None
summary: jkdskldjf
files: []
inreplyto: None
recipients: []
author: 1
date: 2003-02-15.01:59:11
messageid: None
summary: jlkfjadsf

or status:

shell% roundup-admin get name `/tools/roundup/bin/roundup-admin \
      -dc -i /var/roundup/sysadmin get status issue3,issue1`
unread
deferred

or status on a single line:

shell% echo `roundup-admin get name \`/tools/roundup/bin/roundup-admin \
         -dc -i /var/roundup/sysadmin get status issue3,issue1\``
unread deferred

which is the same as:

shell% roundup-admin -s get name `/tools/roundup/bin/roundup-admin \
         -dc -i /var/roundup/sysadmin get status issue3,issue1`
unread deferred

Also the tautological:

shell% roundup-admin get name \
   `roundup-admin -dc get status \`roundup-admin -dc find issue \
       status=chatting\``
chatting
chatting

Remember the roundup commands that accept multiple designators accept them ‘,’ separated so using ‘-dc’ is almost always required.