| Version 5.4
Introduction
Installation
SysAdmin
Network
Objects
Storage
E-mail
Real-Time
Access
Services
Directory
Clusters
Applications
Miscellaneous
Licensing
WebMail
Pronto!
PBX
Overview | Rules | Filters | SMTP | Local | RPOP | LIST | PIPE
# PIPE Module The Submitted Folder Delivering to External Applications Serialized Delivery Command Tags Configuring the PIPE module Foreign Queue Processing The PIPE module allows external applications running on the Server computer to submit messages to the CommuniGate Pro Server bypassing TCP/IP connections and Internet protocols, and it allows the Server to deliver messages to external applications. The PIPE module is also used to submit messages composed with the `mail` and `sendmail` programs that come with the CommuniGate Pro server software. These programs are designed as drop-in substitutions for legacy `mail` and `sendmail` programs.

## The Submitted Folder

The CommuniGate Pro PIPE module creates the `Submitted` folder inside the CommuniGate Pro base folder.

The PIPE module scans this folder periodically, and processes the files with the `.sub` file name extension. When such a file is found, the module copies it into a message queue file and submits that file to the Server kernel for processing.

The `.sub` text files should contain messages in the RFC822 format. The module uses the data in the RFC822 header fields to compose the message envelope.

- The RFC821 "channel format" should NOT be used: submitted files should use the system native "End of Line" character(s), the dot (.) symbol in the first position should not be doubled, and the file should not end with the dot (.) symbol.
- Addresses specified in the `To:`, `Cc:`, `Bcc:` header fields are used to create the message envelope (recipient addresses).
- The `Bcc:` header fields are removed from the submitted message.
- If at least one `Envelope-To:` header field is detected, message envelope (recipient) addresses are formed using the addresses specified in these header fields, and addresses in all remaining To, Cc, and Bcc headers are not placed into the message envelope. The `Envelope-To:` header fields are removed from the submitted message.
- If no `Envelope-To:` header field exists, and the `Envelope-Ignore` field or fields exist, the addresses specified in To/Cc/Bcc header fields and also listed in the Envelope-Ignore fields are NOT included into the message envelope.
- If the `Sender:` and/or `From:` header fields contain addresses without the domain part, the Server domain name is added to those addresses.
- If the `Return-Path:` header field exists, the address specified in that header is used to compose the Return-Path envelope address, and this header field is removed from the submitted message.
- If the `Return-Path:` header field does not exist, the address specified in the `From:` or `Sender:` header field is used to compose the envelope Return-Path address.
- If the `Envelope-ID:` header field exists, its content is used as the message Envelope ID. This header field is removed from the submitted message.
- If the `Envelope-Notify:` header field exists, it should contain one or several keywords `SUCCESS`,`FAILURE`,`DELAY`,`RELAY` separated with comma symbols, or it should contain one keyword `NEVER`.
  This header field specifies the DSN parameters applied to all envelope addresses composed after this field is met. This header field is removed from the submitted message.
  Several `Envelope-Notify:` header fields can be used to specify different DSN parameters for different addresses.

If processing of a `.sub` file fails (for example, if the file does not have any recipient address), the module places a record into the System Log, and changes the file extension to `.bad`.

If a .sub file is submitted successfully, the file is deleted from the Submitted folder.

Because of the way the PIPE module processes the Submitted folder, it is recommended to compose messages in a different file directory and then move the composed .sub files to the Submitted folder, or to compose messages in the Submitted folder, in files with the `.tmp` file name extension, and then change the file name extension to `.sub`.

Messages submitted via the PIPE module are marked as "received from a trusted source", so they can be relayed without restrictions.

The Submitted folder is used for Legacy Mail Emulation.

## Delivering to External Applications

The PIPE module accepts all messages directed to the `pipe` domain.

The local part of the message address specifies the external application to launch. The part can contain parameters, and can be enclosed into the quotation marks.

Example:
  A message directed to the `"execjoe -l store"@pipe` address will be sent to the application `execjoe` started with the `-l store` parameters.

You usually use the PIPE delivery via the Router:
<*@somedomain> = exec*@pipe
this Router record will direct messages sent to the `joe@somedomain` address to the `execjoe` application.

<*@somedomain> = "execall\ -u\ *"@pipe
this Router record will direct messages sent to the `joe@somedomain` address to the `execall` application started with the `-u joe` parameters.

To limit the set of applications that can be started via the PIPE module, the external application directory is specified as one of the PIPE module settings. The application names specified in message addresses can not include the slash (`/`) or the backslash (`\\`) symbols, and they cannot start with the dot (`.`) symbol, and it specified the name of the application (program) file in the external application directory.

The message text (including the message headers and the message body) is passed to the external application as its standard input.
Note: the application must read the entire stdin data stream, otherwise message processing fails.

When the external application completes, the PIPE module reads and discards the application standard output. Make sure that your application does not write anything to its standard output, so it is not blocked when the communication channel (pipe) buffer between the application and the Server is full.

When the external application completes, the PIPE module reads its standard error channel. If it is not empty, the message delivery fails, and the text written to standard error is sent as an error report to the message sender.

### Serialized Delivery

In order to allow several PIPE processors to deliver messages simultaneously, the PIPE module creates a separate queue for each message it has to deliver. If you want to serialize processing, you can use the following form of the PIPE address:
"queue[name] application parameters"@pipe
All messages directed to these addresses will be placed into the name queue, and a single PIPE processor will send the enqueued messages to the application(s) specified in those addresses. You can use any alphanumeric string as a queue name, and you can specify as many queues as you need.

The following Router records can be used to maintain two serialized PIPE queues (the PROC1 and ARCH queues):
<incoming> = "queue[PROC1] procin -mark"@pipe
<control> = "queue[PROC1] procin1 -control"@pipe
<archiver> = "queue[ARCH] appendfile /var/archive"@pipe
All messages sent to the <incoming@maindomain.com> and <control@maindomain.com> will be processed one-by-one using one PIPE processor.

For PIPE addresses that do not have the `queue[name]` prefix, the PIPE module creates separate queues with numeric names.

### Command Tags

The message text (the header and the body) is sent to the task as that task standard input (stdin).

An application name can be prefixed with the `[FILE]` tag:
[FILE] application parameters
When this prefix is used, the application standard input will be empty (closed), or it will contain only the message header fields added by Rules. The string -f Queue/fileid.msg (the -f flag and the Message file name, relative to the base directory) will be appended to the end of the application parameters:
-f Queue/12002345.msg

Note: The beginning of a Queue file contains the service information (envelope, options, etc.). The application should ignore this information skipping the file data till the first empty line. The message itself starts after that first empty line in the Queue file.

Note: Header fields added to the message by Server-wide and Cluster-wide Rules are not stored in the message Queue file, they are sent via the task standard input.

Note: this prefix should not be used on MS Windows and IBM OS/2 platforms, as the Server keeps the message file open, making it impossible for an external Task to read the file.

An application name can be prefixed with the `[RETPATH]` tag:
[RETPATH] application parameters
When this prefix is specified, the string "-p" followed by the message return-path address is added to the end of the application parameters:
-p address@domain.com

An application name can be prefixed with the `[RCPT]` tag:
[RCPT]application parameters
When this prefix is specified the string "-r" followed by the original recipient address is added to the end of the application parameters:
-r address1@domain1.com

An application name can be prefixed with the `[STDERR]` tag (see below).

An application name can have several prefix strings, and they can be specified in any order. If several of `[FILE]`, `[RETPATH]`, and `[RCPT]` prefix strings are specified, the `-f` flag and its parameter are added first, followed with the `-p` flag and its parameter, followed with the `-r` flag and its parameter.

If the `[STDERR]` prefix is specified and the external application completes sending some data to its standard error channel, the standard error data is used to compose the error report text.

## Configuring the PIPE module

Use the WebAdmin Interface to configure the PIPE module. Open the Queue pages in the Settings realm, then open the PIPE page.

Processing
Log Level: | Processors:
Application Directory: |
Processing Time Limit: |

`Log`
  Use the Log setting to specify what kind of information the PIPE module should put in the Server Log. Usually you should use the `Major` (message transfer reports) or `Problems` (message transfer and non-fatal errors) levels. But when you experience problems with the PIPE module, you may want to set the Log Level setting to `Low-Level` or `All Info`: in this case protocol-level or link-level details will be recorded in the System Log as well. When the problem is solved, set the Log Level setting to its regular value, otherwise your System Log files will grow in size very quickly.

  The PIPE module records in the System Log are marked with the `PIPE` tag.
`Processors`
  This option specifies the number of threads used to deliver messages. If some of your external applications are slow, you may want to use several PIPE delivery threads, so several messages can be processed at the same time.
`Application Directory`
  This option specifies the directory with the applications the PIPE module can launch. If an empty string is specified for this option, all messages directed to the PIPE module are rejected.
`Processing Time Limit`
  This option limits the time an external application uses to process a message. If an external application does not complete within the specified period of time, the application process is interrupted and the message is rejected.

Submitting
Check Directory every: |

`Check `Submitted` Directory`
  This option specifies how often the PIPE module should scan the Submitted directory and deliver the .sub files stored there.

## Foreign Queue Processing

In emergency situations you may need to process an additional Queue directory without stopping your server. These situations include a server hardware failure when the rescued Queue files should be processed with some other, already running server.

To process an additional `Queue` directory, move it to the base directory of a running server as the `ForeignQueue` directory. If you prefer to use symbolic links, make sure that the `Queue` and `ForeignQueue` directories are created on the same file system.

Every 3 minutes the PIPE module checks if the `ForeignQueue` directory exists in the server base directory. If the `ForeignQueue` directory is found, the module moves all `.msg` files from the `ForeignQueue` to the `Queue` directory (it can rename the files in the process), and deletes all `.tmp` files found in the `ForeignQueue` directory. Files with other extensions are left in the `ForeignQueue` directory.

All moved `.msg` files are submitted to the Server kernel, into the ENQUEUER queue, and the Server starts to process them in the same way it processes all submitted messages.

When the scanning process is completed, the `ForeignQueue` directory can be removed from the base directory.

CommuniGate® Pro Guide. Copyright © 1998-2012, Stalker Software, Inc.