Writing a Pipe Module

4 views Markdown

A Pipe module reads a support mailbox and returns a normalised message array, which the ticket cron chain turns into tickets and replies.

Overview

Pipe is the ticket e-mail ingestion type. Three modules ship: Google and Microsoft over OAuth, Pop3 with host and password. There is no base class.

The unit of configuration is a department, not the module. Every method that touches connection state takes a department id.

In the request cycle only the settings screen and the OAuth callback touch your module. A cron chain reads: discover dispatches a fetch job per department, fetch calls inbox(), a third job makes the ticket.

Prerequisites

Write access to coremio/modules/Pipe/.
A reachable mailbox, plus the PHP imap extension or an OAuth application.
Ticket departments already created.
Adding a Scheduled Task.

Structure

Pipe classes are namespaced: WISECP\Modules\Pipe. The cron builds that name from the provider string.

layout

coremio/modules/Pipe/Acme/
├── Acme.php                    namespace WISECP\Modules\Pipe; class Acme
├── config.php                  global provider block + one block per department id
├── lang/en.php                 field labels, description, setup-guide
├── lang/tr.php
├── logo.svg
└── views/
    ├── credentialsForm.php     the per department fields, rendered into the settings tab
    └── providerForm.php        OAuth applications only: the global client id and secret

The OAuth redirect router is shared; coremio/modules/Pipe/router.php maps each slug to a module.

Walkthrough

The Module Class

Create coremio/modules/Pipe/Acme/Acme.php with namespace WISECP\Modules\Pipe; and class Acme.
Give it a public $name matching the directory.
Implement save_config(array $data) as a recursive merge.
Implement is_connected(int $did); false means skipped.
Implement inbox(int $did).
Implement test_connection(int $did).

The Department Form

Create views/credentialsForm.php. It receives $mv (your instance under init), $mk and $did.
Name fields module[{moduleKey}][{did}][{field}].
Read current values from $mv["init"]->config[$did] and labels from $mv["lang"].
A longer walkthrough goes in the setup-guide key.

Adding OAuth

Declare is_configured(). Its existence is the switch for the provider card.
Add get_global_config(), save_provider_config(), views/providerForm.php.
Build the redirect URI from the shared route key: LinkGenerator::client('pipe-oauth-callback', [strtolower($this->name)]).
Add oAuth2(int $did), returning the authorisation URL, plus callback_handle().
Register your slug in the router.
Add clear_tokens(int $did).

Reference

Method Contract

Each is probed with method_exists before it is called.

Method	Called by	If it is missing
`inbox(int $did): array`	fetch cron	the job fails, reason `inbox-missing`
`is_connected(int $did): bool`	discover cron	dispatched regardless
`save_config(array $data)`	settings save	fields are not stored
`test_connection($did): array`	settings screen	test button unavailable
`is_configured(): bool`	settings screen	no provider card
`get_global_config(): array`	settings screen	empty provider card
`save_provider_config($id, $secret): bool`	provider save	saving throws
`oAuth2($did): array`	settings screen	connect button unavailable
`callback_handle(): array`	shared OAuth route	no handler
`clear_tokens($did)`	settings screen	cannot disconnect
`get_connected_email($did): string`	credentials form	mailbox not shown

Only oAuth2, test_connection and clear_tokens are reachable from the panel.

Signatures

signatures

public function __construct();

// Recursive merge, then rewrite config.php. A plain replace erases other departments.
public function save_config($data = []);

// Readiness for ONE department. Tokens present, or hostname+username+password filled.
public function is_connected(int $did): bool;

// The one method the pipeline cannot do without. Return shape below.
public function inbox(int $did): array;

// ['status' => 'successful'] or ['status' => 'error', 'message' => '...'].
public function test_connection($did = 0): array;

// OAuth modules only.
public function is_configured(): bool;
public function get_global_config(): array;                                    // client_id, client_secret, redirect_uri
public function save_provider_config(string $client_id, string $client_secret_plain): bool;
public function oAuth2($did = 0): array;                                       // ['status' => 'successful', 'redirect' => $url]
public function callback_handle(): array;                                      // ['status' => 'connected', 'did' => 4, 'email' => '...']
public function clear_tokens($did = 0);
public function get_connected_email(int $did): string;

The message array is the real contract. Every key is read by name, and a missing one degrades silently.

inbox() return

return [
    'status' => 'successful',            // anything else is treated as a soft error
    'data'   => [
        [
            'ip'          => '203.0.113.9',              // sender IP if the headers carry one, else ''
            'date'        => '2026-08-03 09:41:00',      // Y-m-d H:i:s, used in the duplicate hash
            'subject'     => 'Cannot reach my panel',    // becomes the ticket subject
            'spam'        => false,                      // true makes the handler drop the message
            'msgid'       => '',                         // optional; a stable hash is synthesised when empty
            'from'        => ['name' => 'Ada L.', 'address' => '[email protected]'],
            'to'          => ['name' => 'Support', 'address' => '[email protected]'],
            'message'     => '<p>The panel times out.</p>',
            'attachments' => [
                [
                    'file_name' => 'screenshot.png',     // the name the sender used
                    'name'      => 'a1b2c3d4e5f6.png',   // the randomised stored name
                    'file_ext'  => 'png',
                    'size'      => 20481,
                    'content'   => 'iVBORw0KGgoAAAANS',  // base64 of the raw bytes
                ],
            ],
        ],
    ],
];

// On failure, either throw or return the soft error form:
return ['status' => 'error', 'message' => 'Cannot connect to server', 'data' => []];

A throw sets the breaker, notifies and records the job failed; a returned error marks it cancelled. Return for configuration, throw for transport.

Config

config.php mixes string keys for module wide settings with integer keys for department state. That is why save_config() must merge.

config.php

<?php
return [
    'lookback_days' => 3,                        // module wide
    'provider'      => [                         // module wide, OAuth applications only
        'client_id'     => '...',
        'client_secret' => '...',                // Crypt::encode with crypt/system
    ],
    4 => [                                       // department 4
        'tokens' => '...',                       // Crypt::encode of the token JSON
        'email'  => '[email protected]',
    ],
    7 => [                                       // department 7, credential based
        'protocol' => 'imap',
        'hostname' => 'mail.example.com',
        'port'     => 993,
        'username' => '[email protected]',
        'password' => '...',
        'ssl'      => true,
    ],
];

The mailbox to department mapping lives in the shared options file:

options/ticket-pipe/status The master switch; off, discover cancels with pipe-disabled.

options/ticket-pipe/mail Department id to a row with provider, from, fname. An incomplete row is dropped.

options/ticket-pipe/prefix The reference tag in outgoing subjects, matched on the way back. Defaults to REF.

options/ticket-pipe/existing-client Reject or create when the sender matches no account.

options/ticket-pipe/spam-control When on, spam checks run first.

Config::save() Nested: write the whole subtree with Config::save("options", Config::set("options", ['ticket-pipe' => $ticketPipe])).

The Cron Chain

Three queue handlers:

Handler	What it does	Touches your module
discover	Walks the departments, dispatches one fetch job each	`is_connected()` and a class check
fetch	Collects messages, dispatches one message job each	`inbox()`
message	Turns a message into a ticket or reply	nothing; it reads your array

Four conditions make discover skip a department without an error: no class, is_connected() false, a fetch in flight, or cooldown. Cooldown lasts three hours; a clean fetch clears it.

Example

A credential based module, then the cron that calls it.

Acme.php

<?php
namespace WISECP\Modules\Pipe;

class Acme
{
    public $name = "Acme";
    public $config = [];
    public $lang = [];
    public $test = false;

    public function __construct()
    {
        $this->config = \Modules::Config("Pipe", $this->name);
        $this->lang   = \Modules::Lang("Pipe", $this->name);
    }

    // Recursive merge: the settings screen posts one department at a time.
    public function save_config($data = [])
    {
        $merged       = array_replace_recursive($this->config ?: [], $data);
        $this->config = $merged;

        return \FileManager::file_write(__DIR__ . DS . "config.php", \Utility::array_export($merged, ['pwith' => true]));
    }

    public function is_connected(int $did): bool
    {
        $cfg = $this->config[$did] ?? null;
        if (!is_array($cfg)) return false;

        return !empty($cfg['hostname']) && !empty($cfg['username']) && !empty($cfg['password']);
    }

    public function test_connection($did = 0): array
    {
        try {
            // The form posts module[Acme][{did}][field], so that is the path to read.
            $host = trim((string) \Filter::init("POST/module/" . $this->name . "/" . $did . "/hostname"));
            $user = trim((string) \Filter::init("POST/module/" . $this->name . "/" . $did . "/username"));
            $pass = (string) \Filter::init("POST/module/" . $this->name . "/" . $did . "/password", "password");

            if ($host === '' || $user === '' || $pass === '')
                throw new \Exception($this->lang["credentials-required"] ?? 'Hostname, username and password are required.');

            $this->open($host, $user, $pass);
        }
        catch (\Exception $e) {
            return ['status' => "error", 'message' => $e->getMessage()];
        }

        return ['status' => "successful"];
    }

    public function inbox(int $did): array
    {
        $cfg = $this->config[$did] ?? null;

        // A configuration problem is a soft error: the job is cancelled, not failed.
        if (!$cfg) return ['status' => 'error', 'message' => 'Department config not found', 'data' => []];

        // A transport problem throws: the breaker opens and the job is recorded as failed.
        $session = $this->open($cfg['hostname'] ?? '', $cfg['username'] ?? '', $cfg['password'] ?? '');

        $lookback = max(1, (int) ($this->config['lookback_days'] ?? 3));
        $since    = date('Y-m-d', strtotime('-' . $lookback . ' days'));

        $messages = [];
        foreach ($this->unread($session, $since) as $raw) {
            $messages[] = [
                'ip'          => (string) ($raw['sender_ip'] ?? ''),
                'date'        => \DateManager::format("Y-m-d H:i:s", $raw['date'] ?? ''),
                'subject'     => (string) ($raw['subject'] ?? ''),
                'spam'        => false,
                'from'        => ['name' => (string) ($raw['from_name'] ?? ''), 'address' => (string) ($raw['from'] ?? '')],
                'to'          => ['name' => (string) ($raw['to_name'] ?? ''),   'address' => (string) ($raw['to'] ?? '')],
                'message'     => (string) ($raw['html'] ?? ($raw['text'] ?? '')),
                'attachments' => $this->attachments($raw['parts'] ?? []),
            ];
        }

        return ['status' => "successful", 'data' => $messages];
    }
}

coremio/cronjobs/TicketPipeFetch.php

// getInstance, not new: it runs Modules::Load first and fills the module config cache.
// With a bare new, the constructor reads null from Modules::Config and inbox() bails
// out with "Department config not found".
$module = \Modules::getInstance("Pipe", $provider);

if (!$module) return ['success' => false, 'result' => ['reason' => 'module-missing']];
if (!method_exists($module, 'inbox')) return ['success' => false, 'result' => ['reason' => 'inbox-missing']];

try {
    $response = $module->inbox($did);
}
catch (\Throwable $e) {
    self::set_cooldown($did, $e->getMessage());
    Admin::notify('ticket-pipe-failure', self::failure_payload($did, $provider, $depName, $depFrom, $e->getMessage()), 'error', [
        'dedupe_keys' => ['did', 'provider'],
        'recurring'   => true,
    ]);
    throw $e;
}

$status = (string) ($response['status'] ?? '');
$data   = (array)  ($response['data']   ?? []);

if ($status !== 'successful') {
    self::set_cooldown($did, (string) ($response['message'] ?? 'Unknown module error'));
    // ... notify, then return a cancelled job
}

// Clean run: drop the pending failure notice and release the breaker.
Admin::notify_resolve('ticket-pipe-failure', ['did' => $did, 'provider' => $provider]);
self::clear_cooldown($did);

Modules do not return a message id, so the fetch handler hashes department, sender, date and subject. A real msgid makes deduplication exact.

Pitfalls

A flat key never reaches the cron

Everything lives under options/ticket-pipe. A similarly named top level key saves and reads back fine while the cron keeps the old value.

save_config must merge, not replace

The settings post carries only the edited department. A save_config() that assigns instead of merging wipes the other departments' credentials, and the only symptom is that those mailboxes stop being polled.

Build the instance with the factory

Modules::getInstance() runs the loader first, filling the config cache your constructor reads. A bare new leaves the config null and fails later inside inbox().

A failure sleeps the department for three hours

The circuit breaker keeps a broken mailbox from notifying every minute. While testing, your fix appears dead: the department is skipped with cooldown. Re run the fetch job directly.

Tokens go through the system key

OAuth tokens and client secrets use Crypt::encode($value, Config::get("crypt/system")): the system key, not the user one. Store the token JSON encoded.

Was this helpful?

Thanks for your feedback!

Still Need Help?

Our support team is here around the clock for anything you can't find above.

Overview

Prerequisites

Structure

Walkthrough

The Module Class

The Department Form

Adding OAuth

Reference

Method Contract

Signatures

Config

The Cron Chain

Example

Pitfalls

Related Articles