Kettle

Kettle is an integration technology which promotes the expression of servers handling HTTP and WebSockets endpoints. With a few exceptions, Kettle implements no primary functionality of its own, but aggregates the facilities of express and ws, as well as middleware held in the wider pillarjs "Bring your own HTTP Framework Framework" ecosystem. Kettle applications can easily incorporate any express-standard middleware, as well as coexisting with standard express apps targeted at the same node.js http.Server. Since Kettle applications are expressed declaratively, in the JSON format encoding Infusion's component trees, it is possible to adapt existing applications easily, as well as inserting middleware and new handlers anywhere in the pipeline without modifying the original application's code. This makes Kettle suitable for uses where application functionality needs to be deployed flexibly in a variety of different configurations.

In fact, Kettle's dependency on express itself is minimal, since the entirety of the Kettle request handling pipeline is packaged as a single piece of express-compatible middleware – Kettle could be deployed against any other consumer of middleware or even a raw node.js HTTP server.

Notes on Kettle 2.x releases

The 2.x line of Kettle releases are (at the time of writing) still code-compatible with the 1.x releases in terms of support for user code, but the 2.x Kettle releases break compatibility with older versions of Infusion (1.x through 3.x). Kettle 1.x has been compatible with mainline releases of Infusion as well as those from in-progress FLUID-6145 and FLUID-6148 branches. Kettle 2.x releases are only compatible with 4.x releases of Infusion (and historically, those from the FLUID-6145 branch, newer than and including 3.0.0-dev.20200930T151056Z.d0b9e348d.FLUID-6145 ).

Contents of this repository

Core Kettle implementation

This is packaged as Infusion grades derived from kettle.server, kettle.request and kettle.app. The first two of these exist in variants specialized both for plain HTTP (with the .http suffix) and for WebSockets (with the .ws suffix) – kettle.app does not specialize.

Contents - Testing

As well as the integration technology implementing Kettle itself, this repository also contains functionality helpful for testing HTTP and WebSockets servers written in arbitrary technologies. This is accessed by running kettle.loadTestingSupport() after having called require("kettle"). Kettle testing support allows HTTP and WebSockets client requests to be packaged as Infusion components, suitable for use with Infusion's IoC Testing Framework. Any user of Kettle's testing support needs to have node-jqunit registered as a member of their own project's devDependencies in their own package.json.

Kettle runs on node.js version 12.x and upward (see package.json for current dependency profile).

Contents - DataSources

The Kettle repository also contains a few implementations of the simple DataSource contract for read/write access to data with a simple semantic (broadly the same as that encoded in CRUD although the current DataSource semantic does not provide explicitly for deletion). See the documentation section on DataSources for details of this contract, the available implementations and how to use them.

This repository contains DataSource implementations suitable for HTTP endpoints (with a particular variety specialised for accessing CouchDB databases with CRUDlike semantics) as well as the filesystem, with an emphasis on JSON payloads.

Getting Started and Community

Installation instructions

Firstly, install node and npm by running a standard installer from node.js. Clone this repository and then run npm install.

Issue Tracking

Issue tracking is at https://issues.fluidproject.org/browse/KETTLE.

IRC

Visit #fluid-work on Freenode – community resources are linked at Fluid's IRC Channels.

Mailing list

Contact us on the fluid-work mailing list with any problems or comments.

Uses of Kettle and related projects

The primary user of Kettle is the GPII's autopersonalisation infrastructure, held at GPII/universal. Kettle is used to provide a flexible means of deploying the GPII's "Flow Manager" and related components distributed across multiple local and remote installations.

A closely related project to Kettle is gpii-express which is used in other GPII projects such as the terms registry and unified listing. This is similar in architecture to Kettle (wrapping express primitives such as servers and requests into dynamically constructed Infusion components) but slightly different in emphasis –

• gpii-express allows independently mounted application units with nested routing, in the Express 4.x style – whereas Kettle is currently limited to flat Express 3.x-style routing
• Kettle incorporates support for WebSockets endpoints, whereas gpii-express does not
• Kettle incorporates support for DataSources (see DataSources )

The request handling architecture for gpii-express and Kettle is quite similar and the projects will probably converge over time. gpii-express currently already depends on Kettle to get access to its HTTP testing support.

Documentation

Documentation and sample code for working with Kettle is contained in the docs directory. Kettle is based on Fluid Infusion's component model. If you aren't familiar with the syntax and meaning of Infusion component trees, it is a good idea to browse the documentation, tutorials and examples at the Infusion documentation site.

It contains the following topics:

1. Defining top-level Kettle applications using "config" files.
2. Defining HTTP and Websockets servers using the grades kettle.server and kettle.server.ws.
3. Defining Kettle request handlers derived from grades kettle.request grouped into app units derived from kettle.app
4. Working with standard express middleware – incorporating any standard middleware from the express community and registering it into a Kettle application
5. Working with DataSources to abstract over asynchronous access to (primarily JSON-formatted) data stored locally or remotely
6. Defining conversational, asynchronous test fixtures against HTTP and WebSockets servers using the Kettle testing framework

Of these elements of this module, those described in topics 1, 5 and 6 (configs, DataSources and the testing framework) are portable and do not depend specifically on the Kettle server and request handling infrastructure – they can be used together with any technologies defining node.js HTTP and WebSockets servers (or in the case of configs, any node.js enabled Infusion application).