Writing a rule#

A rule is a short declaration in a YAML file: a name, a sentence in plain words, some settings and the name of a predicate. A predicate is a function in Python that looks at one file, one folder or one pair and says what it finds, or says why it cannot decide. You read and change the rules. The predicates do the reading.

There are three levels. Changing a rule and adding one from a predicate that exists need no code, and most people stop there. Writing a predicate is Python, in a copy of the source. You need a project (Your first project) and a text editor.

Where a project’s rules go#

A project keeps its rules in its rulebook folder. Each file there is one project revision: a list of changes and new rules, numbered by project_revision. Revisions apply in number order after the rules Ingence ships, and never edit them. To change something, add a file with the next number instead of editing one a run has used. The Dashboard writes project-0001.yaml, project-0002.yaml and so on when you commit a change on its Rules page. Name yours the same way.

ingence rule show -p PROJECT

This loads every revision, prints the rules in force and changes nothing. If a file is wrong it says why, in words, and stops. When it accepts the files, ingence infer -p PROJECT applies them.

Change a rule without code#

A name family is a set of files whose names share their words once numbers and stage words are set aside. This file raises from 2 to 3 the number of shared words a family needs, and switches off the rule that reads a date from a file’s name.

project-0001.yaml#

project_revision: 1
summary: Names here share at least three words, and no name holds a date.
changes:
  - override: name_family_without_stages
    parameters:
      minimum_components: 3
  - disable: date_neighbourhood_of_a_file

Save it as PROJECT/rulebook/project-0001.yaml and run ingence rule show -p PROJECT. The rule now reads “3 or more”, its version gains p1 (the project revision that changed it), and the other rule is listed as switched off in your project’s rules.

A change is one of four words, each naming one rule: override sets parameters, disable and enable switch a rule off and on, and supersede retires it in this project. A parameter has a type and, for a number, a lowest and highest value. The loader refuses a value outside them, a parameter the rule does not have, a value of the wrong type, a rule that does not exist and a rule marked provisional, which stays as shipped. A refused file stops the project from loading until it is fixed.

Each rule’s page in the reference gives its parameters with their allowed values, and says whether it is built to run. Check that first: a rule that is not built to run concludes nothing, so changing it changes nothing. The Dashboard’s Rules page writes the same override files under the same checks. It changes settings only; to switch a rule off, use a file.

Add a rule from a predicate that exists#

The predicate same_component_values groups files whose names agree on chosen kinds of word. The shipped rule same_experimental_condition uses it for conditions. This second file, saved as PROJECT/rulebook/project-0002.yaml so it does not replace the first, uses it for stains: files with DAPI in the name form one class and files with GFP another.

project-0002.yaml#

project_revision: 2
rules:
  - id: same_stain
    predicate: same_component_values
    effect: groups
    primitive: equivalent
    respect: stain
    tier: defeasible
    defeated_by: [filesystem_noise_from_spelling, placeholder_from_entry_kind]
    statement: >-
      Two files share a stain when they agree on every component of kind
      {component_kinds} bound to a sense of kind {counted_theme_kinds}, and
      each carries {minimum_components} or more of them.
    rationale: Images of one sample in the same stain are read together.
    parameters:
      - {name: component_kinds, type: string_list, default: [word]}
      - {name: counted_theme_kinds, type: string_list, default: [reagent]}
      - {name: minimum_components, type: integer, default: 1, minimum: 1, maximum: 5}

id is a name no shipped or earlier rule uses. predicate is a registered predicate; misspell it and the loader prints every name it knows. effect is classifies, groups, proposes, flags or constraint, and must be the one the predicate serves: it decides when in a run the rule runs.
primitive and respect say what a grouping asserts: files are equivalent (or identical) in a stated respect. Ingence has eight relations and no others. concerns cannot define a group: A concerning B and B concerning C does not make A concern C, and a group needs that.
tier: defeasible means the conclusion can be withdrawn, and defeated_by names the rules whose conclusions would withdraw it. Without a defeater the rule runs and publishes nothing.
statement is the sentence rule show and the Rules page print. Every {name} in it must be a parameter and every parameter must appear in it, so the sentence always shows the settings in force.
rationale, the reason the rule exists, is required. parameters lists the settings: only ones its predicate reads, each default inside its bounds. A parameter may carry a description, which the Rules page shows.

A file named for both DAPI and GFP gets a class of its own. After ingence infer, each claim names same_stain and its version.

Write a new predicate#

You need a development install: a copy of the source and pip install -e into a virtual environment. Python then runs the code in your copy.

The example classifies an image file smaller than a set size as a thumbnail, like the small previews image programs write beside an image. Copy the shape of the shipped placeholder_from_entry_kind, in src/ingence/rulebook/predicates/classify.py: about sixty lines that read one file’s record and return a word or nothing. Read it first. Save this as src/ingence/rulebook/predicates/thumbnails.py.

"""A classifying predicate: a small image file is probably a thumbnail."""

from __future__ import annotations

from collections.abc import Mapping

from ingence.core.schema import Entry, RuleEffect
from ingence.core.tiers import ResolutionStatus
from ingence.rulebook.registry import Abstention, Predicate, Reads, declare


class ThumbnailBySize:
    """An image file smaller than a stated size is a thumbnail or a preview."""

    name = "thumbnail_by_size"
    version = "1"

    def evaluate(self, subject: object, parameters: Mapping[str, object]) -> object:
        """Return ``thumbnail`` for a small image file, and None for anything else."""
        if not isinstance(subject, Entry) or not _is_an_image(subject, parameters):
            return None
        limit = parameters["maximum_bytes"]
        if isinstance(limit, int) and subject.size_bytes is not None and subject.size_bytes < limit:
            return "thumbnail"
        return None

    def explain_abstention(self, subject: object, parameters: Mapping[str, object]) -> Abstention:
        """Say why the file was not called a thumbnail."""
        if not isinstance(subject, Entry) or not _is_an_image(subject, parameters):
            return Abstention(reason="the file is not one of the image types the rule names")
        if subject.size_bytes is None:
            return Abstention(status=ResolutionStatus.UNEXAMINED, reason="no size was recorded")
        return Abstention(reason="the image is as large as a full image or larger")


def _is_an_image(subject: Entry, parameters: Mapping[str, object]) -> bool:
    """Whether the file's extension is one the rule names."""
    extensions = parameters["image_extensions"]
    return isinstance(extensions, list) and subject.extension in extensions


@declare(
    "thumbnail_by_size",
    version="1",
    summary="Classify an image file smaller than a stated size as a thumbnail",
    effect=RuleEffect.CLASSIFIES,
    reads=Reads.ENTRY,
    parameters=("image_extensions", "maximum_bytes"),
    support_families=("filesystem_fact",),
    reach="entry",
)
def _build_thumbnail_by_size() -> Predicate:
    return ThumbnailBySize()

The class. name is what a rule’s predicate field holds. version, here and in @declare, goes up whenever the verdict could change for the same input, so earlier runs are decided again. evaluate is handed the subject, here one file’s record and not the file, and the rule’s settings. It returns a word, or None when it has nothing to say. A plain “no” is None, not an error.
The declaration. @declare states, before any instance exists, what the loader checks a rule against: the effect the predicate serves, what it reads (here one file’s record, which is what evaluate is handed), the parameters it reads, the evidence families it draws on and its reach. entry says it needs nothing beyond that record, so a large set of files can be read one folder at a time where that is switched on. The loader refuses a rule whose effect differs or that sets a parameter not listed, and stops a predicate that reads a setting it did not list.
The reason for abstaining. explain_abstention is optional. It turns “no result” into a sentence. Looked at with no evidence, never looked at (UNEXAMINED, as with a missing size) and evidence pointing two ways are three different states. Do not report one as another.

Register the module with one line in load_builtins, in src/ingence/rulebook/predicates/__init__.py, with the lines already there:

    from ingence.rulebook.predicates import thumbnails as _thumbnails  # noqa: F401

A project rule can now name it. Its settings are described as in the last section.

thumbnail_rule.yaml#

project_revision: 1
rules:
  - id: small_image_is_a_thumbnail
    predicate: thumbnail_by_size
    effect: classifies
    tier: defeasible
    defeated_by: [filesystem_noise_from_spelling, placeholder_from_entry_kind]
    statement: >-
      An image file with an extension among {image_extensions} is a thumbnail
      when it is smaller than {maximum_bytes} bytes.
    rationale: Image programs write small previews beside the images they come from.
    parameters:
      - {name: image_extensions, type: string_list, default: [tif, tiff, png, jpg]}
      - {name: maximum_bytes, type: integer, default: 50000, minimum: 1, maximum: 100000000}

Test it on invented files. In the clone’s tests folder, save the rule as thumbnail_rule.yaml and this as test_thumbnails.py. It builds a project on two files, reads their names and applies the rules, and checks the claim. The run registers the predicate itself, through the load_builtins line.

import shutil
from pathlib import Path

from ingence.core.project import Project
from ingence.infer.run import run_inference
from ingence.scan.census import run_census
from ingence.storage.catalog import open_catalog


def test_a_small_image_is_a_thumbnail(tmp_path):
    folder = tmp_path / "files"
    folder.mkdir()
    for name, size in [("HeLa_DAPI_01.tif", 400_000), ("HeLa_DAPI_01_preview.png", 2_000)]:
        (folder / name).write_bytes(b"x" * size)
    project = Project.create(tmp_path / "project", created_at="t", roots=[folder])
    rules = project.path / "rulebook"
    rules.mkdir(exist_ok=True)
    shutil.copy(Path(__file__).parent / "thumbnail_rule.yaml", rules / "project-0001.yaml")

    run_census(project, run_id="run_20260101T000001Z_111111111111", started_at="t")
    run_inference(project, run_id="run_20260101T000002Z_222222222222", started_at="t")

    with open_catalog(project.path) as catalog:
        rows = catalog.query(
            "SELECT e.name, c.object_reference FROM claims c JOIN entries e "
            "ON e.entry_id = c.subject WHERE c.rule_id = 'small_image_is_a_thumbnail'"
        ).fetchall()
    assert rows == [("HeLa_DAPI_01_preview.png", "thumbnail")]

Run it from the tests folder with python -m pytest test_thumbnails.py.

What needs a change to the package#

The ingence command and the Dashboard load only the predicates the package registers, and there is no plugin mechanism. So the steps above edit the source. A project whose rules name your predicate cannot be opened by an ordinary installation, which refuses the rule as unregistered. The package’s own tests also expect every predicate to be named by a shipped rule, so one test fails in your copy until a shipped rule names yours. A predicate others should use belongs in the project: send it to hello@ingence.org.