Skip to content

Add stored procedures and triggers#47

Merged
dumbbell merged 1 commit into
mainfrom
stored-procedures
Feb 10, 2022
Merged

Add stored procedures and triggers#47
dumbbell merged 1 commit into
mainfrom
stored-procedures

Conversation

@dumbbell

@dumbbell dumbbell commented Feb 8, 2022

Copy link
Copy Markdown
Collaborator

There are two parts in this patch:

Stored procedures

We leverage the khepri_fun framework used to extract anonymous functions (used for transactions) to extract and store one as the payload of a Khepri tree node. The function will be replicated on all Khepri nodes in the cluster and therefore can be executed anywhere, regardless of the availability of the initial module(s) the function was extracted from.

To store a function, we introduce a new payload record:

Payload = #kpayload_sproc{sproc = fun() -> ... end}.

This payload record can be passed to khepri_machine:put/3 and higher-level functions in khepri.

To execute a stored procedure, there is a new khepri_machine public API:

khepri_machine:run_sproc(StoredId, StoredProcPath, Args).

StoredProcPath is the path where the function was stored in the tree and Args is a (possibly empty) list of terms, like one would pass to erlang:apply/2.

Stored procedures may do whatever they want and have side effects, unlike transaction functions. Stored procedures don't have direct access to the tree and can't modify it directly, again unlike transaction functions. However, they are free to use the Khepri API to update the tree as any caller.

The return value of khepri_machine:run_sproc/3 is the return value of the stored procedure.

When a path is queried using e.g. khepri_machine:get/2, the node properties map has a sproc member instead of a data one.

Triggers

A trigger is a way to associate an event with a stored procedure: when the event occurs, the stored procedure is executed.

This commit supports only one type of event: changes to the tree (the database). Whenever a tree node is added, modified or deleted, the list of triggers is evaluated to see if one or more are interested in this event. If some are, their associated stored procedure is executed.

To register a new trigger, one first defines an event filter, stores a procedure and finally registers the association between this event filter and the stored procedure (i.e. the trigger):

EventFilter = #kevf_tree{path = PathPattern,
                         props = #{on_actions => [create, update]}},
khepri_machine:register_trigger(
  StoredId, TriggerId, EventFilter, StoredProcPath).

An event filter is evaluted using:

  • the PathPattern which can contains path conditions;
  • the event's action, i.e. whether a node was created, updated or
    removed.

The return value of the stored procedure is ignored when executed in the context of a trigger.

See #16.

@dumbbell dumbbell added the enhancement New feature or request label Feb 8, 2022
@dumbbell dumbbell self-assigned this Feb 8, 2022
@coveralls

coveralls commented Feb 8, 2022

Copy link
Copy Markdown

Pull Request Test Coverage Report for Build 325

Warning: This coverage report may be inaccurate.

This pull request's base commit is no longer the HEAD commit of its target branch. This means it includes changes from outside the original pull request, including, potentially, unrelated coverage changes.

Details

  • 142 of 190 (74.74%) changed or added relevant lines in 8 files are covered.
  • 34 unchanged lines in 2 files lost coverage.
  • Overall coverage decreased (-1.8%) to 83.681%

Changes Missing Coverage Covered Lines Changed/Added Lines %
src/khepri_utils.erl 15 16 93.75%
src/khepri_app.erl 1 3 33.33%
src/khepri_tx.erl 17 20 85.0%
src/khepri_event_handler.erl 14 20 70.0%
src/khepri_sproc.erl 9 15 60.0%
src/khepri.erl 4 12 33.33%
src/khepri_machine.erl 77 99 77.78%
Files with Coverage Reduction New Missed Lines %
src/khepri_utils.erl 1 97.44%
src/khepri_machine.erl 33 86.25%
Totals Coverage Status
Change from base Build 309: -1.8%
Covered Lines: 1405
Relevant Lines: 1679

💛 - Coveralls

@dumbbell
dumbbell force-pushed the stored-procedures branch 3 times, most recently from 43a07f7 to e6c3d4d Compare February 10, 2022 13:28
There are two parts in this patch:

First, **stored procedures**.

We leverage the `khepri_fun` framework used to extract anonymous
functions (used for transactions) to extract and store one as the
payload of a Khepri tree node. The function will be replicated on all
Khepri nodes in the cluster and therefore can be executed anywhere,
regardless of the availability of the initial module(s) the function was
extracted from.

To store a function, we introduce a new payload record:

    Payload = #kpayload_sproc{sproc = fun() -> ... end}.

This payload record can be passed to `khepri_machine:put/3` and
higher-level functions in `khepri`.

To execute a stored procedure, there is a new `khepri_machine` public
API:

    khepri_machine:run_sproc(StoredId, StoredProcPath, Args).

`StoredProcPath` is the path where the function was stored in the tree
and `Args` is a (possibly empty) list of terms, like one would pass to
`erlang:apply/2`.

Stored procedures may do whatever they want and have side effects,
unlike transaction functions. Stored procedures don't have direct access
to the tree and can't modify it directly, again unlike transaction
functions. However, they are free to use the Khepri API to update the
tree as any caller.

The return value of `khepri_machine:run_sproc/3` is the return value of
the stored procedure.

When a path is queried using e.g. `khepri_machine:get/2`, the node
properties map has a `sproc` member instead of a `data` one.

Second, **triggers**.

A trigger is a way to associate an event with a stored procedure: when
the event occurs, the stored procedure is executed.

This commit supports only one type of event: changes to the tree (the
database). Whenever a tree node is added, modified or deleted, the list
of triggers is evaluated to see if one or more are interested in this
event. If some are, their associated stored procedure is executed.

To register a new trigger, one first defines an event filter, stores a
procedure and finally registers the association between this event
filter and the stored procedure (i.e. the trigger):

    EventFilter = #kevf_tree{path = PathPattern,
                             props = #{on_actions => [create, update]}},
    khepri_machine:register_trigger(
      StoredId, TriggerId, EventFilter, StoredProcPath).

An event filter is evaluted using:
* the `PathPattern` which can contains path conditions;
* the event's action, i.e. whether a node was created, updated or
  removed.

The return value of the stored procedure is ignored when executed in the
context of a trigger.

See #16.
@dumbbell
dumbbell marked this pull request as ready for review February 10, 2022 15:04
@dumbbell
dumbbell merged commit 9fbf7ec into main Feb 10, 2022
@dumbbell
dumbbell deleted the stored-procedures branch February 10, 2022 15:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants