Server-side support for event-driven notebook execution

[andrii-i]

Adds server-side support for event-driven notebook execution, fixes #406 (for UI see #414).

System now defines EventTypes and Events. EventType is specific condition identified by a name and a set of parameters. Event is a unique occurrence of the certain event type.

Event types can be defined through RuntimeEnvironment.event_types[]. Event occurrences are supposed to originate from outside of JupyterLab, from the broader environment where it operates. When an event of a certain type occurs, all job definitions associated with the event type (JobDefinition.events[] has an item with matching name) are supposed to be queried. This PR does not provide a mechanism to kick-off the event occurrence and does not manage querying of matching job definitions, it just provides models and handlers to support the process.

Changes

Non-breaking REST API changes:

POST /job-definition

Existing endpoint to create or update a job definition. Add new optional parameter events that contains array of event types that trigger job definition execution

Request schema:

POST /job-definition
{
   ...
   // Optional; array of event types that trigger job definition execution
   "events": Array<{ "name": string, "parameters": object }>
}

POST /job

Existing endpoint to create or update a job. Add optional parameter triggered_by that contains an event that triggered the job execution

Request schema:

POST /job
{  
   ...
   // Optional; event that triggered this job exectution
   "create_event": { 
      "event_id": string,
      "event_type": string,
      "parameters": object
   }
}

Changes to existing models

RuntimeEnvironment:

class RuntimeEnvironment(BaseModel):
    ...
    event_types: Optional[List[EventType]]

CreateJob, DescribeJob:

class CreateJob(BaseModel):
    ...
    create_event: Optional[Event]

CreateJobDefinition, DescribeJobDefinition, UpdateJobDefinition:

    ...
    events: List[EventType] = []

New models

class EventType(BaseModel):
    name: str
    parameters: Dict[str, Any]

class Event(BaseModel):
    event_id: str
    event_type: str
    parameters: Dict[str, Any]

Changes to existing models

ORM changes

class Job(CommonColumns, Base):
    ...
    create_event = Column(JsonType(1024), nullable=True)

class JobDefinition(CommonColumns, Base):
    ...
    events = Column(JsonType(1024), default=list)

[3coins]

@andrii-i
Thanks for working on this. Left some comments about naming of events attribute.

[andrii-i]

Thank you for the review @3coins