Skip to content

Commit 2109352

Browse files
feat: add FastStream worker integration to temporal-boost (#35)
* feat: add FastStream worker integration to temporal-boost - Add FastStreamBoostWorker class with anyio.run support - Implement add_faststream_worker method in BoostApp - Add FastStream worker to workers module exports - Update logging configuration for FastStream loggers - Add example usage with Redis and Kafka brokers This enables running FastStream message broker applications alongside Temporal workers in the same BoostApp instance. * fix(no-ref): update docs * Bump version --------- Co-authored-by: Oleg Romanov <[email protected]>
1 parent 40c5b20 commit 2109352

File tree

15 files changed

+329
-45
lines changed

15 files changed

+329
-45
lines changed

.vscode/launch.json

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -4,6 +4,7 @@
44
// Для получения дополнительной информации посетите: https://go.microsoft.com/fwlink/?linkid=830387
55
"version": "0.2.0",
66
"configurations": [
7+
78
{
89
"name": "Python Debugger: Current File with Arguments",
910
"type": "debugpy",
@@ -21,6 +22,15 @@
2122
"args": ["run", "worker_3"],
2223
"justMyCode": false
2324
},
25+
{
26+
"name": "Python Debugger: Example Simple Faststream",
27+
"type": "debugpy",
28+
"request": "launch",
29+
"program": "examples/example_simple_faststream.py",
30+
"console": "integratedTerminal",
31+
"args": ["run", "message_processor"],
32+
"justMyCode": false
33+
},
2434
{
2535
"name": "Python Debugger: Example App ASGI",
2636
"type": "debugpy",

README.md

Lines changed: 74 additions & 20 deletions
Original file line numberDiff line numberDiff line change
@@ -6,23 +6,41 @@
66
[![PyPI](https://img.shields.io/pypi/v/temporal-boost.svg?style=for-the-badge)](https://pypi.org/project/temporal-boost)
77
[![MIT](https://img.shields.io/pypi/l/temporalio.svg?style=for-the-badge)](LICENSE)
88

9-
Documentation is available [on GH pages](https://northpowered.github.io/temporal-boost/)
9+
Documentation is available on [GitHub Pages](https://northpowered.github.io/temporal-boost/)
1010

1111
Small framework based on [temporalio/sdk-python](https://github.com/temporalio/sdk-python) - create [Temporal](https://temporal.io/) microservices as fast as you can
1212

13-
# Requirements
13+
## Requirements
1414

15-
* Python >= 3.10
15+
- Python >= 3.10
1616

17-
# Features
17+
## Features
1818

19-
* Create Temporal workers with FastAPI-style
20-
* Add CRON workers with one code line
21-
* Append ASGI (ex. FastAPI) workers like Temporal
22-
* Auto documentation with web UI (like SwaggerUI in FastAPI)
23-
* Build-in logger and OTLP tracer
19+
- Create Temporal workers with FastAPI-style
20+
- Add CRON workers with one code line
21+
- Append ASGI (ex. FastAPI) workers like Temporal
22+
- Auto documentation with web UI (like SwaggerUI in FastAPI)
23+
- Build-in logger and OTLP tracer
24+
25+
## Installation
26+
27+
Install core:
28+
29+
```bash
30+
pip install temporal-boost
31+
# or
32+
poetry add temporal-boost
33+
```
34+
35+
Optional extras:
36+
37+
- faststream integration: `pip install "temporal-boost[faststream]"` or `poetry add temporal-boost -E faststream`
38+
- uvicorn ASGI: `pip install "temporal-boost[uvicorn]"` or `poetry add temporal-boost -E uvicorn`
39+
- hypercorn ASGI: `pip install "temporal-boost[hypercorn]"` or `poetry add temporal-boost -E hypercorn`
40+
- granian ASGI: `pip install "temporal-boost[granian]"` or `poetry add temporal-boost -E granian`
41+
42+
## Quick start
2443

25-
# Quick start
2644
```python
2745
from temporal_boost import BoostApp
2846
from temporalio import activity
@@ -83,7 +101,7 @@ app.run()
83101
```bash
84102
python3 main.py
85103

86-
# Usage: main.py [OPTIONS] COMMAND [ARGS]...
104+
Usage: main.py [OPTIONS] COMMAND [ARGS]...
87105

88106
# Options:
89107
# --install-completion [bash|zsh|fish|powershell|pwsh]
@@ -93,25 +111,25 @@ python3 main.py
93111
# copy it or customize the installation.
94112
# --help Show this message and exit.
95113

96-
# Commands:
97-
# cron
98-
# run
114+
Commands:
115+
cron
116+
run
99117

100118
```
101119

102120
```bash
103121
python3 main.py run
104122

105-
# Usage: main.py run [OPTIONS] COMMAND [ARGS]...
123+
Usage: main.py run [OPTIONS] COMMAND [ARGS]...
106124

107125
# Options:
108126
# --help Show this message and exit.
109127

110-
# Commands:
111-
# all
112-
# test_cron
113-
# worker_1
114-
# worker_2
128+
Commands:
129+
all
130+
test_cron
131+
worker_1
132+
worker_2
115133
```
116134

117135
```bash
@@ -123,3 +141,39 @@ python3 main.py run worker_1
123141
# 2023-09-20T21:25:12 | INFO | Worker worker_1 started on task_q_1 queue
124142

125143
```
144+
145+
## Environment variables
146+
147+
Core configuration is managed via environment variables (see `temporal_boost/temporal/config.py`):
148+
149+
- `TEMPORAL_TARGET_HOST` (default: `localhost:7233`)
150+
- `TEMPORAL_NAMESPACE` (default: `default`)
151+
- `TEMPORAL_TLS` (default: `false`)
152+
- `TEMPORAL_API_KEY` (optional)
153+
- `TEMPORAL_IDENTITY` (optional)
154+
- `TEMPORAL_USE_PYDANTIC_DATA_CONVERTER` (default: `false`)
155+
156+
Worker tuning:
157+
158+
- `TEMPORAL_MAX_CONCURRENT_WORKFLOW_TASKS` (default: `300`)
159+
- `TEMPORAL_MAX_CONCURRENT_ACTIVITIES` (default: `300`)
160+
- `TEMPORAL_MAX_CONCURRENT_LOCAL_ACTIVITIES` (default: `100`)
161+
- `TEMPORAL_MAX_WORKFLOW_TASK_POLLS` (default: `10`)
162+
- `TEMPORAL_MAX_ACTIVITY_TASK_POLLS` (default: `10`)
163+
- `TEMPORAL_NONSTICKY_TO_STICKY_RATIO` (default: `0.2`)
164+
- `TEMPORAL_GRACEFUL_SHUTDOWN_TIMEOUT` (seconds, default: `30`)
165+
166+
Telemetry (Prometheus runtime):
167+
168+
- `TEMPORAL_PROMETHEUS_BIND_ADDRESS` (e.g. `0.0.0.0:8801`)
169+
- `TEMPORAL_PROMETHEUS_COUNTERS_TOTAL_SUFFIX` (default: `false`)
170+
- `TEMPORAL_PROMETHEUS_UNIT_SUFFIX` (default: `false`)
171+
- `TEMPORAL_PROMETHEUS_DURATIONS_AS_SECONDS` (default: `false`)
172+
173+
Example:
174+
175+
```bash
176+
export TEMPORAL_TARGET_HOST=temporal.example.com:7233
177+
export TEMPORAL_NAMESPACE=production
178+
export TEMPORAL_USE_PYDANTIC_DATA_CONVERTER=true
179+
```

docs/index.md

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -11,7 +11,7 @@ Temporal-boost is a lightweight framework for fast and comfortable development o
1111
- [x] [Temporal SDK (python)](https://github.com/temporalio/sdk-python)
1212
- [x] [Pydantic - for serialization](https://github.com/pydantic/pydantic)
1313
- [x] [Typer - for CLI interface](https://github.com/fastapi/typer)
14-
- [x] [Loguru - for extended logging](https://github.com/Delgan/loguru)
14+
- [x] [Python logging - built-in logging configuration]
1515
- [x] [Hypercorn, Uvicorn, Granian - for running ASGI applications](https://github.com/pgjones/hypercorn)
1616

1717
### Main features
@@ -92,7 +92,7 @@ if __name__ == "__main__":
9292

9393
### Configuration
9494

95-
All configuration (Temporal endpoint, namespace, TLS, metrics, etc.) is now handled via environment variables. See the documentation for `temporal_boost/temporal/config.py` for all available options.
95+
All configuration (Temporal endpoint, namespace, TLS, metrics, etc.) is handled via environment variables. See `temporal_boost/temporal/config.py` for available options.
9696

9797
### Start example application
9898

File renamed without changes.

examples/example_app.py

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -47,7 +47,7 @@ async def test_boost_activity_2(payload: TestModel) -> TestModel: # noqa: RUF02
4747

4848
@activity.defn(name="custom_test_boost_activity_3")
4949
async def test_boost_activity_3(payload: TestModel, foo: str, bar: int) -> TestModel: # noqa: RUF029
50-
payload.foo = f"{payload.foo}+activity2"
50+
payload.foo = f"{payload.foo}+activity3"
5151
payload.bar += 1
5252
return payload
5353

@@ -56,7 +56,7 @@ async def test_boost_activity_3(payload: TestModel, foo: str, bar: int) -> TestM
5656
class MyWorkflow:
5757
@workflow.run
5858
async def run(self, foo: str) -> TestModel: # noqa: ARG002
59-
logger.info("Sync logger")
59+
logger.info("Sync logger: starting workflow")
6060

6161
start_payload: TestModel = TestModel(foo="hello", bar=0)
6262
result_1 = await workflow.execute_activity(
Lines changed: 41 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,41 @@
1+
import logging
2+
3+
from faststream import FastStream
4+
from faststream.redis import RedisBroker
5+
from pydantic import BaseModel
6+
7+
from temporal_boost import BoostApp
8+
9+
10+
logging.basicConfig(level=logging.INFO)
11+
logger = logging.getLogger(__name__)
12+
13+
faststream_logger = logger.getChild("faststream")
14+
15+
16+
class TaskMessage(BaseModel):
17+
task_id: str
18+
description: str
19+
priority: int
20+
21+
22+
broker = RedisBroker("redis://localhost:6379")
23+
app = FastStream(broker)
24+
25+
26+
@broker.subscriber("tasks")
27+
async def process_task(message: TaskMessage) -> None: # noqa: RUF029
28+
logger.info(f"Processing task: {message.task_id} - {message.description}")
29+
30+
if message.priority > 5: # noqa: PLR2004
31+
logger.info(f"High priority task {message.task_id} processed immediately")
32+
else:
33+
logger.info(f"Normal priority task {message.task_id} queued for processing")
34+
35+
36+
boost_app = BoostApp("simple-faststream-example")
37+
boost_app.add_faststream_worker("message_processor", app)
38+
39+
40+
if __name__ == "__main__":
41+
boost_app.run()

mkdocs.yml

Lines changed: 4 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,5 @@
11
site_name: Temporal-boost
2-
site_description: Extemely fast development for Temporal-based microservices
2+
site_description: Extremely fast development for Temporal-based microservices
33
repo_url: https://github.com/northpowered/temporal-boost
44
repo_name: northpowered/temporal-boost
55

@@ -11,6 +11,8 @@ theme:
1111
features:
1212
- navigation.tracking
1313
- content.code.copy
14+
- navigation.instant
15+
- content.tabs.link
1416
icon:
1517
annotation: material/arrow-right-circle
1618

@@ -38,4 +40,4 @@ markdown_extensions:
3840
nav:
3941
- Getting started: index.md
4042
- Creating application: creating_application.md
41-
- Running application: runnig_application.md
43+
- Running application: running_application.md

0 commit comments

Comments
 (0)