Skip to content

Visualization of the spleeter codebase #939

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 1 commit into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
115 changes: 115 additions & 0 deletions .codeboarding/Audio_Processing_Utilities.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,115 @@
```mermaid

graph LR

AudioAdapter["AudioAdapter"]

FFMPEGProcessAudioAdapter["FFMPEGProcessAudioAdapter"]

AudioConvertor["AudioConvertor"]

AudioSpectrogram["AudioSpectrogram"]

AudioAdapter -- "uses" --> FFMPEGProcessAudioAdapter

Dataset -- "uses" --> AudioAdapter

Separator -- "uses" --> AudioAdapter

Dataset -- "uses" --> AudioConvertor

Separator -- "uses" --> AudioConvertor

Dataset -- "uses" --> AudioSpectrogram

```

[![CodeBoarding](https://img.shields.io/badge/Generated%20by-CodeBoarding-9cf?style=flat-square)](https://github.com/CodeBoarding/GeneratedOnBoardings)[![Demo](https://img.shields.io/badge/Try%20our-Demo-blue?style=flat-square)](https://www.codeboarding.org/demo)[![Contact](https://img.shields.io/badge/Contact%20us%20-%[email protected]?style=flat-square)](mailto:[email protected])



## Component Details



The `Audio Processing Utilities` component in Spleeter provides a robust set of tools for handling, transforming, and augmenting audio data. It is fundamental to Spleeter's operation as it underpins all audio input, output, and manipulation required for source separation and dataset preparation.



### AudioAdapter

This abstract component defines the interface for loading and saving audio waveforms. It provides methods for converting audio descriptors into usable waveform data and manages the default audio adapter instance, which is typically `FFMPEGProcessAudioAdapter`. It also includes error handling for audio loading operations.





**Related Classes/Methods**:



- <a href="https://github.com/deezer/spleeter/blob/master/spleeter/audio/adapter.py#L29-L199" target="_blank" rel="noopener noreferrer">`spleeter.audio.adapter.AudioAdapter` (29:199)</a>





### FFMPEGProcessAudioAdapter

A concrete implementation of the `AudioAdapter` that leverages FFmpeg for efficient audio loading and saving. It handles the actual interaction with the FFmpeg process to perform audio conversions and manipulations.





**Related Classes/Methods**:



- <a href="https://github.com/deezer/spleeter/blob/master/spleeter/audio/ffmpeg.py#L34-L184" target="_blank" rel="noopener noreferrer">`spleeter.audio.ffmpeg.FFMPEGProcessAudioAdapter` (34:184)</a>





### AudioConvertor

This component specializes in converting audio properties, specifically between decibels (dB) and gain. It offers utilities for transforming spectrogram data into a decibel representation and vice-versa, which is essential for audio normalization, dynamic range compression, and other signal processing tasks.





**Related Classes/Methods**:



- <a href="https://github.com/deezer/spleeter/blob/master/spleeter/audio/convertor.py#L1-L1" target="_blank" rel="noopener noreferrer">`spleeter.audio.convertor.AudioConvertor` (1:1)</a>





### AudioSpectrogram

This component focuses on transformations related to audio spectrograms, primarily for data augmentation purposes. It includes functionalities like time stretching and pitch shifting, which are vital for generating diverse training data for machine learning models, thereby improving model robustness and generalization.





**Related Classes/Methods**:



- <a href="https://github.com/deezer/spleeter/blob/master/spleeter/audio/spectrogram.py#L1-L1" target="_blank" rel="noopener noreferrer">`spleeter.audio.spectrogram.AudioSpectrogram` (1:1)</a>









### [FAQ](https://github.com/CodeBoarding/GeneratedOnBoardings/tree/main?tab=readme-ov-file#faq)
181 changes: 181 additions & 0 deletions .codeboarding/CLI_Application.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,181 @@
```mermaid

graph LR

CLI_Application["CLI Application"]

Audio_Adapter["Audio Adapter"]

Separator["Separator"]

Estimator_TensorFlow_["Estimator (TensorFlow)"]

Dataset_Provider["Dataset Provider"]

Model_Provider["Model Provider"]

Configuration_Loader["Configuration Loader"]

CLI_Application -- "uses" --> Audio_Adapter

CLI_Application -- "uses" --> Separator

CLI_Application -- "uses" --> Estimator_TensorFlow_

CLI_Application -- "uses" --> Dataset_Provider

CLI_Application -- "uses" --> Model_Provider

CLI_Application -- "uses" --> Configuration_Loader

click CLI_Application href "https://github.com/deezer/spleeter/blob/main/.codeboarding//CLI_Application.md" "Details"

click Separator href "https://github.com/deezer/spleeter/blob/main/.codeboarding//Separator.md" "Details"

```

[![CodeBoarding](https://img.shields.io/badge/Generated%20by-CodeBoarding-9cf?style=flat-square)](https://github.com/CodeBoarding/GeneratedOnBoardings)[![Demo](https://img.shields.io/badge/Try%20our-Demo-blue?style=flat-square)](https://www.codeboarding.org/demo)[![Contact](https://img.shields.io/badge/Contact%20us%20-%[email protected]?style=flat-square)](mailto:[email protected])



## Component Details



The `CLI Application` in Spleeter serves as the primary user interface, orchestrating various core functionalities based on user commands. It acts as a dispatcher, delegating tasks to specialized components for audio processing, model training, and separation.



### CLI Application

The main entry point for Spleeter, responsible for parsing command-line arguments (`train`, `separate`, `evaluate`) and coordinating the overall workflow. It initializes and invokes other core components to fulfill user requests.





**Related Classes/Methods**:



- <a href="https://github.com/deezer/spleeter/blob/master/spleeter/__main__.py#L1-L1" target="_blank" rel="noopener noreferrer">`spleeter.__main__` (1:1)</a>





### Audio Adapter

Provides a unified interface for handling audio input and output operations. It abstracts the underlying audio processing backend, allowing Spleeter to load audio from various sources and save separated tracks in different formats.





**Related Classes/Methods**:



- <a href="https://github.com/deezer/spleeter/blob/master/spleeter/audio/adapter.py#L1-L1" target="_blank" rel="noopener noreferrer">`spleeter.audio.adapter` (1:1)</a>





### Separator

Encapsulates the logic for performing audio source separation. It initializes the necessary machine learning model (Estimator) and uses the Audio Adapter to process input audio and save the separated output tracks.





**Related Classes/Methods**:



- <a href="https://github.com/deezer/spleeter/blob/master/spleeter/separator.py#L1-L1" target="_blank" rel="noopener noreferrer">`spleeter.separator` (1:1)</a>





### Estimator (TensorFlow)

Represents the underlying machine learning model, specifically a TensorFlow Estimator, responsible for the actual source separation or model training. It processes audio features and generates the separated audio streams or updates model parameters during training.





**Related Classes/Methods**:



- <a href="https://github.com/deezer/spleeter/blob/master/spleeter/model/functions/unet.py#L198-L202" target="_blank" rel="noopener noreferrer">`spleeter.model.functions.unet` (198:202)</a>

- `tensorflow.estimator.Estimator` (1:1)





### Dataset Provider

Manages the preparation and provision of datasets for model training and validation. It handles the loading and preprocessing of audio data into a format suitable for the Estimator.





**Related Classes/Methods**:



- <a href="https://github.com/deezer/spleeter/blob/master/spleeter/dataset.py#L1-L1" target="_blank" rel="noopener noreferrer">`spleeter.dataset` (1:1)</a>





### Model Provider

Handles operations related to the management of trained models, such as writing probe files to indicate model availability or status.





**Related Classes/Methods**:



- <a href="https://github.com/deezer/spleeter/blob/master/spleeter/model/provider/github.py#L1-L1" target="_blank" rel="noopener noreferrer">`spleeter.model.provider.github` (1:1)</a>





### Configuration Loader

Responsible for loading and parsing configuration files, particularly those defining model parameters and other operational settings for Spleeter.





**Related Classes/Methods**:



- <a href="https://github.com/deezer/spleeter/blob/master/spleeter/utils/configuration.py#L1-L1" target="_blank" rel="noopener noreferrer">`spleeter.utils.configuration` (1:1)</a>









### [FAQ](https://github.com/CodeBoarding/GeneratedOnBoardings/tree/main?tab=readme-ov-file#faq)
Loading