Skip to content

docs: Add Sound Effects documentation. #753

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 7 commits into from
Sep 6, 2022
283 changes: 247 additions & 36 deletions docs/audio.rst
Original file line number Diff line number Diff line change
Expand Up @@ -6,29 +6,57 @@ Audio
This module allows you play sounds with the micro:bit.

By default sound output will be via the edge connector on pin 0 and the
:doc:`built-in speaker <speaker>` **V2**. You can connect wired headphones or
:doc:`built-in speaker <speaker>` (**V2**). You can connect wired headphones or
a speaker to pin 0 and GND on the edge connector to hear the sounds.

The ``audio`` module can be imported as ``import audio`` or accessed via
the ``microbit`` module as ``microbit.audio``.

There are three different kinds of audio sources that can be played using the
:py:meth:`audio.play` function:

1. `Built in sounds <#built-in-sounds-v2>`_ (**V2**),
e.g. ``audio.play(Sound.HAPPY)``
2. `Sound Effects <#sound-effects-v2>`_ (**V2**), a way to create custom sounds
by configuring its parameters::

my_effect = audio.SoundEffect(freq_start=400, freq_end=2500, duration=500)
audio.play(my_effect)

3. `Audio Frames <##audioframe>`_, an iterable (like a list or a generator)
of Audio Frames, which are lists of 32 samples with values from 0 to 255::

square_wave = audio.AudioFrame()
for i in range(16):
square_wave[i] = 0
square_wave[i + 16] = 255
audio.play([square_wave] * 64)


Functions
=========

.. py:function:: play(source, wait=True, pin=pin0, return_pin=None)

Play the source to completion.
Play the audio source to completion.

:param source: ``Sound``: The ``microbit`` module contains a list of
built-in sounds that your can pass to ``audio.play()``.
:param source: There are three types of data that can be used as a source:

- ``Sound``: The ``microbit`` module contains a list of
built-in sounds, e.g. ``audio.play(Sound.TWINKLE)``. A full list can
be found in the `Built in sounds <#built-in-sounds-v2>`_ section.
- ``SoundEffect``: A sound effect, or an iterable of sound effects,
created via the :py:meth:`audio.SoundEffect` class
- ``AudioFrame``: An iterable of ``AudioFrame`` instances as described
in the `AudioFrame Technical Details <#id2>`_ section

``AudioFrame``: The source agrument can also be an iterable
of ``AudioFrame`` elements as described below.
:param wait: If ``wait`` is ``True``, this function will block until the
source is exhausted.

:param pin: An optional argument to specify the output pin can be used to
override the default of ``pin0``. If we do not want any sound to play
we can use ``pin=None``.

:param return_pin: specifies a differential edge connector pin to connect
to an external speaker instead of ground. This is ignored for the **V2**
revision.
Expand All @@ -41,34 +69,9 @@ Functions

Stops all audio playback.

Classes
=======

.. py:class::
AudioFrame

An ``AudioFrame`` object is a list of 32 samples each of which is an unsigned byte
(whole number between 0 and 255).

It takes just over 4 ms to play a single frame.

.. py:function:: copyfrom(other)

Overwrite the data in this ``AudioFrame`` with the data from another
``AudioFrame`` instance.

:param other: ``AudioFrame`` instance from which to copy the data.


Using audio
===========

You will need a sound source, as input to the ``play`` function. You can use
the built-in sounds **V2** from the ``microbit`` module, ``microbit.Sound``, or
generate your own, like in ``examples/waveforms.py``.

Built-in sounds **V2**
----------------------
======================

The built-in sounds can be called using ``audio.play(Sound.NAME)``.

Expand All @@ -83,8 +86,216 @@ The built-in sounds can be called using ``audio.play(Sound.NAME)``.
* ``Sound.TWINKLE``
* ``Sound.YAWN``

Sounds Example
--------------

::

from microbit import *

while True:
if button_a.is_pressed() and button_b.is_pressed():
# When pressing both buttons only play via the edge connector
audio.play(Sound.HELLO, pin=pin0)
elif button_a.is_pressed():
# On button A play a sound and when it's done show an image
audio.play(Sound.HAPPY)
display.show(Image.HAPPY)
elif button_b.is_pressed():
# On button B play a sound and show an image at the same time
audio.play(Sound.TWINKLE, wait=False)
display.show(Image.BUTTERFLY)

sleep(500)
display.clear()


Sound Effects **V2**
====================

.. py:class::
SoundEffect(preset=None, freq_start=500, freq_end=2500, duration=500, vol_start=255, vol_end=0, wave=WAVE_SQUARE, fx=None, interpolation=INTER_LOG)

An ``SoundEffect`` instance represents a sound effect, composed by a set of
parameters configured via the constructor or attributes.

All the parameters are optional, with default values as shown above, and
they can all be modified via attributes of the same name. For example, we
can first create an effect ``my_effect = SoundEffect(duration=1000)``,
and then change its attributes ``my_effect.duration = 500``.

:param preset: An existing SoundEffect instance to use as a base, its values
are cloned in the new instance, and any additional arguments provided
overwrite the base values.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The interaction with a preset and default keyword arguments is problematic.

Consider SoundEffect(). Does that create a new instance with all default values?

Consider SoundEffect(vol_end=255). Does that create a new instance with all default values except vol_end which is overridden to 255?

Consider SoundEffect(audio.SoundEffect.CLICK). Does that copy CLICK and then override everything with the default values? If not, how do you explain the difference between that and SoundEffect() behaviour?


How about changing the constructor signature to:

SoundEffect(preset=SoundEffect.DEFAULT, *, freq_start, freq_end, duration, vol_start, vol_end, wave, fx, interpolation)

Then SoundEffect.DEFAULT has the default values for the keyword-only args as previously specified above (freq_start=500 etc).

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Consider SoundEffect(). Does that create a new instance with all default values?

Yes, I think this one is straight forward.

Consider SoundEffect(vol_end=255). Does that create a new instance with all default values except vol_end which is overridden to 255?

Yes, as it would if the preset argument didn't exist.

Consider SoundEffect(audio.SoundEffect.CLICK). Does that copy CLICK and then override everything with the default values? If not, how do you explain the difference between that and SoundEffect() behaviour?

Okay, I could see how maybe that is a bit less obvious.
The way I would see it is that SoundEffect(audio.SoundEffect.CLICK) is like a copy constructor, so it ignores any of the __init__ default values.
I think SoundEffect(audio.SoundEffect.CLICK, duration=2000) could be easily read as "copy all audio.SoundEffect.CLICK values, but overwrite duration to 2000, but I could see how that might not be as obvious if written as SoundEffect(duration=2000, preset=audio.SoundEffect.CLICK).

Then SoundEffect.DEFAULT has the default values for the keyword-only args as previously specified above (freq_start=500 etc).

The disadvantage of that is we wouldn't be able to created shorter strings for serialisation with positional arguments.
So this shorter string wouldn't eval:

SoundEffect(988, 440, 190, 255, 255, 0, 1, 8)

If the confusion arising from having a preset argument in the constructor, what if the only way to duplicate and modify would be to use a copy() method?

my_effect = audio.SoundEffect.CLICK.copy()
my.effect.duration = 2000

It's an extra line, but might be easier to understand?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

SoundEffect(988, 440, 190, 255, 255, 0, 1, 8)

This won't work because 988 is not a valid preset. It would need to be SoundEffect(None, 988, 440, 190, 255, 255, 0, 1, 8).

Actually, the way it's implemented at the moment is that if you pass None as the preset argument then the preset is the default set of values. And any extra keyword args override that default /None preset value.

The signature for that is:

SoundEffect(preset=None, *, freq_start, freq_end, duration, vol_start, vol_end, wave, fx, interpolation)

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

SoundEffect(988, 440, 190, 255, 255, 0, 1, 8)

This won't work because 988 is not a valid preset. It would need to be SoundEffect(None, 988, 440, 190, 255, 255, 0, 1, 8).

Yes, sorry, that would only work like that if we moved preset to be the last parameter, as it was suggested in #753 (comment). I thought I had added a comment somewhere to indicate that having it as the last parameter would help with __repr__, but I couldn't find it, so I probably forgot to write it down.


The more I think about it, the better the copy() method sounds. Do you have any thoughts about it?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's been agreed to remove the preset constructor parameter, and add a copy() method to be able to clone SoundEffects.

:param freq_start: Start Frequency in Hertz (Hz), eg: ``400``
:param freq_end: End Frequency in Hertz (Hz), eg: ``2000``
:param duration: Duration of the sound (ms), eg: ``500``
:param vol_start: Start volume value, range 0-255, eg: ``120``
:param vol_end: End volume value, range 0-255, eg: ``255``
:param wave: Type of wave shape, one of these values: ``WAVE_SINE``,
``WAVE_SAWTOOTH``, ``WAVE_TRIANGLE``, ``WAVE_SQUARE``,
``WAVE_NOISE`` (randomly generated noise).
:param fx: Effect to add on the sound, one of the following values:
``FX_TREMOLO``, ``FX_VIBRATO``, ``FX_WARBLE``, or ``None``.
:param interpolation: The type of curve between the start and end
frequencies, different wave shapes have different rates of change
in frequency. One of the following values: ``INTER_LINEAR``,
``INTER_CURVE``, ``INTER_LOG``.

.. py:attribute:: freq_start

Start Frequency in Hertz (Hz)

.. py:attribute:: freq_end

End Frequency in Hertz (Hz)

.. py:attribute:: duration

Duration of the sound (ms), eg: ``500``

.. py:attribute:: vol_start

Start volume value, range 0-255, eg: ``120``

.. py:attribute:: vol_end

End volume value, range 0-255, eg: ``255``

.. py:attribute:: wave

Type of wave shape, one of these values: ``WAVE_SINE``,
``WAVE_SAWTOOTH``, ``WAVE_TRIANGLE``, ``WAVE_SQUARE``,
``WAVE_NOISE`` (randomly generated noise).

.. py:attribute:: fx

Effect to add on the sound, one of the following values:
``FX_TREMOLO``, ``FX_VIBRATO``, ``FX_WARBLE``, or ``None``.

.. py:attribute:: interpolation

The type of curve between the start and end
frequencies, different wave shapes have different rates of change
in frequency. One of the following values: ``INTER_LINEAR``,
``INTER_CURVE``, ``INTER_LOG``.

The arguments used to create any Sound Effect, including the built in ones,
can be inspected by looking at each of the SoundEffect instance attributes,
or by converting the instance into a string (which can be done via ``str()``
function, or by using a function that does the conversion automatically like
``print()``).

For example, with the :doc:`REPL </devguide/repl>` you can inspect the built
in Effects::

>>> audio.SoundEffect.CROAK
SoundEffect(freq_start=..., freq_end=..., duration=..., vol_start=..., vol_end=..., wave=..., fx=..., interpolation=...)

The built in Effects are immutable, so they cannot be changed. Trying to modify
a built in SoundEffect will throw an exception::

>>> audio.SoundEffect.CLICK.duration = 1000
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: effect cannot be modified

But a new one can be created like this::

>>> click_clone = SoundEffect(audio.SoundEffect.CLICK)
>>> click_clone.duration = 1000
>>>

Built in Sound Effects
----------------------

Some pre-created Sound Effects are already available as examples. These can
be played directly ``audio.play(audio.SoundEffect.SQUEAK)``,
or used as a base to create new effects
``audio.SoundEffect(audio.SoundEffect.SQUEAK, duration=2000)``.

* ``audio.SoundEffect.SQUEAK``
* ``audio.SoundEffect.WARBLE``
* ``audio.SoundEffect.CHIRP``
* ``audio.SoundEffect.CROAK``
* ``audio.SoundEffect.CLICK``

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was really happy with including these, but I think their relationship to the built-in sounds is curious to explain. I don't know that this means we shouldn't have them, but I'm struggling to work out what to call them.

audio.SoundEffect.CROAK for example is sort of weird to distinguish from Sound.TWINKLE - why can one of them be used as a preset and not the other? ("because one is a SoundEffect and one is a Sound"?)

This is one downside to "SoundEffect" over pure "Effect" - it ties the built-in sounds more closely to the SoundEffects - though @microbit-carlos points out that it does make sense that a 'Sound' is made out of SoundEffects...

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The existing built-in sounds could be rewritten as a (constant) tuple of (constant) SoundEffect instances. This is similar to sequences of images, like all the clock faces.

Eg:

>>> audio.Sound.SPRING
(SoundEffect(...), SoundEffect(...))

Then users could really see how Sound's are made and tweak them if they like.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, this is something that was quite desirable, our only concern is all the extra flash usage, specially if the strings are already in CODAL and probably cannot be excluded as they are.
Would be good to get an estimation of how much extra flash space this would consume?

Sound Effects Example
---------------------

::

from microbit import *

# Play a built in Sound Effect
audio.play(audio.SoundEffect.CHIRP)

# Create a Sound Effect and immediately play it
audio.play(audio.SoundEffect(
freq_start=400,
freq_end=2000,
duration=500,
vol_start=100,
vol_end=255,
wave=audio.WAVE_TRIANGLE,
fx=audio.FX_VIBRATO,
interpolation=audio.LOG
))

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I like this, it's nice and clear, but also think we should give some thought to when to document the positional, not keyword options.

@microbit-matt-hillsdon when autocomplete works well with positional arguments, it can be a really smooth editing experience - are there ways we could document the code here so that we get good positional argument help during autocomplete as long as the user hasn't used any keyword arguments

(for example, if I start typing audio.play(Effect(400 will the autocomplete in the editor be showing the parameter help from freq_start (as long as we re-order the parameters so start is first))

# Play a Sound Effect instance, modify an attribute, and play it again
my_effect = audio.SoundEffect(
preset=audio.CHIRP
freq_start=400,
freq_end=2000,
)
audio.play(my_effect)
my_effect.duration = 1000
audio.play(my_effect)

# You can also create a new effect based on an existing one, and modify
# any of its characteristics via arguments
audio.play(audio.SoundEffect.WARBLE)
my_modified_effect = SoundEffect(audio.SoundEffect.WARBLE, duration=1000)
audio.play(my_modified_effect)

# Use sensor data to modify and play the existing Sound Effect instance
while True:
my_effect.freq_start=accelerometer.get_x()
my_effect.freq_end=accelerometer.get_y()
audio.play(my_effect)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is really nice and much clearer than also containing all the parameters and @microbit-giles I'd like us to make sure we include some nice lean examples in the edu content that encourage people to use this cleaner interface.


if button_a.is_pressed():
# On button A play an effect and once it's done show an image
audio.play(audio.SoundEffect.CHIRP)
display.show(Image.DUCK)
sleep(500)
elif button_b.is_pressed():
# On button B play an effect while showing an image
audio.play(audio.SoundEffect.CLICK, wait=False)
display.show(Image.SQUARE)
sleep(500)


AudioFrame
==========

.. py:class::
AudioFrame

An ``AudioFrame`` object is a list of 32 samples each of which is an unsigned byte
(whole number between 0 and 255).

It takes just over 4 ms to play a single frame.

.. py:function:: copyfrom(other)

Overwrite the data in this ``AudioFrame`` with the data from another
``AudioFrame`` instance.

:param other: ``AudioFrame`` instance from which to copy the data.

Technical Details
=================
-----------------

.. note::
You don't need to understand this section to use the ``audio`` module.
Expand All @@ -104,11 +315,11 @@ samples. When reading reaches the start or the mid-point of the buffer, it
triggers a callback to fetch the next ``AudioFrame`` which is then copied into
the buffer. This means that a sound source has under 4ms to compute the next
``AudioFrame``, and for reliable operation needs to take less 2ms (which is
32000 cycles, so should be plenty).
32k cycles in micro:bit V1 or 128k in V2, so should be plenty).


Example
=======
AudioFrame Example
------------------

.. include:: ../examples/waveforms.py
:code: python