Merge pull request #91 from wnienhaus/s2s3_support

ESP32-S2/S3 support

Author: wnienhaus

.gitignore

@@ -1,8 +1,16 @@
-tests/compat/*.bin
-tests/compat/*.elf
-tests/compat/*.o
-tests/compat/*.ulp
-tests/compat/*.log
+tests/binutils-gdb
+tests/esp-idf
+tests/ulptool
+tests/**/*.bin
+tests/**/*.elf
+tests/**/*.o
+tests/**/*.ulp
+tests/**/*.log
+tests/**/*.pre
+tests/log
+tests/*.lst
+tests/*.log
+tests/defines*.db
 demo.ulp
 *.pyc
 *.pyo

README.rst

@@ -15,7 +15,7 @@ micropython-esp32-ulp is an assembler toolchain for the ESP32 ULP (Ultra Low-Pow
 Co-Processor, written in MicroPython.
 
 It can translate small assembly language programs to a loadable/executable
-ULP machine code binary, directly on the ESP32 microcontroller.
+ULP-FSM (not RISC-V) machine code binary, directly on a ESP32 microcontroller.
 
 This is intended as an alternative approach to assembling such programs using
 the `binutils-gdb toolchain <https://github.com/espressif/binutils-gdb/tree/esp32ulp-elf-2.35>`_
@@ -30,13 +30,30 @@ Features
 The following features are supported:
 
 * the entire `ESP32 ULP instruction set <https://docs.espressif.com/projects/esp-idf/en/latest/esp32/api-reference/system/ulp_instruction_set.html>`_
+* the entire `ESP32-S2 ULP instruction set <https://docs.espressif.com/projects/esp-idf/en/latest/esp32s2/api-reference/system/ulp_instruction_set.html>`_
+  (this also covers the ESP32-S3) [#f1]_ [#f2]_
 * constants defined with ``.set``
 * constants defined with ``#define``
 * expressions in assembly code and constant definitions
 * RTC convenience macros (e.g. ``WRITE_RTC_REG``)
 * many ESP32 ULP code examples found on the web will work unmodified
 * a simple disassembler is also provided
 
+.. [#f1] Note: the ESP32-S2 and ESP32-S3 have the same ULP binary format between each other
+         but the binary format is different than that of the original ESP32 ULP. You need to
+         select the ``esp32s2`` cpu (`see docs </docs/index.rst>`_) when assembling code for
+         use on an ESP32-S2/S3.
+
+.. [#f2] Note: The ESP32-S2 and ESP32-S3 have the same ULP binary format, but the peripheral
+         register addresses (those accessed with REG_RD and REG_WR) are different. For best
+         results, use the correct peripheral register addresses for the specific variant you
+         are working with. The assembler (when used with ``cpu=esp32s2``) will accept
+         addresses for any of the 3 variants, because they are translated into relative
+         offsets anyway and many registers live at the same relative offset on all 3 variants.
+         This conveniently means that the same assembly code can assembled unmodified for each
+         variant and produce a correctly working binary - as long as only peripheral registers
+         are used, which have the same relative offset across the variants. Use with care!
+
 
 Quick start
 -----------
@@ -66,10 +83,12 @@ See `docs/index.rst </docs/index.rst>`_.
 Requirements
 ------------
 
-The minimum supported version of MicroPython is v1.12.
+The minimum supported version of MicroPython is v1.12. (For ESP32-S2 and S3
+devices, a version greater than v1.20 is required as versions before that
+did not enable the ``esp32.ULP`` class).
 
-An ESP32 is required to run the ULP machine code binary produced by micropython-esp32-ulp
-(the ESP32-S2 will not work as it is not binary compatible with the ESP32).
+An ESP32 device is required to run the ULP machine code binary produced by
+micropython-esp32-ulp.
 
 
 License

docs/disassembler.rst

@@ -25,6 +25,8 @@ You can also specify additional options to ``disassemble.py`` as follows:
 +--------------------------+----------------------------------------------------------------+
 | Option                   | Description                                                    |
 +==========================+================================================================+
+| ``-c`` or ``--mcpu``     | Choose ULP variant: either esp32 or esp32s2                    |
++--------------------------+----------------------------------------------------------------+
 | ``-h``                   | Show help text                                                 |
 +--------------------------+----------------------------------------------------------------+
 || ``-m <bytes sequence>`` || Disassemble a provided sequence of hex bytes                  |
@@ -43,18 +45,31 @@ specified file.
 Note that the ULP header is validates and files with unknown magic bytes will be
 rejected. The correct 4 magic bytes at the start of a ULP binary are ``ulp\x00``.
 
-Example:
+Example disassembling an ESP32 ULP binary:
 
 .. code-block:: shell
 
    $ micropython -m tools.disassemble path/to/binary.ulp
    .text
    0000  040000d0  LD r0, r1, 0
-   0004  0e0400d0  LD r2, r3, 1
+   0004  0e0000d0  LD r2, r3, 0
+   0008  04000068  ST r0, r1, 0
+   000c  0b000068  ST r3, r2, 0
+   .data
+   0010  00000000  <empty>
+
+Example disassembling an ESP32-S2 ULP binary:
+
+.. code-block:: shell
+
+   $ micropython -m tools.disassemble -c esp32s2 path/to/binary.ulp
+   .text
+   0000  040000d0  LD r0, r1, 0
+   0004  0e0000d0  LD r2, r3, 0
    0008  84010068  ST r0, r1, 0
-   000c  8b090068  ST r3, r2, 2
+   000c  8b010068  ST r3, r2, 0
    .data
-   0000  00000000  <empty>
+   0010  00000000  <empty>
 
 
 Disassembling a byte sequence
@@ -129,18 +144,20 @@ For example:
 Disassembling on device
 -----------------------------
 
-The disassembler also works when used on an ESP32.
+The disassembler also works when used on an ESP32 device.
 
 To use the disassembler on a real device:
 
 * ensure ``micropython-esp32-ulp`` is installed on the device (see `docs/index.rst </docs/index.rst>`_).
-* upload ``tools/disassemble.py`` to the device (any directory will do)
-* run the following:
+* upload ``tools/disassemble.py`` ``tools/decode.py`` and ``tools/decode_s2.py`` to the device
+  (any directory will do, as long as those 3 files are in the same directory)
+* the following example code assumes you placed the 3 files into the device's "root" directory
+* run the following (note, we must specify which the cpu the binary is for):
 
   .. code-block:: python
 
      from disassemble import disassemble_file
      # then either:
-     disassemble_file('path/to/file.ulp')  # normal mode
+     disassemble_file('path/to/file.ulp', cpu='esp32s2')  # normal mode
      # or:
-     disassemble_file('path/to/file.ulp', True)  # verbose mode
+     disassemble_file('path/to/file.ulp', cpu='esp32s2', verbose=True)  # verbose mode

docs/index.rst

@@ -58,6 +58,13 @@ follows:
    cd micropython-esp32-ulp
    micropython -m esp32_ulp path/to/code.S  # this results in path/to/code.ulp
 
+The assembler supports selecting a CPU to assemble for using the ``-c`` option
+(valid cpu's are ``esp32`` and ``esp32s2``):
+
+.. code-block:: shell
+
+   micropython -m esp32_ulp -c esp32s2 path/to/code.S  # assemble for an ESP32-S2
+
 
 More examples
 +++++++++++++
@@ -86,12 +93,13 @@ assembly source file into a machine code binary file with a ``.ulp`` extension.
 That file can then be loaded directly without assembling the source again.
 
 1. Create/upload an assembly source file and run the following to get a
-   loadable ULP binary as a ``.ulp`` file:
+   loadable ULP binary as a ``.ulp`` file (specify ``cpu='esp32s2'`` if you
+   have an ESP32-S2 or ESP32-S3 device):
 
    .. code-block:: python
 
       import esp32_ulp
-      esp32_ulp.assemble_file('code.S')  # this results in code.ulp
+      esp32_ulp.assemble_file('code.S', cpu='esp32')  # this results in code.ulp
 
 2. The above prints out the offsets of all global symbols/labels. For the next
    step, you will need to note down the offset of the label, which represents
@@ -153,7 +161,6 @@ Currently the following are not supported:
 * assembler macros using ``.macro``
 * preprocessor macros using ``#define A(x,y) ...``
 * including files using ``#include``
-* ESP32-S2 (not binary compatible with the ESP32)
 
 
 Testing
@@ -164,7 +171,8 @@ output is identical with what Espressif's esp32-elf-as (from their `binutils-gdb
 <https://github.com/espressif/binutils-gdb/tree/esp32ulp-elf-2.35>`_) produces.
 
 micropython-esp32-ulp has been tested on the Unix port of MicroPython and on real ESP32
-devices with the chip type ESP32D0WDQ6 (revision 1) without SPIRAM.
+devices with the chip type ESP32D0WDQ6 (revision 1) without SPIRAM as well as ESP32-S2
+(ESP32-S2FH4) and ESP32-S3 (ESP32-S3R8) devices.
 
 Consult the Github Actions `workflow definition file </.github/workflows/run_tests.yaml>`_
 for how to run the different tests.

docs/preprocess.rst

@@ -95,6 +95,21 @@ are not needed on the device either.)
       micropython -m esp32_ulp.parse_to_db \
         esp-idf/components/soc/esp32/include/soc/{soc,soc_ulp,rtc_cntl_reg,rtc_io_reg,sens_reg}.h
 
+
+   .. warning::
+
+      `:warning:` Ensure that you include the header files for the correct
+      variant you are working with. In the example code above, simply switch
+      ``esp32`` to ``esp32s2`` or ``esp32s3`` in the path to the include files.
+
+      There are subtle differences across the ESP32 variants such as which
+      constants are available or the value of certain constants. For example,
+      peripheral register addresses differ between the 3 variants even though
+      many constants for peripheral registers are available on all 3 variants.
+      Other constants such as those relating to the HOLD functionality of touch
+      pads are only available on the original ESP32.
+
+
 2. Using the defines database during preprocessing
 
    The preprocessor will automatically use a defines database, when using the
@@ -108,6 +123,7 @@ are not needed on the device either.)
    or instantiate the ``Preprocessor`` class directly, without passing it a
    DefinesDB instance via ``use_db``.
 
+
 Design choices
 --------------
 

esp32_ulp/__init__.py

@@ -6,8 +6,8 @@
 garbage_collect('after import')
 
 
-def src_to_binary(src):
-    assembler = Assembler()
+def src_to_binary(src, cpu):
+    assembler = Assembler(cpu)
     src = preprocess(src)
     assembler.assemble(src, remove_comments=False)  # comments already removed by preprocessor
     garbage_collect('before symbols export')
@@ -19,11 +19,11 @@ def src_to_binary(src):
     return make_binary(text, data, bss_len)
 
 
-def assemble_file(filename):
+def assemble_file(filename, cpu):
     with open(filename) as f:
         src = f.read()
 
-    binary = src_to_binary(src)
+    binary = src_to_binary(src, cpu)
 
     if filename.endswith('.s') or filename.endswith('.S'):
         filename = filename[:-2]

esp32_ulp/__main__.py

@@ -2,10 +2,18 @@
 from . import assemble_file
 
 
-def main(fn):
-    assemble_file(fn)
+def main(fn, cpu):
+    assemble_file(fn, cpu)
 
 
 if __name__ == '__main__':
-    main(sys.argv[1])
+    cpu = 'esp32'
+    filename = sys.argv[1]
+    if len(sys.argv) > 3:
+        if sys.argv[1] in ('-c', '--mcpu'):
+            cpu = sys.argv[2].lower()
+            if cpu not in ('esp32', 'esp32s2'):
+                raise ValueError('Invalid cpu')
+            filename = sys.argv[3]
+    main(filename, cpu)
 

esp32_ulp/assemble.py

@@ -3,7 +3,6 @@
 """
 
 import re
-from . import opcodes
 from .nocomment import remove_comments as do_remove_comments
 from .util import garbage_collect
 
@@ -88,9 +87,19 @@ def set_global(self, symbol):
 
 class Assembler:
 
-    def __init__(self, symbols=None, bases=None, globals=None):
+    def __init__(self, cpu='esp32', symbols=None, bases=None, globals=None):
+        if cpu == 'esp32':
+            opcode_module = 'opcodes'
+        elif cpu == 'esp32s2':
+            opcode_module = 'opcodes_s2'
+        else:
+            raise ValueError('Invalid cpu')
+
+        relative_import = 1 if '/' in __file__ else 0
+        self.opcodes = __import__(opcode_module, None, None, [], relative_import)
+
         self.symbols = SymbolTable(symbols or {}, bases or {}, globals or {})
-        opcodes.symbols = self.symbols  # XXX dirty hack
+        self.opcodes.symbols = self.symbols  # XXX dirty hack
 
         # regex for parsing assembly lines
         # format: [[whitespace]label:][whitespace][opcode[whitespace arg[,arg...]]]
@@ -223,7 +232,7 @@ def d_align(self, align=4, fill=None):
             self.fill(self.section, amount, fill)
 
     def d_set(self, symbol, expr):
-        value = int(opcodes.eval_arg(expr))
+        value = int(self.opcodes.eval_arg(expr))
         self.symbols.set_sym(symbol, ABS, None, value)
 
     def d_global(self, symbol):
@@ -264,13 +273,13 @@ def assembler_pass(self, lines):
                 else:
                     # machine instruction
                     opcode_lower = opcode.lower()
-                    func = getattr(opcodes, 'i_' + opcode_lower, None)
+                    func = getattr(self.opcodes, 'i_' + opcode_lower, None)
                     if func is not None:
                         if self.a_pass == 1:
                             # during the first pass, symbols are not all known yet.
                             # so we add empty instructions to the section, to determine
                             # section sizes and symbol offsets for pass 2.
-                            result = (0,) * opcodes.no_of_instr(opcode_lower, args)
+                            result = (0,) * self.opcodes.no_of_instr(opcode_lower, args)
                         else:
                             result = func(*args)
 

esp32_ulp/opcodes.py

@@ -379,7 +379,7 @@ def i_reg_wr(reg, high_bit, low_bit, val):
         _wr_reg.addr = reg & 0xff
         _wr_reg.periph_sel = (reg & 0x300) >> 8
     else:
-        _wr_reg.addr = (reg & 0xff) >> 2
+        _wr_reg.addr = (reg >> 2) & 0xff
         _wr_reg.periph_sel = _soc_reg_to_ulp_periph_sel(reg)
     _wr_reg.data = get_imm(val)
     _wr_reg.low = get_imm(low_bit)
@@ -394,7 +394,7 @@ def i_reg_rd(reg, high_bit, low_bit):
         _rd_reg.addr = reg & 0xff
         _rd_reg.periph_sel = (reg & 0x300) >> 8
     else:
-        _rd_reg.addr = (reg & 0xff) >> 2
+        _rd_reg.addr = (reg >> 2) & 0xff
         _rd_reg.periph_sel = _soc_reg_to_ulp_periph_sel(reg)
     _rd_reg.unused = 0
     _rd_reg.low = get_imm(low_bit)