docs: Add a "Reset and Boot Sequence" reference page.

Previously individual ports documented these aspects to varying degrees,
but most of the information is common to all ports.

In particular, this adds a canonical explanation of `boot.py` and
`main.py`.

This work was funded through GitHub Sponsors.

Signed-off-by: Angus Gratton <angus@redyak.com.au>

docs/esp8266/general.rst

@@ -74,40 +74,7 @@ as possible after use.
 Boot process
 ------------
 
-On boot, MicroPython EPS8266 port executes ``_boot.py`` script from internal
+See :doc:`/reference/reset_boot`.
-frozen modules. It mounts filesystem in FlashROM, or if it's not available,
-performs first-time setup of the module and creates the filesystem. This
-part of the boot process is considered fixed, and not available for customization
-for end users (even if you build from source, please refrain from changes to
-it; customization of early boot process is available only to advanced users
-and developers, who can diagnose themselves any issues arising from
-modifying the standard process).
-
-Once the filesystem is mounted, ``boot.py`` is executed from it. The standard
-version of this file is created during first-time module set up and has
-commands to start a WebREPL daemon (disabled by default, configurable
-with ``webrepl_setup`` module), etc. This
-file is customizable by end users (for example, you may want to set some
-parameters or add other services which should be run on
-a module start-up). But keep in mind that incorrect modifications to boot.py
-may still lead to boot loops or lock ups, requiring to reflash a module
-from scratch. (In particular, it's recommended that you use either
-``webrepl_setup`` module or manual editing to configure WebREPL, but not
-both).
-
-As a final step of boot procedure, ``main.py`` is executed from filesystem,
-if exists. This file is a hook to start up a user application each time
-on boot (instead of going to REPL). For small test applications, you may
-name them directly as ``main.py``, and upload to module, but instead it's
-recommended to keep your application(s) in separate files, and have just
-the following in ``main.py``::
-
-    import my_app
-    my_app.main()
-
-This will allow to keep the structure of your application clear, as well as
-allow to install multiple applications on a board, and switch among them.
-
 
 Known Issues
 ------------

docs/esp8266/quickref.rst

@@ -163,10 +163,10 @@ sys.stdin.read() if it's needed to read characters from the UART(0)
 while it's also used for the REPL (or detach, read, then reattach).
 When detached the UART(0) can be used for other purposes.
 
-If there are no objects in any of the dupterm slots when the REPL is
+If there are no objects in any of the dupterm slots when the REPL is started (on
-started (on hard or soft reset) then UART(0) is automatically attached.
+:doc:`hard or soft reset </reference/reset_boot>`) then UART(0) is automatically
-Without this, the only way to recover a board without a REPL would be to
+attached. Without this, the only way to recover a board without a REPL would be
-completely erase and reflash (which would install the default boot.py which
+to completely erase and reflash (which would install the default boot.py which
 attaches the REPL).
 
 To detach the REPL from UART0, use::

docs/library/builtins.rst

@@ -170,6 +170,10 @@ Exceptions
 
 .. exception:: KeyboardInterrupt
 
+   |see_cpython| `python:KeyboardInterrupt`.
+
+   See also in the context of :ref:`soft_bricking`.
+
 .. exception:: KeyError
 
 .. exception:: MemoryError

docs/library/machine.RTC.rst

@@ -83,7 +83,7 @@ Methods
    a `bytes` object.
 
    Data written to RTC user memory is persistent across restarts, including
-   `machine.soft_reset()` and `machine.deepsleep()`.
+   :ref:`soft_reset` and `machine.deepsleep()`.
 
    The maximum length of RTC user memory is 2048 bytes by default on esp32,
    and 492 bytes on esp8266.

docs/library/machine.USBDevice.rst

@@ -32,10 +32,10 @@ Managing a runtime USB interface can be tricky, especially if you are communicat
 with MicroPython over a built-in USB-CDC serial port that's part of the same USB
 device.
 
-- A MicroPython soft reset will always clear all runtime USB interfaces, which
+- A MicroPython :ref:`soft reset <soft_reset>` will always clear all runtime USB
-  results in the entire USB device disconnecting from the host. If MicroPython
+  interfaces, which results in the entire USB device disconnecting from the
-  is also providing a built-in USB-CDC serial port then this will re-appear
+  host. If MicroPython is also providing a built-in USB-CDC serial port then
-  after the soft reset.
+  this will re-appear after the soft reset.
 
   This means some functions (like ``mpremote run``) that target the USB-CDC
   serial port will immediately fail if a runtime USB interface is active,
@@ -44,9 +44,9 @@ device.
   no more runtime USB interface.
 
 - To configure a runtime USB device on every boot, it's recommended to place the
-  configuration code in the ``boot.py`` file on the :ref:`device VFS
+  configuration code in the :ref:`boot.py` file on the :ref:`device VFS
   <filesystem>`. On each reset this file is executed before the USB subsystem is
-  initialised (and before ``main.py``), so it allows the board to come up with the runtime
+  initialised (and before :ref:`main.py`), so it allows the board to come up with the runtime
   USB device immediately.
 
 - For development or debugging, it may be convenient to connect a hardware

docs/library/machine.rst

@@ -62,14 +62,13 @@ Reset related functions
 
 .. function:: reset()
 
-   Resets the device in a manner similar to pushing the external RESET
+   :ref:`Hard resets <hard_reset>` the device in a manner similar to pushing the
-   button.
+   external RESET button.
 
 .. function:: soft_reset()
 
-   Performs a soft reset of the interpreter, deleting all Python objects and
+   Performs a :ref:`soft reset <soft_reset>` of the interpreter, deleting all
-   resetting the Python heap.  It tries to retain the method by which the user
+   Python objects and resetting the Python heap.
-   is connected to the MicroPython REPL (eg serial, USB, Wifi).
 
 .. function:: reset_cause()
 

docs/library/pyb.rst

@@ -147,10 +147,10 @@ Power related functions
    (internal oscillator) directly.  The higher frequencies use the HSE to
    drive the PLL (phase locked loop), and then use the output of the PLL.
 
-   Note that if you change the frequency while the USB is enabled then
+   Note that if you change the frequency while the USB is enabled then the USB
-   the USB may become unreliable.  It is best to change the frequency
+   may become unreliable. It is best to change the frequency in :ref:`boot.py`,
-   in boot.py, before the USB peripheral is started.  Also note that sysclk
+   before the USB peripheral is started. Also note that sysclk frequencies below
-   frequencies below 36MHz do not allow the USB to function correctly.
+   36MHz do not allow the USB to function correctly.
 
 .. function:: wfi()
 
@@ -205,8 +205,9 @@ Miscellaneous functions
 
 .. function:: main(filename)
 
-   Set the filename of the main script to run after boot.py is finished.  If
+   Set the filename of the main script to run after :ref:`boot.py` is finished.
-   this function is not called then the default file main.py will be executed.
+   If this function is not called then the default file :ref:`main.py` will be
+   executed.
 
    It only makes sense to call this function from within boot.py.
 

docs/pyboard/tutorial/reset.rst

@@ -10,6 +10,8 @@ execution of ``boot.py`` and ``main.py`` and gives default USB settings.
 If you have problems with the filesystem you can do a factory reset,
 which restores the filesystem to its original state.
 
+For more information, see :doc:`/reference/reset_boot`.
+
 Safe mode
 ---------
 

docs/reference/index.rst

@@ -21,6 +21,7 @@ implementation and the best practices to use them.
 
    glossary.rst
    repl.rst
+   reset_boot.rst
    mpremote.rst
    mpyfiles.rst
    isr_rules.rst

docs/reference/mpremote.rst

@@ -547,9 +547,9 @@ device at ``/dev/ttyACM1``, printing each result.
 
   mpremote resume exec "print_state_info()" soft-reset
 
-Connect to the device without triggering a soft reset and execute the
+Connect to the device without triggering a :ref:`soft reset <soft_reset>` and
-``print_state_info()`` function (e.g. to find out information about the current
+execute the ``print_state_info()`` function (e.g. to find out information about
-program state), then trigger a soft reset.
+the current program state), then trigger a soft reset.
 
 .. code-block:: bash
 

docs/reference/repl.rst

@@ -143,10 +143,12 @@ the auto-indent feature, and changes the prompt from ``>>>`` to ``===``. For exa
 Paste Mode allows blank lines to be pasted. The pasted text is compiled as if
 it were a file. Pressing Ctrl-D exits paste mode and initiates the compilation.
 
+.. _repl_soft_reset:
+
 Soft reset
 ----------
 
-A soft reset will reset the python interpreter, but tries not to reset the
+A :ref:`soft_reset` will reset the python interpreter, but tries not to reset the
 method by which you're connected to the MicroPython board (USB-serial, or Wifi).
 
 You can perform a soft reset from the REPL by pressing Ctrl-D, or from your python
@@ -182,6 +184,9 @@ variables no longer exist:
     ['__name__', 'pyb']
     >>>
 
+For more information about reset types and the startup process, see
+:doc:`/reference/reset_boot`.
+
 The special variable _ (underscore)
 -----------------------------------
 
@@ -196,6 +201,8 @@ So you can use the underscore to save the result in a variable. For example:
     15
     >>>
 
+.. _raw_repl:
+
 Raw mode and raw-paste mode
 ---------------------------
 

docs/reference/reset_boot.rst

@@ -0,0 +1,260 @@
+Reset and Boot Sequence
+=======================
+
+A device running MicroPython follows a particular boot sequence to start up and
+initialise itself after a reset.
+
+.. _hard_reset:
+
+Hard reset
+----------
+
+Booting from hard reset is what happens when a board is first powered up, a cold
+boot. This is a complete reset of the MCU hardware.
+
+The MicroPython port code initialises all essential hardware (including embedded
+clocks and power regulators, internal serial UART, etc), and then starts the
+MicroPython environment. Existing :doc:`RTC </library/machine.RTC>`
+configuration may be retained after a hard reset, but all other hardware state
+is cleared.
+
+The same hard reset boot sequence can be triggered by a number of events such as:
+
+- Python code executing :func:`machine.reset()`.
+- User presses a physical Reset button on the board (where applicable).
+- Waking from deep sleep (on most ports).
+- MCU hardware watchdog reset.
+- MCU hardware brown out detector.
+
+The details of hardware-specific reset triggers depend on the port and
+associated hardware. The :func:`machine.reset_cause()` function can be used to
+further determine the cause of a reset.
+
+.. _soft_reset:
+
+Soft Reset
+----------
+
+When MicroPython is already running, it's possible to trigger a soft reset by
+:ref:`typing Ctrl-D in the REPL <repl_soft_reset>` or executing
+:func:`machine.soft_reset()`.
+
+A soft reset clears the Python interpreter, frees all Python memory, and starts
+the MicroPython environment again.
+
+State which is cleared by a soft reset includes:
+
+- All Python variables, objects, imported modules, etc.
+- Most peripherals configured using the :doc:`machine module
+  </library/machine>`. There are very limited exceptions, for example
+  :doc:`machine.Pin </library/machine.Pin>` modes (i.e. if a pin is input or
+  output, high or low) are not reset on most ports. More advanced configuration
+  such as :func:`Pin.irq()` is always reset.
+- Bluetooth.
+- Network sockets. Open TCP sockets are closed cleanly with respect to the other party.
+- Open files. The filesystem is left in a valid state.
+
+Some system state remains the same after a soft reset, including:
+
+- Any existing network connections (Ethernet, Wi-Fi, etc) remain active at the
+  IP Network layer. Querying the :doc:`network interface from code
+  </library/network>` may indicate the network interface is still active with a
+  configured IP address, etc.
+- An active :doc:`REPL <repl>` appears continuous before and after soft reset,
+  except in some unusual cases:
+
+  * If the :ref:`machine.USBDevice <machine.USBDevice>` class has been used to
+    create a custom USB interface then any built-in USB serial device will
+    appear to disconnect and reconnect as the custom USB interface must be
+    cleared during reset.
+  * A serial UART REPL will restore its default hardware configuration (baud
+    rate, etc).
+
+- CPU clock speed is usually not changed by a soft reset.
+- :doc:`RTC </library/machine.RTC>` configuration (i.e. setting of the current
+  time) is not changed by soft reset.
+
+.. _boot_sequence:
+
+Boot Sequence
+-------------
+
+When MicroPython boots following either a hard or soft reset, it follows this
+boot sequence in order:
+
+_boot.py
+^^^^^^^^
+
+This is an internal script :doc:`frozen into the MicroPython firmware
+<manifest>`. It is provided by MicroPython on many ports to do essential
+initialisation.
+
+For example, on most ports ``_boot.py`` will detect the first boot of a new
+device and format the :doc:`internal flash filesystem <filesystem>` ready for
+use.
+
+Unless you're creating a custom MicroPython build or adding a new port then you
+probably don't need to worry about ``_boot.py``. It's best not to change the
+contents unless you really know what you're doing.
+
+.. _boot.py:
+
+boot.py
+^^^^^^^
+
+A file named ``boot.py`` can be copied to the board's internal :ref:`filesystem
+<filesystem>` using :doc:`mpremote <mpremote>`.
+
+If ``boot.py`` is found then it is executed. You can add code in ``boot.py`` to
+perform custom one-off initialisation (for example, to configure the board's
+hardware).
+
+A common practice is to configure a board's network connection in ``boot.py`` so
+that it's always available after reset for use with the :doc:`REPL <repl>`,
+:doc:`mpremote <mpremote>`, etc.
+
+.. warning:: boot.py should always exit and not run indefinitely.
+
+   Depending on the port, some hardware initialisation is delayed until after
+   ``boot.py`` exits. This includes initialising USB on the stm32 port and all
+   ports which support :ref:`machine.USBDevice <machine.USBDevice>`. On these
+   ports, output printed from ``boot.py`` may not be visible on the built-in USB
+   serial port until after ``boot.py`` finishes running.
+
+   The purpose of this late initialisation is so that it's possible to
+   pre-configure particular hardware in ``boot.py``, and then have it start with
+   the correct configuration.
+
+.. note:: It is sometimes simpler to not have a ``boot.py`` file and place any
+          initialisation code at the top of ``main.py`` instead.
+
+.. _main.py:
+
+main.py
+^^^^^^^
+
+Similar to ``boot.py``, a file named ``main.py`` can be copied to the board's
+internal :ref:`filesystem <filesystem>`. If found then it is executed next in the
+startup process.
+
+``main.py`` is for any Python code that you want to run each time your device
+starts.
+
+Some tips for ``main.py`` usage:
+
+- ``main.py`` doesn't have to exit, feel free to put an infinite ``while
+  True`` loop in there.
+- For complex Python applications then you don't need to put all your
+  code in ``main.py``. ``main.py`` can be a simple entry point that
+  imports your application and starts execution::
+
+        import my_app
+        my_app.main()
+
+  This can help keep the structure of your application clear. It also makes
+  it easy to install multiple applications on a board and switch among them.
+- It's good practice when writing robust apps to wrap code in ``main.py`` with an
+  exception handler to take appropriate action if the code crashes. For example::
+
+        import machine, sys
+        import my_app
+        try:
+            my_app.main()
+        except Exception as e:
+            print("Fatal error in main:")
+            sys.print_exception(e)
+
+        # Following a normal Exception or main() exiting, reset the board.
+        # Following a non-Exception error such as KeyboardInterrupt (Ctrl-C),
+        # this code will drop to a REPL. Place machine.reset() in a finally
+        # block to always reset, instead.
+        machine.reset()
+
+  Otherwise MicroPython will drop to the REPL following any crash or if main
+  exits (see below).
+
+- Any global variables that were set in ``boot.py`` will still be set in the
+  global context of ``main.py``.
+
+- To fully optimise flash usage and memory consumption, you can copy
+  :doc:`pre-compiled <mpyfiles>` ``main.mpy`` and/or ``boot.mpy`` files to the
+  filesystem, or even :doc:`freeze <manifest>` them into the firmware build
+  instead.
+- ``main.py`` execution is skipped when a soft reset is initiated from :ref:`raw
+  REPL mode <raw_repl>` (for example, when :doc:`mpremote <mpremote>` or another
+  program is interacting directly with MicroPython).
+
+Interactive Interpreter (REPL)
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+If ``main.py`` is not found, or if ``main.py`` exits, then :doc:`repl`
+will start immediately.
+
+.. note:: Even if ``main.py`` contains an infinite loop, typing Ctrl-C on the
+          REPL serial port will inject a `KeyboardInterrupt`. If no exception
+          handler catches it then ``main.py`` will exit and the REPL will start.
+
+Any global variables that were set in ``boot.py`` and ``main.py`` will still be
+set in the global context of the REPL.
+
+The REPL continues executing until Python code triggers a hard or soft reset.
+
+.. _soft_bricking:
+
+Soft Bricking (failure to boot)
+---------------------------------
+
+It is rare but possible for MicroPython to become unresponsive during startup, a
+state sometimes called "soft bricked". For example:
+
+- If ``boot.py`` execution gets stuck and the native USB serial port
+  never initialises.
+- If Python code reconfigures the REPL interface, making it inaccessible.
+
+Rest assured, recovery is possible!
+
+KeyboardInterrupt
+^^^^^^^^^^^^^^^^^
+
+In many cases, opening the REPL serial port and typing ``Ctrl-C`` will inject
+`KeyboardInterrupt` and may cause the running script to exit and a REPL to
+start. From the REPL, you can use :func:`os.remove()` to remove the misbehaving
+Python file::
+
+    import os
+    os.remove('main.py')
+
+To confirm which files are still present in the internal filesystem::
+
+    import os
+    os.listdir()
+
+Safe Mode and Factory Reset
+^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+If you're unable to easily access the REPL then you may need to perform one of
+two processes:
+
+1. "Safe mode" boot, which skips ``boot.py`` and ``main.py`` and immediately
+   starts a REPL, allowing you to clean up. This is only supported on some ports.
+2. Factory Reset to erase the entire contents of the flash filesystem. This may
+   also be necessary if the internal flash filesystem has become corrupted
+   somehow.
+
+The specific process(es) are different on each port:
+
+- :doc:`pyboard and stm32 port instructions </pyboard/tutorial/reset>`
+- :doc:`renesas-ra port instructions </renesas-ra/tutorial/reset>`
+- :doc:`wipy port instructions </wipy/tutorial/reset>`
+
+For ports without specific instructions linked above, the factory reset process
+involves erasing the board's entire flash and then flashing MicroPython again
+from scratch. Usually this will involve the same tool(s) that were originally
+used to install MicroPython. Consult the installation docs for your board, or
+ask on the `GitHub Discussions`_ if you're not sure.
+
+.. warning:: Re-flashing the MicroPython firmware without erasing the entire
+             flash first will usually not recover from soft bricking, as a
+             firmware update usually preserves the contents of the filesystem.
+
+.. _GitHub Discussions: https://github.com/orgs/micropython/discussions

docs/renesas-ra/tutorial/program_in_flash.rst

@@ -28,6 +28,8 @@ As the factory setting, following 2 files are created in the file system:
 * boot.py : executed first when the system starts
 * main.py : executed after boot.py completes
 
+See :doc:`/reference/reset_boot` for more information.
+
 Write a program in the internal file system
 -------------------------------------------
 

docs/renesas-ra/tutorial/reset.rst

@@ -20,6 +20,8 @@ If that isn't working you can perform a hard reset (turn-it-off-and-on-again)
 by pressing the RESET button. This will end your session, disconnecting
 whatever program (PuTTY, screen, etc) that you used to connect to the board.
 
+For more details, see :doc:`/reference/reset_boot`.
+
 boot mode
 ---------
 
@@ -29,7 +31,9 @@ There are 3 boot modes:
   * safe boot mode
   * factory filesystem boot mode
 
-boot.py and main.py are executed on "normal boot mode".
+boot.py and main.py are executed on "normal boot mode". See :ref:`boot_sequence`.
+
+The other modes can be used to recover from :ref:`soft_bricking`:
 
 boot.py and main.py are *NOT* executed on "safe boot mode".
 
@@ -46,16 +50,4 @@ on the board:
 
 You have created the main.py which executes LED1 blinking in the previous part.
 If you change the boot mode to safe boot mode, the MicroPython starts without
-the execution of main.py. Then you can remove the main.py by following
+the execution of main.py.
-command or change the boot mode to factory file system boot mode.::
-
-    import os
-    os.remove('main.py')
-
-or change the boot mode to factory file system boot mode.
-
-You can confirm that the initialized file system that there are only boot.py and main.py files.::
-
-    import os
-    os.listdir()
-

docs/wipy/tutorial/reset.rst

@@ -16,6 +16,8 @@ There are soft resets and hard resets.
         import machine
         machine.reset()
 
+For more information, see :doc:`/reference/reset_boot`.
+
 Safe boot
 ---------