TonBERRY/micropython
3
0
Fork 0
Code Pull Requests Activity

top: Replace upip with mip everywhere.

Browse Source 
Updates all README.md and docs, and manifests to `require("mip")`.

Also extend and improve the documentation on freezing and packaging.

This work was funded through GitHub Sponsors.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
This commit is contained in:
Jim Mussared
2022-09-29 17:49:58 +10:00
parent ba3652f15d
commit 924a3e03ec
16 changed files with 404 additions and 878 deletions
Download Patch File Download Diff File Expand all files Collapse all files

30
docs/reference/glossary.rst
View File

@@ -52,7 +52,7 @@ Glossary
cross-compiler
Also known as ``mpy-cross``. This tool runs on your PC and converts a
:term:`.py file` containing MicroPython code into a :term:`.mpy file`
containing MicroPython bytecode. This means it loads faster (the board
containing MicroPython :term:`bytecode`. This means it loads faster (the board
doesn't have to compile the code), and uses less space on flash (the
bytecode is more space efficient).
@@ -128,7 +128,7 @@ Glossary
Unlike the :term:`CPython` stdlib, micropython-lib modules are
intended to be installed individually - either using manual copying or
using :term:`upip`.
using :term:`mip`.
MicroPython port
MicroPython supports different :term:`boards <board>`, RTOSes, and
@@ -151,16 +151,26 @@ Glossary
machine-independent features. It can also function in a similar way to
:term:`CPython`'s ``python`` executable.
mip
A package installer for MicroPython (mip - "mip installs packages"). It
installs MicroPython packages either from :term:`micropython-lib`,
GitHub, or arbitrary URLs. mip can be used on-device on
network-capable boards, and internally by tools such
as :term:`mpremote`.
mpremote
A tool for interacting with a MicroPython device. See :ref:`mpremote`.
.mpy file
The output of the :term:`cross-compiler`. A compiled form of a
:term:`.py file` that contains MicroPython bytecode instead of Python
source code.
:term:`.py file` that contains MicroPython :term:`bytecode` instead of
Python source code.
native
Usually refers to "native code", i.e. machine code for the target
microcontroller (such as ARM Thumb, Xtensa, x86/x64). The ``@native``
decorator can be applied to a MicroPython function to generate native
code instead of bytecode for that function, which will likely be
code instead of :term:`bytecode` for that function, which will likely be
faster but use more RAM.
port
@@ -193,8 +203,10 @@ Glossary
as a serial port over USB.
upip
(Literally, "micro pip"). A package manager for MicroPython, inspired
A now-obsolete package manager for MicroPython, inspired
by :term:`CPython`'s pip, but much smaller and with reduced
functionality.
upip runs both on the :term:`Unix port <MicroPython Unix port>` and on
:term:`baremetal` ports which offer filesystem and networking support.
functionality. See its replacement, :term:`mip`.
webrepl
A way of connecting to the REPL (and transferring files) on a device
over the internet from a browser. See https://micropython.org/webrepl

271
docs/reference/manifest.rst
View File

@@ -1,35 +1,177 @@
.. _manifest:
MicroPython manifest files
==========================
When building firmware for a device the following components are included in
the compilation process:
Summary
-------
- the core MicroPython virtual machine and runtime
- port-specific system code and drivers to interface with the
microcontroller/device that the firmware is targeting
- standard built-in modules, like ``sys``
- extended built-in modules, like ``json`` and ``machine``
- extra modules written in C/C++
- extra modules written in Python
MicroPython has a feature that allows Python code to be "frozen" into the
firmware, as an alternative to loading code from the filesystem.
All the modules included in the firmware are available via ``import`` from
Python code. The extra modules written in Python that are included in a build
(the last point above) are called *frozen modules*, and are specified by a
``manifest.py`` file. Changing this manifest requires rebuilding the firmware.
This has the following benefits:
It's also possible to add additional modules to the filesystem of the device
once it is up and running. Adding and removing modules to/from the filesystem
does not require rebuilding the firmware so is a simpler process than rebuilding
firmware. The benefit of using a manifest is that frozen modules are more
efficient: they are faster to import and take up less RAM once imported.
- the code is pre-compiled to bytecode, avoiding the need for the Python
source to be compiled at load-time.
- the bytecode can be executed directly from ROM (i.e. flash memory) rather than
being copied into RAM. Similarly any constant objects (strings, tuples, etc)
are loaded from ROM also. This can lead to significantly more memory being
available for your application.
- on devices that do not have a filesystem, this is the only way to
load Python code.
MicroPython manifest files are Python files and can contain arbitrary Python
code. There are also a set of commands (predefined functions) which are used
to specify the Python source files to include. These commands are described
below.
During development, freezing is generally not recommended as it will
significantly slow down your development cycle, as each update will require
re-flashing the entire firmware. However, it can still be useful to
selectively freeze some rarely-changing dependencies (such as third-party
libraries).
Freezing source code
--------------------
The way to list the Python files to be be frozen into the firmware is via
a "manifest", which is a Python file that will be interpreted by the build
process. Typically you would write a manifest file as part of a board
definition, but you can also write a stand-alone manifest file and use it with
an existing board definition.
Manifest files can define dependencies on libraries from :term:`micropython-lib`
as well as Python files on the filesystem, and also on other manifest files.
Writing manifest files
----------------------
A manifest file is a Python file containing a series of function calls. See the
available functions defined below.
Any paths used in manifest files can include the following variables. These all
resolve to absolute paths.
- ``$(MPY_DIR)`` -- path to the micropython repo.
- ``$(MPY_LIB_DIR)`` -- path to the micropython-lib submodule. Prefer to use
``require()``.
- ``$(PORT_DIR)`` -- path to the current port (e.g. ``ports/stm32``)
- ``$(BOARD_DIR)`` -- path to the current board
(e.g. ``ports/stm32/boards/PYBV11``)
Custom manifest files should not live in the main MicroPython repository. You
should keep them in version control with the rest of your project.
Typically a manifest used for compiling firmware will need to include the port
manifest, which might include frozen modules that are required for the board to
function. If you just want to add additional modules to an existing board, then
include the board manifest (which will in turn include the port manifest).
Building with a custom manifest
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Your manifest can be specified on the ``make`` command line with:
.. code-block:: bash
$ make BOARD=MYBOARD FROZEN_MANIFEST=/path/to/my/project/manifest.py
This applies to all ports, including CMake-based ones (e.g. esp32, rp2), as the
Makefile wrapper that will pass this into the CMake build.
Adding a manifest to a board definition
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
If you have a custom board definition, you can make it include your custom
manifest automatically. On make-based ports (most ports), in your
``mpconfigboard.mk`` set the ``FROZEN_MANIFEST`` variable.
.. code-block:: makefile
FROZEN_MANIFEST ?= $(BOARD_DIR)/manifest.py
On CMake-based ports (e.g. esp32, rp2), instead use ``mpconfigboard.cmake``
.. code-block:: cmake
set(MICROPY_FROZEN_MANIFEST ${MICROPY_BOARD_DIR}/manifest.py)
High-level functions
~~~~~~~~~~~~~~~~~~~~
Note: The ``opt`` keyword argument can be set on the various functions, this controls
the optimisation level used by the cross-compiler.
See :func:`micropython.opt_level`.
.. function:: package(package_path, files=None, base_path=".", opt=None)
This is equivalent to copying the "package_path" directory to the device
(except as frozen code).
In the simplest case, to freeze a package "foo" in the current directory:
.. code-block:: python3
package("foo")
will recursively include all .py files in foo, and will be frozen as
``foo/**/*.py``.
If the package isn't in the same directory as the manifest file, use ``base_path``:
.. code-block:: python3
package("foo", base_path="path/to/libraries")
You can use the variables above, such as ``$(PORT_DIR)`` in ``base_path``.
To restrict to certain files in the package use ``files`` (note: paths
should be relative to the package): ``package("foo", files=["bar/baz.py"])``.
.. function:: module(module_path, base_path=".", opt=None)
Include a single Python file as a module.
If the file is in the current directory:
.. code-block:: python3
module("foo.py")
Otherwise use base_path to locate the file:
.. code-block:: python3
module("foo.py", base_path="src/drivers")
You can use the variables above, such as ``$(PORT_DIR)`` in ``base_path``.
.. function:: require(name, unix_ffi=False)
Require a package by name (and its dependencies) from :term:`micropython-lib`.
Optionally specify unix_ffi=True to use a module from the unix-ffi directory.
.. function:: include(manifest_path)
Include another manifest.
Typically a manifest used for compiling firmware will need to include the
port manifest, which might include frozen modules that are required for
the board to function.
The *manifest* argument can be a string (filename) or an iterable of
strings.
Relative paths are resolved with respect to the current manifest file.
If the path is to a directory, then it implicitly includes the
manifest.py file inside that directory.
You can use the variables above, such as ``$(PORT_DIR)`` in ``manifest_path``.
.. function:: metadata(description=None, version=None, license=None, author=None)
Define metadata for this manifest file. This is useful for manifests for
micropython-lib packages.
Low-level functions
~~~~~~~~~~~~~~~~~~~
These functions are documented for completeness, but with the exception of
``freeze_as_str`` all functionality can be accessed via the high-level functions.
.. function:: freeze(path, script=None, opt=0)
@@ -42,9 +184,7 @@ Freezing source code
module will start after *path*, i.e. *path* is excluded from the module
name.
If *path* is relative, it is resolved to the current ``manifest.py``. Use
``$(MPY_DIR)``, ``$(MPY_LIB_DIR)``, ``$(PORT_DIR)``, ``$(BOARD_DIR)`` if you
need to access specific paths.
If *path* is relative, it is resolved to the current ``manifest.py``.
If *script* is None, all files in *path* will be frozen.
@@ -75,71 +215,48 @@ Freezing source code
Freeze the input, which must be ``.mpy`` files that are frozen directly.
See ``freeze()`` for further details on the arguments.
Including other manifest files
------------------------------
.. function:: include(manifest, **kwargs)
Include another manifest.
The *manifest* argument can be a string (filename) or an iterable of
strings.
Relative paths are resolved with respect to the current manifest file.
Optional *kwargs* can be provided which will be available to the included
script via the *options* variable.
For example:
.. code-block:: python3
include("path.py", extra_features=True)
then in path.py:
.. code-block:: python3
options.defaults(standard_features=True)
# freeze minimal modules.
if options.standard_features:
# freeze standard modules.
if options.extra_features:
# freeze extra modules.
Examples
--------
To freeze a single file which is available as ``import mydriver``, use:
To freeze a single file from the current directory which will be available as
``import mydriver``, use:
.. code-block:: python3
freeze(".", "mydriver.py")
module("mydriver.py")
To freeze a set of files which are available as ``import test1`` and
``import test2``, and which are compiled with optimisation level 3, use:
To freeze a directory of files in a subdirectory "mydriver" of the current
directory which will be available as ``import mydriver``, use:
.. code-block:: python3
freeze("/path/to/tests", ("test1.py", "test2.py"), opt=3)
package("mydriver")
To freeze a module which can be imported as ``import mymodule``, use:
To freeze the "hmac" library from :term:`micropython-lib`, use:
.. code-block:: python3
freeze(
"../relative/path",
(
"mymodule/__init__.py",
"mymodule/core.py",
"mymodule/extra.py",
),
)
require("hmac")
To include a manifest from the MicroPython repository, use:
A more complete example of a custom ``manifest.py`` file for the ``PYBD_SF2``
board is:
.. code-block:: python3
include("$(MPY_DIR)/extmod/uasyncio/manifest.py")
# Include the board's default manifest.
include("$(BOARD_DIR)/manifest.py")
# Add a custom driver
module("mydriver.py")
# Add aiorepl from micropython-lib
require("aiorepl")
Then the board can be compiled with
.. code-block:: bash
$ cd ports/stm32
$ make BOARD=PYBD_SF2 FROZEN_MANIFEST=~/src/myproject/manifest.py
Note that most boards do not have their own ``manifest.py``, rather they use the
port one directly, in which case your manifest should just
``include("$(PORT_DIR)/boards/manifest.py")`` instead.

463
docs/reference/packages.rst
View File

@@ -1,314 +1,153 @@
.. _packages:
Distribution packages, package management, and deploying applications
=====================================================================
Just as the "big" Python, MicroPython supports creation of "third party"
packages, distributing them, and easily installing them in each user's
environment. This chapter discusses how these actions are achieved.
Some familiarity with Python packaging is recommended.
Overview
--------
Steps below represent a high-level workflow when creating and consuming
packages:
1. Python modules and packages are turned into distribution package
archives, and published at the Python Package Index (PyPI).
2. :term:`upip` package manager can be used to install a distribution package
on a :term:`MicroPython port` with networking capabilities (for example,
on the Unix port).
3. For ports without networking capabilities, an "installation image"
can be prepared on the Unix port, and transferred to a device by
suitable means.
4. For low-memory ports, the installation image can be frozen as the
bytecode into MicroPython executable, thus minimizing the memory
storage overheads.
The sections below describe this process in details.
Distribution packages
---------------------
Python modules and packages can be packaged into archives suitable for
transfer between systems, storing at the well-known location (PyPI),
and downloading on demand for deployment. These archives are known as
*distribution packages* (to differentiate them from Python packages
(means to organize Python source code)).
The MicroPython distribution package format is a well-known tar.gz
format, with some adaptations however. The Gzip compressor, used as
an external wrapper for TAR archives, by default uses 32KB dictionary
size, which means that to uncompress a compressed stream, 32KB of
contiguous memory needs to be allocated. This requirement may be not
satisfiable on low-memory devices, which may have total memory available
less than that amount, and even if not, a contiguous block like that
may be hard to allocate due to memory fragmentation. To accommodate
these constraints, MicroPython distribution packages use Gzip compression
with the dictionary size of 4K, which should be a suitable compromise
with still achieving some compression while being able to uncompressed
even by the smallest devices.
Besides the small compression dictionary size, MicroPython distribution
packages also have other optimizations, like removing any files from
the archive which aren't used by the installation process. In particular,
:term:`upip` package manager doesn't execute ``setup.py`` during installation
(see below), and thus that file is not included in the archive.
At the same time, these optimizations make MicroPython distribution
packages not compatible with :term:`CPython`'s package manager, ``pip``.
This isn't considered a big problem, because:
1. Packages can be installed with :term:`upip`, and then can be used with
CPython (if they are compatible with it).
2. In the other direction, majority of CPython packages would be
incompatible with MicroPython by various reasons, first of all,
the reliance on features not implemented by MicroPython.
Summing up, the MicroPython distribution package archives are highly
optimized for MicroPython's target environments, which are highly
resource constrained devices.
``upip`` package manager
------------------------
MicroPython distribution packages are intended to be installed using
the :term:`upip` package manager. :term:`upip` is a Python application which is
usually distributed (as frozen bytecode) with network-enabled
:term:`MicroPython ports <MicroPython port>`. At the very least,
:term:`upip` is available in the :term:`MicroPython Unix port`.
On any :term:`MicroPython port` providing :term:`upip`, it can be accessed as
following::
import upip
upip.help()
upip.install(package_or_package_list, [path])
Where *package_or_package_list* is the name of a distribution
package to install, or a list of such names to install multiple
packages. Optional *path* parameter specifies filesystem
location to install under and defaults to the standard library
location (see below).
An example of installing a specific package and then using it::
>>> import upip
>>> upip.install("micropython-pystone_lowmem")
[...]
>>> import pystone_lowmem
>>> pystone_lowmem.main()
Note that the name of Python package and the name of distribution
package for it in general don't have to match, and oftentimes they
don't. This is because PyPI provides a central package repository
for all different Python implementations and versions, and thus
distribution package names may need to be namespaced for a particular
implementation. For example, all packages from `micropython-lib`
follow this naming convention: for a Python module or package named
``foo``, the distribution package name is ``micropython-foo``.
For the ports which run MicroPython executable from the OS command
prompts (like the Unix port), `upip` can be (and indeed, usually is)
run from the command line instead of MicroPython's own REPL. The
commands which corresponds to the example above are::
micropython -m upip -h
micropython -m upip install [-p <path>] <packages>...
micropython -m upip install micropython-pystone_lowmem
[TODO: Describe installation path.]
Cross-installing packages
-------------------------
For :term:`MicroPython ports <MicroPython port>` without native networking
capabilities, the recommend process is "cross-installing" them into a
"directory image" using the :term:`MicroPython Unix port`, and then
transferring this image to a device by suitable means.
Installing to a directory image involves using ``-p`` switch to :term:`upip`::
micropython -m upip install -p install_dir micropython-pystone_lowmem
After this command, the package content (and contents of every dependency
packages) will be available in the ``install_dir/`` subdirectory. You
would need to transfer contents of this directory (without the
``install_dir/`` prefix) to the device, at the suitable location, where
it can be found by the Python ``import`` statement (see discussion of
the :term:`upip` installation path above).
Cross-installing packages with freezing
---------------------------------------
For the low-memory :term:`MicroPython ports <MicroPython port>`, the process
described in the previous section does not provide the most efficient
resource usage,because the packages are installed in the source form,
so need to be compiled to the bytecome on each import. This compilation
requires RAM, and the resulting bytecode is also stored in RAM, reducing
its amount available for storing application data. Moreover, the process
above requires presence of the filesystem on a device, and the most
resource-constrained devices may not even have it.
The bytecode freezing is a process which resolves all the issues
mentioned above:
* The source code is pre-compiled into bytecode and store as such.
* The bytecode is stored in ROM, not RAM.
* Filesystem is not required for frozen packages.
Using frozen bytecode requires building the executable (firmware)
for a given :term:`MicroPython port` from the C source code. Consequently,
the process is:
1. Follow the instructions for a particular port on setting up a
toolchain and building the port. For example, for ESP8266 port,
study instructions in ``ports/esp8266/README.md`` and follow them.
Make sure you can build the port and deploy the resulting
executable/firmware successfully before proceeding to the next steps.
2. Build :term:`MicroPython Unix port` and make sure it is in your PATH and
you can execute ``micropython``.
3. Change to port's directory (e.g. ``ports/esp8266/`` for ESP8266).
4. Run ``make clean-frozen``. This step cleans up any previous
modules which were installed for freezing (consequently, you need
to skip this step to add additional modules, instead of starting
from scratch).
5. Run ``micropython -m upip install -p modules <packages>...`` to
install packages you want to freeze.
6. Run ``make clean``.
7. Run ``make``.
After this, you should have the executable/firmware with modules as
the bytecode inside, which you can deploy the usual way.
Few notes:
1. Step 5 in the sequence above assumes that the distribution package
is available from PyPI. If that is not the case, you would need
to copy Python source files manually to ``modules/`` subdirectory
of the port directory. (Note that upip does not support
installing from e.g. version control repositories).
2. The firmware for baremetal devices usually has size restrictions,
so adding too many frozen modules may overflow it. Usually, you
would get a linking error if this happens. However, in some cases,
an image may be produced, which is not runnable on a device. Such
cases are in general bugs, and should be reported and further
investigated. If you face such a situation, as an initial step,
you may want to decrease the amount of frozen modules included.
Creating distribution packages
------------------------------
Distribution packages for MicroPython are created in the same manner
as for CPython or any other Python implementation, see references at
the end of chapter. Setuptools (instead of distutils) should be used,
because distutils do not support dependencies and other features. "Source
distribution" (``sdist``) format is used for packaging. The post-processing
discussed above, (and pre-processing discussed in the following section)
is achieved by using custom ``sdist`` command for setuptools. Thus, packaging
steps remain the same as for the standard setuptools, the user just
needs to override ``sdist`` command implementation by passing the
appropriate argument to ``setup()`` call::
from setuptools import setup
import sdist_upip
setup(
...,
cmdclass={'sdist': sdist_upip.sdist}
)
The sdist_upip.py module as referenced above can be found in
`micropython-lib`:
https://github.com/micropython/micropython-lib/blob/master/sdist_upip.py
Application resources
---------------------
A complete application, besides the source code, oftentimes also consists
of data files, e.g. web page templates, game images, etc. It's clear how
to deal with those when application is installed manually - you just put
those data files in the filesystem at some location and use the normal
file access functions.
The situation is different when deploying applications from packages - this
is more advanced, streamlined and flexible way, but also requires more
advanced approach to accessing data files. This approach is treating
the data files as "resources", and abstracting away access to them.
Python supports resource access using its "setuptools" library, using
``pkg_resources`` module. MicroPython, following its usual approach,
implements subset of the functionality of that module, specifically
``pkg_resources.resource_stream(package, resource)`` function.
The idea is that an application calls this function, passing a
resource identifier, which is a relative path to data file within
the specified package (usually top-level application package). It
returns a stream object which can be used to access resource contents.
Thus, the ``resource_stream()`` emulates interface of the standard
`open()` function.
Implementation-wise, ``resource_stream()`` uses file operations
underlyingly, if distribution package is install in the filesystem.
However, it also supports functioning without the underlying filesystem,
e.g. if the package is frozen as the bytecode. This however requires
an extra intermediate step when packaging application - creation of
"Python resource module".
The idea of this module is to convert binary data to a Python bytes
object, and put it into the dictionary, indexed by the resource name.
This conversion is done automatically using overridden ``sdist`` command
described in the previous section.
Let's trace the complete process using the following example. Suppose
your application has the following structure::
my_app/
__main__.py
utils.py
data/
page.html
image.png
``__main__.py`` and ``utils.py`` should access resources using the
following calls::
import pkg_resources
pkg_resources.resource_stream(__name__, "data/page.html")
pkg_resources.resource_stream(__name__, "data/image.png")
You can develop and debug using the :term:`MicroPython Unix port` as usual.
When time comes to make a distribution package out of it, just use
overridden "sdist" command from sdist_upip.py module as described in
the previous section.
This will create a Python resource module named ``R.py``, based on the
files declared in ``MANIFEST`` or ``MANIFEST.in`` files (any non-``.py``
file will be considered a resource and added to ``R.py``) - before
proceeding with the normal packaging steps.
Prepared like this, your application will work both when deployed to
filesystem and as frozen bytecode.
If you would like to debug ``R.py`` creation, you can run::
python3 setup.py sdist --manifest-only
Alternatively, you can use tools/mpy_bin2res.py script from the
MicroPython distribution, in which can you will need to pass paths
to all resource files::
mpy_bin2res.py data/page.html data/image.png
References
----------
* Python Packaging User Guide: https://packaging.python.org/
* Setuptools documentation: https://setuptools.readthedocs.io/
* Distutils documentation: https://docs.python.org/3/library/distutils.html
Package management
==================
Installing packages with ``mip``
--------------------------------
Network-capable boards include the ``mip`` module, which can install packages
from :term:`micropython-lib` and from third-party sites (including GitHub).
``mip`` ("mip installs packages") is similar in concept to Python's ``pip`` tool,
however it does not use the PyPI index, rather it uses :term:`micropython-lib`
as its index by default. ``mip`` will automatically fetch compiled
:term:`.mpy file` when downloading from micropython-lib.
The most common way to use ``mip`` is from the REPL::
>>> import mip
>>> mip.install("pkgname") # Installs the latest version of "pkgname" (and dependencies)
>>> mip.install("pkgname", version="x.y") # Installs version x.y of "pkgname"
>>> mip.install("pkgname", mpy=False) # Installs the source version (i.e. .py rather than .mpy files)
``mip`` will detect an appropriate location on the filesystem by searching
``sys.path`` for the first entry ending in ``/lib``. You can override the
destination using ``target``, but note that this path must be in ``sys.path`` to be
able to subsequently import it.::
>>> mip.install("pkgname", target="third-party")
>>> sys.path.append("third-party")
As well as downloading packages from the micropython-lib index, ``mip`` can also
install third-party libraries. The simplest way is to download a file directly::
>>> mip.install("http://example.com/x/y/foo.py")
>>> mip.install("http://example.com/x/y/foo.mpy")
When installing a file directly, the ``target`` argument is still supported to set
the destination path, but ``mpy`` and ``version`` are ignored.
The URL can also start with ``github:`` as a simple way of pointing to content
hosted on GitHub::
>>> mip.install("github:org/repo/path/foo.py") # Uses default branch
>>> mip.install("github:org/repo/path/foo.py", version="branch-or-tag") # Optionally specify the branch or tag
More sophisticated packages (i.e. with more than one file, or with dependencies)
can be downloaded by specifying the path to their ``package.json``.
>>> mip.install("http://example.com/x/package.json")
>>> mip.install("github:org/user/path/package.json")
If no json file is specified, then "package.json" is implicitly added::
>>> mip.install("http://example.com/x/")
>>> mip.install("github:org/repo")
>>> mip.install("github:org/repo", version="branch-or-tag")
Using ``mip`` on the Unix port
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
On the Unix port, ``mip`` can be used at the REPL as above, and also by using ``-m``::
$ ./micropython -m mip install pkgname-or-url
$ ./micropython -m mip install pkgname-or-url@version
The ``--target=path``, ``--no-mpy``, and ``--index`` arguments can be set::
$ ./micropython -m mip install --target=third-party pkgname
$ ./micropython -m mip install --no-mpy pkgname
$ ./micropython -m mip install --index https://host/pi pkgname
Installing packages with ``mpremote``
-------------------------------------
The :term:`mpremote` tool also includes the same functionality as ``mip`` and
can be used from a host PC to install packages to a locally connected device
(e.g. via USB or UART)::
$ mpremote install pkgname
$ mpremote install pkgname@x.y
$ mpremote install http://example.com/x/y/foo.py
$ mpremote install github:org/repo
$ mpremote install github:org/repo@branch-or-tag
The ``--target=path``, ``--no-mpy``, and ``--index`` arguments can be set::
$ mpremote install --target=/flash/third-party pkgname
$ mpremote install --no-mpy pkgname
$ mpremote install --index https://host/pi pkgname
Installing packages manually
----------------------------
Packages can also be installed (in either .py or .mpy form) by manually copying
the files to the device. Depending on the board this might be via USB Mass Storage,
the :term:`mpremote` tool (e.g. ``mpremote fs cp path/to/package.py :package.py``),
:term:`webrepl`, etc.
Writing & publishing packages
-----------------------------
Publishing to :term:`micropython-lib` is the easiest way to make your package
broadly accessible to MicroPython users, and automatically available via
``mip`` and ``mpremote`` and compiled to bytecode. See
https://github.com/micropython/micropython-lib for more information.
To write a "self-hosted" package that can be downloaded by ``mip`` or
``mpremote``, you need a static webserver (or GitHub) to host either a
single .py file, or a package.json file alongside your .py files.
A typical package.json for an example ``mlx90640`` library looks like::
{
"urls": [
["mlx90640/__init__.py", "github:org/micropython-mlx90640/mlx90640/__init__.py"],
["mlx90640/utils.py", "github:org/micropython-mlx90640/mlx90640/utils.py"]
],
"deps": [
["collections-defaultdict", "latest"],
["os-path", "latest"]
],
"version": "0.2"
}
This includes two files, hosted at a GitHub repo named
``org/micropython-mlx90640``, which install into the ``mlx90640`` directory on
the device. It depends on ``collections-defaultdict`` and ``os-path`` which will
be installed automatically.
Freezing packages
-----------------
When a Python module or package is imported from the device filesystem, it is
compiled into :term:`bytecode` in RAM, ready to be executed by the VM. For
a :term:`.mpy file`, this conversion has been done already, but the bytecode
still ends up in RAM.
For low-memory devices, or for large applications, it can be advantageous to
instead run the bytecode from ROM (i.e. flash memory). This can be done
by "freezing" the bytecode into the MicroPython firmware, which is then flashed
to the device. The runtime performance is the same (although importing is
faster), but it can free up significant amounts of RAM for your program to
use.
The downside of this approach is that it's much slower to develop, because you
have to flash the firmware each time, but it can be still useful to freeze
dependencies that don't change often.
Freezing is done by writing a manifest file and using it in the build, often as
part of a custom board definition. See the :ref:`manifest` guide for more
information.