Release 2.2.0

Bumps [jinja2](https://github.com/pallets/jinja) from 3.1.5 to 3.1.6.
- [Release notes](https://github.com/pallets/jinja/releases)
- [Changelog](https://github.com/pallets/jinja/blob/main/CHANGES.rst)
- [Commits](https://github.com/pallets/jinja/compare/3.1.5...3.1.6)

---
updated-dependencies:
- dependency-name: jinja2
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

CHANGES.md

@@ -1,5 +1,17 @@
 # Microdot change log
 
+**Release 2.2.0** - 2025-03-22
+
+- Support for multipart/form-data requests [#287](https://github.com/miguelgrinberg/microdot/issues/287) ([commit](https://github.com/miguelgrinberg/microdot/commit/11a91a60350518e426b557fae8dffe75912f8823))
+- Support custom path components in URLs ([commit](https://github.com/miguelgrinberg/microdot/commit/c92b5ae28222af5a1094f5d2f70a45d4d17653d5))
+- Expose the Jinja environment as Template.jinja_env ([commit](https://github.com/miguelgrinberg/microdot/commit/953dd9432122defe943f0637bbe7e01f2fc7743f))
+- Delay route compilation to allow late register_type calls ([commit](https://github.com/miguelgrinberg/microdot/commit/aa76e6378b37faab52008a8aab8db75f81b29323))
+- urldecoding should always be done in bytes ([commit](https://github.com/miguelgrinberg/microdot/commit/d203df75fef32c7cc0fe7cc6525e77522b37a289))
+- Simplified urldecode logic ([commit](https://github.com/miguelgrinberg/microdot/commit/3bc31f10b2b2d4460c62366013278d87665f0f97))
+- Additional urldecode tests ([commit](https://github.com/miguelgrinberg/microdot/commit/99f65c0198590c0dfb402c24685b6f8dfba1935d))
+- Update micropython version used in tests to 1.24.1 ([commit](https://github.com/miguelgrinberg/microdot/commit/4cc2e95338a7de3b03742389004147ee21285621))
+- Documentation improvements ([commit](https://github.com/miguelgrinberg/microdot/commit/c6b99b6d8117d4e40e16d5b953dbf4deb023d24d))
+
 **Release 2.1.0** - 2025-02-04
 
 - User login support ([commit](https://github.com/miguelgrinberg/microdot/commit/d807011ad006e53e70c4594d7eac04d03bb08681))

README.md

@@ -43,8 +43,8 @@ describes the backwards incompatible changes that were made.
 The following features are planned for future releases of Microdot, both for
 MicroPython and CPython:
 
-- Support for forms encoded in `multipart/form-data` format
-- Authentication support, similar to [Flask-Login](https://github.com/maxcountryman/flask-login) for Flask
+- Authentication support, similar to [Flask-Login](https://github.com/maxcountryman/flask-login) for Flask (**Added in version 2.1**)
+- Support for forms encoded in `multipart/form-data` format (**Added in version 2.2**)
 - OpenAPI integration, similar to [APIFairy](https://github.com/miguelgrinberg/apifairy) for Flask
 
 In addition to the above, the following extensions are also under consideration,
@@ -53,4 +53,4 @@ but only for CPython:
 - Database integration through [SQLAlchemy](https://github.com/sqlalchemy/sqlalchemy)
 - Socket.IO support through [python-socketio](https://github.com/miguelgrinberg/python-socketio)
 
-Do you have other ideas to propose? Let's [discuss them](https://github.com/miguelgrinberg/microdot/discussions/new?category=ideas)!
+Do you have other ideas to propose? Let's [discuss them](https://github.com/:miguelgrinberg/microdot/discussions/new?category=ideas)!

docs/api.rst

@@ -14,6 +14,12 @@ Core API
    :members:
 
 
+Multipart Forms
+---------------
+
+.. automodule:: microdot.multipart
+   :members:
+
 WebSocket
 ---------
 

docs/extensions.rst

@@ -5,8 +5,82 @@ Microdot is a highly extensible web application framework. The extensions
 described in this section are maintained as part of the Microdot project in
 the same source code repository.
 
+Multipart Forms
+~~~~~~~~~~~~~~~
+
+.. list-table::
+   :align: left
+
+   * - Compatibility
+     - | CPython & MicroPython
+
+   * - Required Microdot source files
+     -  | `multipart.py <https://github.com/miguelgrinberg/microdot/tree/main/src/microdot/multipart.py>`_
+        | `helpers.py <https://github.com/miguelgrinberg/microdot/tree/main/src/microdot/helpers.py>`_
+
+   * - Required external dependencies
+     - | None
+
+   * - Examples
+     - | `formdata.py <https://github.com/miguelgrinberg/microdot/blob/main/examples/uploads/formdata.py>`_
+
+The multipart extension handles multipart forms, including those that have file
+uploads.
+
+The :func:`with_form_data <microdot.multipart.with_form_data>` decorator
+provides the simplest way to work with these forms. With this decorator added
+to the route, whenever the client sends a multipart request the
+:attr:`request.form <microdot.Request.form>` and
+:attr:`request.files <microdot.Request.files>` properties are populated with
+the submitted data. For form fields the field values are always strings. For
+files, they are instances of the
+:class:`FileUpload <microdot.multipart.FileUpload>` class.
+
+Example::
+
+    from microdot.multipart import with_form_data
+
+    @app.post('/upload')
+    @with_form_data
+    async def upload(request):
+        print('form fields:', request.form)
+        print('files:', request.files)
+
+One disadvantage of the ``@with_form_data`` decorator is that it has to copy
+any uploaded files to memory or temporary disk files, depending on their size.
+The :attr:`FileUpload.max_memory_size <microdot.multipart.FileUpload.max_memory_size>`
+attribute can be used to control the cutoff size above which a file upload
+is transferred to a temporary file.
+
+A more performant alternative to the ``@with_form_data`` decorator is the
+:class:`FormDataIter <microdot.multipart.FormDataIter>` class, which iterates
+over the form fields sequentially, giving the application the option to parse
+the form fields on the fly and decide what to copy and what to discard. When
+using ``FormDataIter`` the ``request.form`` and ``request.files`` attributes
+are not used.
+
+Example::
+
+
+    from microdot.multipart import FormDataIter
+
+    @app.post('/upload')
+    async def upload(request):
+        async for name, value in FormDataIter(request):
+            print(name, value)
+
+For fields that contain an uploaded file, the ``value`` returned by the
+iterator is the same ``FileUpload`` instance. The application can choose to
+save the file with the :meth:`save() <microdot.multipart.FileUpload.save>`
+method, or read it with the :meth:`read() <microdot.multipart.FileUpload.read>`
+method, optionally passing a size to read it in chunks. The
+:meth:`copy() <microdot.multipart.FileUpload.copy>` method is also available to
+apply the copying logic used by the ``@with_form_data`` decorator, which is
+inefficient but allows the file to be set aside to be processed later, after
+the remaining form fields.
+
 WebSocket
-~~~~~~~~-
+~~~~~~~~~
 
 .. list-table::
    :align: left
@@ -16,6 +90,7 @@ WebSocket
 
    * - Required Microdot source files
      -  | `websocket.py <https://github.com/miguelgrinberg/microdot/tree/main/src/microdot/websocket.py>`_
+        | `helpers.py <https://github.com/miguelgrinberg/microdot/tree/main/src/microdot/helpers.py>`_
 
    * - Required external dependencies
      - | None
@@ -32,12 +107,14 @@ messages respectively.
 
 Example::
 
-        @app.route('/echo')
-        @with_websocket
-        async def echo(request, ws):
-            while True:
-                message = await ws.receive()
-                await ws.send(message)
+    from microdot.websocket import with_websocket
+
+    @app.route('/echo')
+    @with_websocket
+    async def echo(request, ws):
+        while True:
+            message = await ws.receive()
+            await ws.send(message)
 
 Server-Sent Events
 ~~~~~~~~~~~~~~~~~~
@@ -50,6 +127,7 @@ Server-Sent Events
 
    * - Required Microdot source files
      -  | `sse.py <https://github.com/miguelgrinberg/microdot/tree/main/src/microdot/sse.py>`_
+        | `helpers.py <https://github.com/miguelgrinberg/microdot/tree/main/src/microdot/helpers.py>`_
 
    * - Required external dependencies
      - | None
@@ -65,6 +143,8 @@ asynchronous method to send an event to the client.
 
 Example::
 
+    from microdot.sse import with_sse
+
     @app.route('/events')
     @with_sse
     async def events(request, sse):
@@ -213,6 +293,7 @@ Secure User Sessions
 
    * - Required Microdot source files
      - | `session.py <https://github.com/miguelgrinberg/microdot/tree/main/src/microdot/session.py>`_
+       | `helpers.py <https://github.com/miguelgrinberg/microdot/tree/main/src/microdot/helpers.py>`_
 
    * - Required external dependencies
      - | CPython: `PyJWT <https://pyjwt.readthedocs.io/>`_
@@ -369,6 +450,7 @@ User Logins
    * - Required Microdot source files
      - | `login.py <https://github.com/miguelgrinberg/microdot/tree/main/src/microdot/auth.py>`_
        | `session.py <https://github.com/miguelgrinberg/microdot/tree/main/src/microdot/session.py>`_
+       | `helpers.py <https://github.com/miguelgrinberg/microdot/tree/main/src/microdot/helpers.py>`_
    * - Required external dependencies
      - | CPython: `PyJWT <https://pyjwt.readthedocs.io/>`_
        | MicroPython: `jwt.py <https://github.com/micropython/micropython-lib/blob/master/python-ecosys/pyjwt/jwt.py>`_,

docs/freezing.rst

@@ -1,5 +1,8 @@
-Cross-Compiling and Freezing Microdot (MicroPython Only)
---------------------------------------------------------
+Cross-Compiling and Freezing Microdot
+-------------------------------------
+
+.. note::
+   This section only applies when using Microdot on MicroPython.
 
 Microdot is a fairly small framework, so its size is not something you need to
 be concerned about unless you are working with MicroPython on hardware with a
@@ -36,7 +39,7 @@ Cross-Compiling
 
 An issue that is common with low-end microcontroller boards is that they do not
 have enough RAM for the MicroPython compiler to compile the source files, but
-once the code is compiled they are able to run it without problems.
+once the code is compiled they are able to run it just fine.
 
 To address this, MicroPython allows you to cross-compile source files on your
 desktop or laptop computer and then upload their compiled versions to the
@@ -82,8 +85,8 @@ imported directly from the device's ROM, leaving more RAM available for
 application use.
 
 The process to create a custom firmware is unfortunately non-trivial and
-different depending on the device, so you will need to consult the MicroPython
-documentation that applies to your device to learn how to do this.
+different for each microcontroller platform, so you will need to consult the
+MicroPython documentation that applies to your device to learn how to do this.
 
 The part of the process that is common to all devices is the creation of a
 `manifest file <https://docs.micropython.org/en/latest/reference/manifest.html>`_

docs/intro.rst

@@ -329,15 +329,52 @@ URL::
     async def get_test(request, path):
         return 'Test: ' + path
 
-For the most control, the ``re`` type allows the application to provide a
-custom regular expression for the dynamic component. The next example defines
-a route that only matches usernames that begin with an upper or lower case
-letter, followed by a sequence of letters or numbers::
+The ``re`` type allows the application to provide a custom regular expression
+for the dynamic component. The next example defines a route that only matches
+usernames that begin with an upper or lower case letter, followed by a sequence
+of letters or numbers::
 
     @app.get('/users/<re:[a-zA-Z][a-zA-Z0-9]*:username>')
     async def get_user(request, username):
         return 'User: ' + username
 
+The ``re`` type returns the URL component as a string, which sometimes may not
+be the most convenient. To convert a path component to something more
+meaningful than a string, the application can register a custom URL component
+type and provide a parser function that performs the conversion. In the
+following example, a ``hex`` custom type is registered to automatically
+convert hex numbers given in the path to numbers::
+
+    from microdot import URLPattern
+
+    URLPattern.register_type('hex', parser=lambda value: int(value, 16))
+
+    @app.get('/users/<hex:user_id>')
+    async def get_user(request, user_id):
+        user = get_user_by_id(user_id)
+        # ...
+
+In addition to the parser, the custom URL component can include a pattern,
+given as a regular expression. When a pattern is provided, the URL component
+will only match if the regular expression matches the value passed in the URL.
+The ``hex`` example above can be expanded with a pattern as follows::
+
+    URLPattern.register_type('hex', pattern='[0-9a-fA-F]+',
+                             parser=lambda value: int(value, 16))
+
+In cases where a pattern isn't provided, or when the pattern is unable to
+filter out all invalid values, the parser function can return ``None`` to
+indicate a failed match. The next example shows how the parser for the ``hex``
+type can be expanded to do that::
+
+    def hex_parser(value):
+        try:
+            return int(value, 16)
+        except ValueError:
+            return None
+
+    URLPattern.register_type('hex', parser=hex_parser)
+
 .. note::
    Dynamic path components are passed to route functions as keyword arguments,
    so the names of the function arguments must match the names declared in the
@@ -895,18 +932,36 @@ Another option is to create a response object directly in the route function::
 Concurrency
 ~~~~~~~~~~~
 
-Microdot implements concurrency through the ``asyncio`` package. Applications
-must ensure their handlers do not block, as this will prevent other concurrent
-requests from being handled.
+Microdot implements concurrency through the ``asyncio`` package, which means
+that applications must be careful to prevent blocking in their handlers.
 
-When running under CPython, ``async def`` handler functions run as native 
-asyncio tasks, while ``def`` handler functions are executed in a
-`thread executor <https://docs.python.org/3/library/asyncio-eventloop.html#asyncio.loop.run_in_executor>`_
-to prevent them from blocking the asynchronous loop.
+"async def" handlers
+^^^^^^^^^^^^^^^^^^^^
+
+The recommendation for route handlers in Microdot is to use asynchronous
+functions, declared as ``async def``. Microdot executes these handler
+functions as native asynchronous tasks. The standard considerations for writing
+asynchronous code apply, and in particular blocking calls should be avoided to
+ensure the application runs smoothly and is always responsive.
+
+"def" handlers
+^^^^^^^^^^^^^^
+
+Microdot also supports the use of synchronous route handlers, declared as
+standard ``def`` functions. These handlers are handled differently under
+CPython and MicroPython.
+
+When running on CPython, Microdot executes synchronous handlers in a
+`thread executor <https://docs.python.org/3/library/asyncio-eventloop.html#asyncio.loop.run_in_executor>`_,
+which uses a thread pool. The use of blocking or CPU intensive code in these
+handlers does not have such a negative effect on the application, because
+handlers do not run on the same thread as the asynchronous loop. On the other
+hand, the application will be affected by threading issues such as those caused
+by the Global Interpreter Lock.
 
 Under MicroPython the situation is different. Most microcontroller boards
-implementing MicroPython do not have threading support or executors, so ``def``
-handler functions in this platform can only run in the main and only thread.
-These functions will block the asynchronous loop when they take too long to
-complete so ``async def`` handlers properly written to allow other handlers to
-run in parallel should be preferred.
+do not have or have very limited threading support, so Microdot executes
+synchronous handlers in the main and often only thread available. This means
+that these functions will block the asynchronous loop when they take too long
+to complete. The use of properly written asynchronous handlers should be
+preferred.

examples/benchmark/requirements.txt

@@ -57,7 +57,7 @@ itsdangerous==2.1.2
     # via
     #   flask
     #   quart
-jinja2==3.1.4
+jinja2==3.1.6
     # via
     #   flask
     #   quart
@@ -82,7 +82,7 @@ pydantic-core==2.14.5
     # via pydantic
 pyproject-hooks==1.0.0
     # via build
-quart==0.19.7
+quart==0.20.0
     # via -r requirements.in
 requests==2.32.0
     # via -r requirements.in

examples/uploads/README.md

@@ -1 +1,4 @@
 This directory contains file upload examples.
+
+- `simple_uploads.py` demonstrates how to upload a single file.
+- `formdata.py` demonstrates how to process a form that includes file uploads.

examples/uploads/formdata.html

@@ -0,0 +1,17 @@
+<!doctype html>
+<html>
+  <head>
+    <title>Microdot Multipart Form-Data Example</title>
+    <meta charset="UTF-8">
+  </head>
+  <body>
+    <h1>Microdot Multipart Form-Data Example</h1>
+    <form method="POST" action="" enctype="multipart/form-data">
+      <p>Name: <input type="text" name="name" /></p>
+      <p>Age: <input type="text" name="age" /></p>
+      <p>Comments: <textarea name="comments" rows="4"></textarea></p>
+      <p>File: <input type="file" id="file" name="file" /></p>
+      <input type="submit" value="Submit" />
+    </form>
+  </body>
+</html>

examples/uploads/formdata.py

@@ -0,0 +1,26 @@
+from microdot import Microdot, send_file, Request
+from microdot.multipart import with_form_data
+
+app = Microdot()
+Request.max_content_length = 1024 * 1024  # 1MB (change as needed)
+
+
+@app.get('/')
+async def index(request):
+    return send_file('formdata.html')
+
+
+@app.post('/')
+@with_form_data
+async def upload(request):
+    print('Form fields:')
+    for field, value in request.form.items():
+        print(f'- {field}: {value}')
+    print('\nFile uploads:')
+    for field, value in request.files.items():
+        print(f'- {field}: {value.filename}, {await value.read()}')
+    return 'We have received your data!'
+
+
+if __name__ == '__main__':
+    app.run(debug=True)

examples/uploads/simple_uploads.py

@@ -6,7 +6,7 @@ Request.max_content_length = 1024 * 1024  # 1MB (change as needed)
 
 @app.get('/')
 async def index(request):
-    return send_file('index.html')
+    return send_file('simple_uploads.html')
 
 
 @app.post('/upload')

pyproject.toml

@@ -1,6 +1,6 @@
 [project]
 name = "microdot"
-version = "2.1.0"
+version = "2.2.0"
 authors = [
     { name = "Miguel Grinberg", email = "miguel.grinberg@gmail.com" },
 ]

src/microdot/__init__.py

@@ -1,2 +1,2 @@
 from microdot.microdot import Microdot, Request, Response, abort, redirect, \
-    send_file  # noqa: F401
+    send_file, URLPattern, AsyncBytesIO, iscoroutine  # noqa: F401

src/microdot/jinja.py

@@ -1,19 +1,27 @@
 from jinja2 import Environment, FileSystemLoader, select_autoescape
 
-_jinja_env = None
-
 
 class Template:
     """A template object.
 
     :param template: The filename of the template to render, relative to the
                      configured template directory.
+    :param kwargs: any additional options to be passed to the Jinja
+                   environment's ``get_template()`` method.
     """
+    #: The Jinja environment. The ``initialize()`` method must be called before
+    #: this attribute is accessed.
+    jinja_env = None
+
     @classmethod
     def initialize(cls, template_dir='templates', enable_async=False,
                    **kwargs):
         """Initialize the templating subsystem.
 
+        This method is automatically invoked when the first template is
+        created. The application can call it explicitly if custom options need
+        to be provided.
+
         :param template_dir: the directory where templates are stored. This
                              argument is optional. The default is to load
                              templates from a *templates* subdirectory.
@@ -23,20 +31,19 @@ class Template:
         :param kwargs: any additional options to be passed to Jinja's
                        ``Environment`` class.
         """
-        global _jinja_env
-        _jinja_env = Environment(
+        cls.jinja_env = Environment(
             loader=FileSystemLoader(template_dir),
             autoescape=select_autoescape(),
             enable_async=enable_async,
             **kwargs
         )
 
-    def __init__(self, template):
-        if _jinja_env is None:  # pragma: no cover
+    def __init__(self, template, **kwargs):
+        if self.jinja_env is None:  # pragma: no cover
             self.initialize()
-        #: The name of the template
+        #: The name of the template.
         self.name = template
-        self.template = _jinja_env.get_template(template)
+        self.template = self.jinja_env.get_template(template, **kwargs)
 
     def generate(self, *args, **kwargs):
         """Return a generator that renders the template in chunks, with the

src/microdot/microdot.py

@@ -8,6 +8,7 @@ servers for MicroPython and standard Python.
 import asyncio
 import io
 import json
+import re
 import time
 
 try:
@@ -56,23 +57,9 @@ MUTED_SOCKET_ERRORS = [
 ]
 
 
-def urldecode_str(s):
-    s = s.replace('+', ' ')
-    parts = s.split('%')
-    if len(parts) == 1:
-        return s
-    result = [parts[0]]
-    for item in parts[1:]:
-        if item == '':
-            result.append('%')
-        else:
-            code = item[:2]
-            result.append(chr(int(code, 16)))
-            result.append(item[2:])
-    return ''.join(result)
-
-
-def urldecode_bytes(s):
+def urldecode(s):
+    if isinstance(s, str):
+        s = s.encode()
     s = s.replace(b'+', b' ')
     parts = s.split(b'%')
     if len(parts) == 1:
@@ -384,6 +371,7 @@ class Request:
         self.sock = sock
         self._json = None
         self._form = None
+        self._files = None
         self.after_request_handlers = []
 
     @staticmethod
@@ -440,12 +428,12 @@ class Request:
             if isinstance(urlencoded, str):
                 for kv in [pair.split('=', 1)
                            for pair in urlencoded.split('&') if pair]:
-                    data[urldecode_str(kv[0])] = urldecode_str(kv[1]) \
+                    data[urldecode(kv[0])] = urldecode(kv[1]) \
                         if len(kv) > 1 else ''
             elif isinstance(urlencoded, bytes):  # pragma: no branch
                 for kv in [pair.split(b'=', 1)
                            for pair in urlencoded.split(b'&') if pair]:
-                    data[urldecode_bytes(kv[0])] = urldecode_bytes(kv[1]) \
+                    data[urldecode(kv[0])] = urldecode(kv[1]) \
                         if len(kv) > 1 else b''
         return data
 
@@ -478,7 +466,13 @@ class Request:
     def form(self):
         """The parsed form submission body, as a
         :class:`MultiDict <microdot.MultiDict>` object, or ``None`` if the
-        request does not have a form submission."""
+        request does not have a form submission.
+
+        Forms that are URL encoded are processed by default. For multipart
+        forms to be processed, the
+        :func:`with_form_data <microdot.multipart.with_form_data>`
+        decorator must be added to the route.
+        """
         if self._form is None:
             if self.content_type is None:
                 return None
@@ -488,6 +482,17 @@ class Request:
             self._form = self._parse_urlencoded(self.body)
         return self._form
 
+    @property
+    def files(self):
+        """The files uploaded in the request as a dictionary, or ``None`` if
+        the request does not have any files.
+
+        The :func:`with_form_data <microdot.multipart.with_form_data>`
+        decorator must be added to the route that receives file uploads for
+        this property to be set.
+        """
+        return self._files
+
     def after_request(self, f):
         """Register a request-specific function to run after the request is
         handled. Request-specific after request handlers run at the very end,
@@ -805,13 +810,23 @@ class Response:
 
 
 class URLPattern():
+    segment_patterns = {
+        'string': '/([^/]+)',
+        'int': '/(-?\\d+)',
+        'path': '/(.+)',
+    }
+    segment_parsers = {
+        'int': lambda value: int(value),
+    }
+
     def __init__(self, url_pattern):
         self.url_pattern = url_pattern
         self.segments = []
         self.regex = None
+
+    def compile(self):
         pattern = ''
-        use_regex = False
-        for segment in url_pattern.lstrip('/').split('/'):
+        for segment in self.url_pattern.lstrip('/').split('/'):
             if segment and segment[0] == '<':
                 if segment[-1] != '>':
                     raise ValueError('invalid URL pattern')
@@ -822,82 +837,44 @@ class URLPattern():
                     type_ = 'string'
                     name = segment
                 parser = None
-                if type_ == 'string':
-                    parser = self._string_segment
-                    pattern += '/([^/]+)'
-                elif type_ == 'int':
-                    parser = self._int_segment
-                    pattern += '/(-?\\d+)'
-                elif type_ == 'path':
-                    use_regex = True
-                    pattern += '/(.+)'
-                elif type_.startswith('re:'):
-                    use_regex = True
+                if type_.startswith('re:'):
                     pattern += '/({pattern})'.format(pattern=type_[3:])
                 else:
-                    raise ValueError('invalid URL segment type')
+                    if type_ not in self.segment_patterns:
+                        raise ValueError('invalid URL segment type')
+                    pattern += self.segment_patterns[type_]
+                    parser = self.segment_parsers.get(type_)
                 self.segments.append({'parser': parser, 'name': name,
                                       'type': type_})
             else:
                 pattern += '/' + segment
-                self.segments.append({'parser': self._static_segment(segment)})
-        if use_regex:
-            import re
-            self.regex = re.compile('^' + pattern + '$')
+                self.segments.append({'parser': None})
+        self.regex = re.compile('^' + pattern + '$')
+        return self.regex
+
+    @classmethod
+    def register_type(cls, type_name, pattern='[^/]+', parser=None):
+        cls.segment_patterns[type_name] = '/({})'.format(pattern)
+        cls.segment_parsers[type_name] = parser
 
     def match(self, path):
         args = {}
-        if self.regex:
-            g = self.regex.match(path)
-            if not g:
-                return
-            i = 1
-            for segment in self.segments:
-                if 'name' not in segment:
-                    continue
-                value = g.group(i)
-                if segment['type'] == 'int':
-                    value = int(value)
-                args[segment['name']] = value
-                i += 1
-        else:
-            if len(path) == 0 or path[0] != '/':
-                return
-            path = path[1:]
-            args = {}
-            for segment in self.segments:
-                if path is None:
-                    return
-                arg, path = segment['parser'](path)
+        g = (self.regex or self.compile()).match(path)
+        if not g:
+            return
+        i = 1
+        for segment in self.segments:
+            if 'name' not in segment:
+                continue
+            arg = g.group(i)
+            if segment['parser']:
+                arg = self.segment_parsers[segment['type']](arg)
                 if arg is None:
                     return
-                if 'name' in segment:
-                    args[segment['name']] = arg
-            if path is not None:
-                return
+            args[segment['name']] = arg
+            i += 1
         return args
 
-    def _static_segment(self, segment):
-        def _static(value):
-            s = value.split('/', 1)
-            if s[0] == segment:
-                return '', s[1] if len(s) > 1 else None
-            return None, None
-        return _static
-
-    def _string_segment(self, value):
-        s = value.split('/', 1)
-        if len(s[0]) == 0:
-            return None, None
-        return s[0], s[1] if len(s) > 1 else None
-
-    def _int_segment(self, value):
-        s = value.split('/', 1)
-        try:
-            return int(s[0]), s[1] if len(s) > 1 else None
-        except ValueError:
-            return None, None
-
     def __repr__(self):  # pragma: no cover
         return 'URLPattern: {}'.format(self.url_pattern)
 

src/microdot/multipart.py

@@ -0,0 +1,291 @@
+import os
+from random import choice
+from microdot import abort, iscoroutine, AsyncBytesIO
+from microdot.helpers import wraps
+
+
+class FormDataIter:
+    """Asynchronous iterator that parses a ``multipart/form-data`` body and
+    returns form fields and files as they are parsed.
+
+    :param request: the request object to parse.
+
+    Example usage::
+
+        from microdot.multipart import FormDataIter
+
+        @app.post('/upload')
+        async def upload(request):
+            async for name, value in FormDataIter(request):
+                print(name, value)
+
+    The iterator returns no values when the request has a content type other
+    than ``multipart/form-data``. For a file field, the returned value is of
+    type :class:`FileUpload`, which supports the
+    :meth:`read() <FileUpload.read>` and :meth:`save() <FileUpload.save>`
+    methods. Values for regular fields are provided as strings.
+
+    The request body is read efficiently in chunks of size
+    :attr:`buffer_size <FormDataIter.buffer_size>`. On iterations in which a
+    file field is encountered, the file must be consumed before moving on to
+    the next iteration, as the internal stream stored in ``FileUpload``
+    instances is invalidated at the end of the iteration.
+    """
+    #: The size of the buffer used to read chunks of the request body.
+    buffer_size = 256
+
+    def __init__(self, request):
+        self.request = request
+        self.buffer = None
+        try:
+            mimetype, boundary = request.content_type.rsplit('; boundary=', 1)
+        except ValueError:
+            return  # not a multipart request
+        if mimetype.split(';', 1)[0] == \
+                'multipart/form-data':  # pragma: no branch
+            self.boundary = b'--' + boundary.encode()
+            self.extra_size = len(boundary) + 4
+            self.buffer = b''
+
+    def __aiter__(self):
+        return self
+
+    async def __anext__(self):
+        if self.buffer is None:
+            raise StopAsyncIteration
+
+        # make sure we have consumed the previous entry
+        while await self._read_buffer(self.buffer_size) != b'':
+            pass
+
+        # make sure we are at a boundary
+        s = self.buffer.split(self.boundary, 1)
+        if len(s) != 2 or s[0] != b'':
+            abort(400)  # pragma: no cover
+        self.buffer = s[1]
+        if self.buffer[:2] == b'--':
+            # we have reached the end
+            raise StopAsyncIteration
+        elif self.buffer[:2] != b'\r\n':
+            abort(400)  # pragma: no cover
+        self.buffer = self.buffer[2:]
+
+        # parse the headers of this part
+        name = ''
+        filename = None
+        content_type = None
+        while True:
+            await self._fill_buffer()
+            lines = self.buffer.split(b'\r\n', 1)
+            if len(lines) != 2:
+                abort(400)  # pragma: no cover
+            line, self.buffer = lines
+            if line == b'':
+                # we reached the end of the headers
+                break
+            header, value = line.decode().split(':', 1)
+            header = header.lower()
+            value = value.strip()
+            if header == 'content-disposition':
+                parts = value.split(';')
+                if len(parts) < 2 or parts[0] != 'form-data':
+                    abort(400)  # pragma: no cover
+                for part in parts[1:]:
+                    part = part.strip()
+                    if part.startswith('name="'):
+                        name = part[6:-1]
+                    elif part.startswith('filename="'):  # pragma: no branch
+                        filename = part[10:-1]
+            elif header == 'content-type':  # pragma: no branch
+                content_type = value
+
+        if filename is None:
+            # this is a regular form field, so we read the value
+            value = b''
+            while True:
+                v = await self._read_buffer(self.buffer_size)
+                value += v
+                if len(v) < self.buffer_size:  # pragma: no branch
+                    break
+            return name, value.decode()
+        return name, FileUpload(filename, content_type, self._read_buffer)
+
+    async def _fill_buffer(self):
+        self.buffer += await self.request.stream.read(
+            self.buffer_size + self.extra_size - len(self.buffer))
+
+    async def _read_buffer(self, n=-1):
+        data = b''
+        while n == -1 or len(data) < n:
+            await self._fill_buffer()
+            s = self.buffer.split(self.boundary, 1)
+            data += s[0][:n] if n != -1 else s[0]
+            self.buffer = s[0][n:] if n != -1 else b''
+            if len(s) == 2:  # pragma: no branch
+                # the end of this part is in the buffer
+                if len(self.buffer) < 2:
+                    # we have read all the way to the end of this part
+                    data = data[:-(2 - len(self.buffer))]  # remove last "\r\n"
+                self.buffer += self.boundary + s[1]
+                return data
+        return data
+
+
+class FileUpload:
+    """Class that represents an uploaded file.
+
+    :param filename: the name of the uploaded file.
+    :param content_type: the content type of the uploaded file.
+    :param read: a coroutine that reads from the uploaded file's stream.
+
+    An uploaded file can be read from the stream using the :meth:`read()`
+    method or saved to a file using the :meth:`save()` method.
+
+    Instances of this class do not normally need to be created directly.
+    """
+    #: The size at which the file is copied to a temporary file.
+    max_memory_size = 1024
+
+    def __init__(self, filename, content_type, read):
+        self.filename = filename
+        self.content_type = content_type
+        self._read = read
+        self._close = None
+
+    async def read(self, n=-1):
+        """Read up to ``n`` bytes from the uploaded file's stream.
+
+        :param n: the maximum number of bytes to read. If ``n`` is -1 or not
+                  given, the entire file is read.
+        """
+        return await self._read(n)
+
+    async def save(self, path_or_file):
+        """Save the uploaded file to the given path or file object.
+
+        :param path_or_file: the path to save the file to, or a file object
+                             to which the file is to be written.
+
+        The file is read and written in chunks of size
+        :attr:`FormDataIter.buffer_size`.
+        """
+        if isinstance(path_or_file, str):
+            f = open(path_or_file, 'wb')
+        else:
+            f = path_or_file
+        while True:
+            data = await self.read(FormDataIter.buffer_size)
+            if not data:
+                break
+            f.write(data)
+        if f != path_or_file:
+            f.close()
+
+    async def copy(self, max_memory_size=None):
+        """Copy the uploaded file to a temporary file, to allow the parsing of
+        the multipart form to continue.
+
+        :param max_memory_size: the maximum size of the file to keep in memory.
+                                If not given, then the class attribute of the
+                                same name is used.
+        """
+        max_memory_size = max_memory_size or FileUpload.max_memory_size
+        buffer = await self.read(max_memory_size)
+        if len(buffer) < max_memory_size:
+            f = AsyncBytesIO(buffer)
+            self._read = f.read
+            return self
+
+        # create a temporary file
+        while True:
+            tmpname = "".join([
+                choice('abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ')
+                for _ in range(12)
+            ])
+            try:
+                f = open(tmpname, 'x+b')
+            except OSError as e:  # pragma: no cover
+                if e.errno == 17:
+                    # EEXIST
+                    continue
+                elif e.errno == 2:
+                    # ENOENT
+                    # some MicroPython platforms do not support mode "x"
+                    f = open(tmpname, 'w+b')
+                    if f.read(1) != b'':
+                        f.close()
+                        continue
+                else:
+                    raise
+            break
+        f.write(buffer)
+        await self.save(f)
+        f.seek(0)
+
+        async def read(n=-1):
+            return f.read(n)
+
+        async def close():
+            f.close()
+            os.remove(tmpname)
+
+        self._read = read
+        self._close = close
+        return self
+
+    async def close(self):
+        """Close an open file.
+
+        This method must be called to free memory or temporary files created by
+        the ``copy()`` method.
+
+        Note that when using the ``@with_form_data`` decorator this method is
+        called automatically when the request ends.
+        """
+        if self._close:
+            await self._close()
+            self._close = None
+
+
+def with_form_data(f):
+    """Decorator that parses a ``multipart/form-data`` body and updates the
+    request object with the parsed form fields and files.
+
+    Example usage::
+
+        from microdot.multipart import with_form_data
+
+        @app.post('/upload')
+        @with_form_data
+        async def upload(request):
+            print('form fields:', request.form)
+            print('files:', request.files)
+
+    Note: this decorator calls the :meth:`FileUpload.copy()
+    <microdot.multipart.FileUpload.copy>` method on all uploaded files, so that
+    the request can be parsed in its entirety. The files are either copied to
+    memory or a temporary file, depending on their size. The temporary files
+    are automatically deleted when the request ends.
+    """
+    @wraps(f)
+    async def wrapper(request, *args, **kwargs):
+        form = {}
+        files = {}
+        async for name, value in FormDataIter(request):
+            if isinstance(value, FileUpload):
+                files[name] = await value.copy()
+            else:
+                form[name] = value
+        if form or files:
+            request._form = form
+            request._files = files
+        try:
+            ret = f(request, *args, **kwargs)
+            if iscoroutine(ret):
+                ret = await ret
+        finally:
+            if request.files:
+                for file in request.files.values():
+                    await file.close()
+        return ret
+    return wrapper

tests/__init__.py

@@ -4,6 +4,7 @@ from tests.test_request import *  # noqa: F401, F403
 from tests.test_response import *  # noqa: F401, F403
 from tests.test_urlencode import *  # noqa: F401, F403
 from tests.test_url_pattern import *  # noqa: F401, F403
+from tests.test_multipart import *  # noqa: F401, F403
 from tests.test_websocket import *  # noqa: F401, F403
 from tests.test_sse import *  # noqa: F401, F403
 from tests.test_cors import *  # noqa: F401, F403

tests/test_multipart.py

@@ -0,0 +1,192 @@
+import asyncio
+import os
+import unittest
+from microdot import Microdot
+from microdot.multipart import with_form_data, FileUpload, FormDataIter
+from microdot.test_client import TestClient
+
+
+class TestMultipart(unittest.TestCase):
+    @classmethod
+    def setUpClass(cls):
+        if hasattr(asyncio, 'set_event_loop'):
+            asyncio.set_event_loop(asyncio.new_event_loop())
+        cls.loop = asyncio.get_event_loop()
+
+    def _run(self, coro):
+        return self.loop.run_until_complete(coro)
+
+    def test_simple_form(self):
+        app = Microdot()
+
+        @app.post('/sync')
+        @with_form_data
+        def sync_route(req):
+            return dict(req.form)
+
+        @app.post('/async')
+        @with_form_data
+        async def async_route(req):
+            return dict(req.form)
+
+        client = TestClient(app)
+
+        res = self._run(client.post(
+            '/sync', headers={
+                'Content-Type': 'multipart/form-data; boundary=boundary',
+            },
+            body=(
+                b'--boundary\r\n'
+                b'Content-Disposition: form-data; name="foo"\r\n\r\nbar\r\n'
+                b'--boundary\r\n'
+                b'Content-Disposition: form-data; name="baz"\r\n\r\nbaz\r\n'
+                b'--boundary--\r\n')
+        ))
+        self.assertEqual(res.status_code, 200)
+        self.assertEqual(res.json, {'foo': 'bar', 'baz': 'baz'})
+
+        res = self._run(client.post(
+            '/async', headers={
+                'Content-Type': 'multipart/form-data; boundary=boundary',
+            },
+            body=(
+                b'--boundary\r\n'
+                b'Content-Disposition: form-data; name="foo"\r\n\r\nbar\r\n'
+                b'--boundary\r\n'
+                b'Content-Disposition: form-data; name="baz"\r\n\r\nbaz\r\n'
+                b'--boundary--\r\n')
+        ))
+        self.assertEqual(res.status_code, 200)
+        self.assertEqual(res.json, {'foo': 'bar', 'baz': 'baz'})
+
+    def test_form_with_files(self):
+        saved_max_memory_size = FileUpload.max_memory_size
+        FileUpload.max_memory_size = 5
+
+        app = Microdot()
+
+        @app.post('/async')
+        @with_form_data
+        async def async_route(req):
+            d = dict(req.form)
+            for name, file in req.files.items():
+                d[name] = '{}|{}|{}'.format(file.filename, file.content_type,
+                                            (await file.read()).decode())
+            return d
+
+        client = TestClient(app)
+
+        res = self._run(client.post(
+            '/async', headers={
+                'Content-Type': 'multipart/form-data; boundary=boundary',
+            },
+            body=(
+                b'--boundary\r\n'
+                b'Content-Disposition: form-data; name="foo"\r\n\r\nbar\r\n'
+                b'--boundary\r\n'
+                b'Content-Disposition: form-data; name="f"; filename="f"\r\n'
+                b'Content-Type: text/plain\r\n\r\nbaz\r\n'
+                b'--boundary\r\n'
+                b'Content-Disposition: form-data; name="g"; filename="g"\r\n'
+                b'Content-Type: text/html\r\n\r\n<p>hello</p>\r\n'
+                b'--boundary\r\n'
+                b'Content-Disposition: form-data; name="x"\r\n\r\ny\r\n'
+                b'--boundary--\r\n')
+        ))
+        self.assertEqual(res.status_code, 200)
+        self.assertEqual(res.json, {'foo': 'bar', 'x': 'y',
+                                    'f': 'f|text/plain|baz',
+                                    'g': 'g|text/html|<p>hello</p>'})
+        FileUpload.max_memory_size = saved_max_memory_size
+
+    def test_file_save(self):
+        app = Microdot()
+
+        @app.post('/async')
+        @with_form_data
+        async def async_route(req):
+            for _, file in req.files.items():
+                await file.save('_x.txt')
+
+        client = TestClient(app)
+
+        res = self._run(client.post(
+            '/async', headers={
+                'Content-Type': 'multipart/form-data; boundary=boundary',
+            },
+            body=(
+                b'--boundary\r\n'
+                b'Content-Disposition: form-data; name="foo"\r\n\r\nbar\r\n'
+                b'--boundary\r\n'
+                b'Content-Disposition: form-data; name="f"; filename="f"\r\n'
+                b'Content-Type: text/plain\r\n\r\nbaz\r\n'
+                b'--boundary--\r\n')
+        ))
+        self.assertEqual(res.status_code, 204)
+        with open('_x.txt', 'rb') as f:
+            self.assertEqual(f.read(), b'baz')
+        os.unlink('_x.txt')
+
+    def test_no_form(self):
+        app = Microdot()
+
+        @app.post('/async')
+        @with_form_data
+        async def async_route(req):
+            return str(req.form)
+
+        client = TestClient(app)
+
+        res = self._run(client.post('/async', body={'foo': 'bar'}))
+        self.assertEqual(res.status_code, 200)
+        self.assertEqual(res.text, 'None')
+
+    def test_upload_iterator(self):
+        app = Microdot()
+
+        @app.post('/async')
+        async def async_route(req):
+            d = {}
+            async for name, value in FormDataIter(req):
+                if isinstance(value, FileUpload):
+                    d[name] = '{}|{}|{}'.format(value.filename,
+                                                value.content_type,
+                                                (await value.read(4)).decode())
+                else:
+                    d[name] = value
+            return d
+
+        client = TestClient(app)
+
+        res = self._run(client.post(
+            '/async', headers={
+                'Content-Type': 'multipart/form-data; boundary=boundary',
+            },
+            body=(
+                b'--boundary\r\n'
+                b'Content-Disposition: form-data; name="foo"\r\n\r\nbar\r\n'
+                b'--boundary\r\n'
+                b'Content-Disposition: form-data; name="f"; filename="f"\r\n'
+                b'Content-Type: text/plain\r\n\r\nbaz\r\n'
+                b'--boundary\r\n'
+                b'Content-Disposition: form-data; name="g"; filename="g.h"\r\n'
+                b'Content-Type: text/html\r\n\r\n<p>hello</p>\r\n'
+                b'--boundary\r\n'
+                b'Content-Disposition: form-data; name="x"\r\n\r\ny\r\n'
+                b'--boundary\r\n'
+                b'Content-Disposition: form-data; name="h"; filename="hh"\r\n'
+                b'Content-Type: text/plain\r\n\r\nyy' + (b'z' * 500) + b'\r\n'
+                b'--boundary\r\n'
+                b'Content-Disposition: form-data; name="i"; filename="i.1"\r\n'
+                b'Content-Type: text/plain\r\n\r\n1234\r\n'
+                b'--boundary--\r\n')
+        ))
+        self.assertEqual(res.status_code, 200)
+        self.assertEqual(res.json, {
+            'foo': 'bar',
+            'f': 'f|text/plain|baz',
+            'g': 'g.h|text/html|<p>h',
+            'x': 'y',
+            'h': 'hh|text/plain|yyzz',
+            'i': 'i.1|text/plain|1234',
+        })

tests/test_url_pattern.py

@@ -119,5 +119,30 @@ class TestURLPattern(unittest.TestCase):
         self.assertIsNone(p.match('/foo/abc/def/123/test'))
 
     def test_invalid_url_patterns(self):
-        self.assertRaises(ValueError, URLPattern, '/users/<foo/bar')
-        self.assertRaises(ValueError, URLPattern, '/users/<badtype:id>')
+        p = URLPattern('/users/<foo/bar')
+        self.assertRaises(ValueError, p.compile)
+        p = URLPattern('/users/<badtype:id>')
+        self.assertRaises(ValueError, p.compile)
+
+    def test_custom_url_pattern(self):
+        URLPattern.register_type('hex', '[0-9a-f]+')
+        p = URLPattern('/users/<hex:id>')
+        self.assertEqual(p.match('/users/a1'), {'id': 'a1'})
+        self.assertIsNone(p.match('/users/ab12z'))
+
+        URLPattern.register_type('hex', '[0-9a-f]+',
+                                 parser=lambda value: int(value, 16))
+        p = URLPattern('/users/<hex:id>')
+        self.assertEqual(p.match('/users/a1'), {'id': 161})
+        self.assertIsNone(p.match('/users/ab12z'))
+
+        def hex_parser(value):
+            try:
+                return int(value, 16)
+            except ValueError:
+                return None
+
+        URLPattern.register_type('hex', parser=hex_parser)
+        p = URLPattern('/users/<hex:id>')
+        self.assertEqual(p.match('/users/a1'), {'id': 161})
+        self.assertIsNone(p.match('/users/ab12z'))

tests/test_urlencode.py

@@ -1,5 +1,5 @@
 import unittest
-from microdot.microdot import urlencode, urldecode_str, urldecode_bytes
+from microdot.microdot import urlencode, urldecode
 
 
 class TestURLEncode(unittest.TestCase):
@@ -7,5 +7,7 @@ class TestURLEncode(unittest.TestCase):
         self.assertEqual(urlencode('?foo=bar&x'), '%3Ffoo%3Dbar%26x')
 
     def test_urldecode(self):
-        self.assertEqual(urldecode_str('%3Ffoo%3Dbar%26x'), '?foo=bar&x')
-        self.assertEqual(urldecode_bytes(b'%3Ffoo%3Dbar%26x'), '?foo=bar&x')
+        self.assertEqual(urldecode('%3Ffoo%3Dbar%26x'), '?foo=bar&x')
+        self.assertEqual(urldecode(b'%3Ffoo%3Dbar%26x'), '?foo=bar&x')
+        self.assertEqual(urldecode('dot%e2%80%a2dot'), 'dot•dot')
+        self.assertEqual(urldecode(b'dot%e2%80%a2dot'), 'dot•dot')