Commit Graph

2341 Commits

Author SHA1 Message Date
Damien George
30a9ccf4ca tests/basics: Move str/bytes tests that give SyntaxWarning to sep file.
In CPython 3.12 these invalid str/bytes/fstring escapes will issue a
SyntaxWarning, and so differ to MicroPython.

Signed-off-by: Damien George <damien@micropython.org>
2024-05-28 10:58:37 +10:00
Damien George
dd4767a7d1 tests/basics: Add .exp file for slice_op test.
CPython 3.12 implemented hashing for slices, so now differs to MicroPython.

Signed-off-by: Damien George <damien@micropython.org>
2024-05-28 10:50:57 +10:00
Damien George
ad6750b22e tests/float: Use "not" instead of ~ to invert bool value.
Otherwise CPython gives a deprecation warning.

This test is not actually testing inversion of bools, rather that bit of
the test is used to compute the pass/fail result.

Signed-off-by: Damien George <damien@micropython.org>
2024-05-28 10:49:22 +10:00
Damien George
a34b5d1b79 tests/net_inet/tls_text_errors.py: Tweak test for newer CPython version.
Signed-off-by: Damien George <damien@micropython.org>
2024-05-27 13:56:55 +10:00
Damien George
e1111d873a tests/net_hosted/ssl_verify_callback.py: Make exp match actual output.
The `cert` argument passed to the verify callback is actually a memoryview.
And the `depth` argument seems to start at 1 for the tested URL.

Signed-off-by: Damien George <damien@micropython.org>
2024-05-27 13:56:55 +10:00
Damien George
1a2fdcac0d tests/basics: Split out generator.throw tests that pass multiple args.
The three-argument form of `.throw()` is deprecated since CPython 3.12.  So
split out into separate tests (with .exp files) the parts of the generator
tests that test more than one argument.

Signed-off-by: Damien George <damien@micropython.org>
2024-05-27 13:56:55 +10:00
Damien George
2e852522b1 tests/extmod: Add .exp test files for asyncio.get_event_loop tests.
And use `asyncio.new_event_loop()` where possible.  This change is needed
because CPython 3.12 deprecated the `get_event_loop()` function.

Signed-off-by: Damien George <damien@micropython.org>
2024-05-27 13:56:55 +10:00
Damien George
1ea06b99d8 tests/extmod: Fix regex strings to be of r"" type.
Otherwise escape characters like \s and \W won't work correctly.

Signed-off-by: Damien George <damien@micropython.org>
2024-05-27 13:56:55 +10:00
Damien George
5d07d0c7b0 tests/run-natmodtests.py: Fix search for supported native tests.
Signed-off-by: Damien George <damien@micropython.org>
2024-05-27 11:44:54 +10:00
Brian Pugh
c624a5c0c4 py/dynruntime: Export mp_load_method_maybe and mp_arg_parse_all* funcs.
Also define `mp_type_bytearray`.  These all help to write native modules.

Signed-off-by: Brian Pugh <bnp117@gmail.com>
Signed-off-by: Damien George <damien@micropython.org>
2024-05-24 13:50:57 +10:00
Damien George
cfd5a8ea3a webassembly/proxy_c: Return undefined if dict lookup failed on JS side.
Instead of raising KeyError.  These semantics match JavaScript behaviour
and make it much more seamless to pass Python dicts through to JavaScript
as though they were JavaScript {} objects.

Signed-off-by: Damien George <damien@micropython.org>
2024-05-16 12:49:42 +10:00
Damien George
aa2e3880c1 webassembly/proxy_js: Create a special "undefined" type for Python.
This adds a new undefined singleton to Python, that corresponds directly to
JavaScript `undefined`.  It's accessible via `js.undefined`.

Signed-off-by: Damien George <damien@micropython.org>
2024-05-16 12:49:10 +10:00
Damien George
0148bbb495 webassembly/proxy_js: Revert back to converting Py None to JS null.
This reverts part of commit fa23e4b093, to
make it so that Python `None` converts to JavaScript `null` (and JavaScript
`null` already converts to Python `None`).  That's consistent with how the
`json` module converts these values back and forth.

Signed-off-by: Damien George <damien@micropython.org>
2024-05-16 12:44:43 +10:00
Damien George
025d10a702 tests/micropython/import_mpy_invalid.py: Skip if target cant import mpy.
Signed-off-by: Damien George <damien@micropython.org>
2024-05-14 16:02:30 +10:00
Damien George
154d602b6e webassembly/mpconfigport: Enable importing of .mpy files.
Signed-off-by: Damien George <damien@micropython.org>
2024-05-14 15:19:27 +10:00
Damien George
fa23e4b093 webassembly/proxy_js: Convert JS undefined and JS null to Py None.
And change Py None conversion so it converts to JS undefined.

The semantics for conversion of these objects are then:
- Python None           -> JavaScript undefined
- JavaScript undefined  -> Python None
- JavaScript null       -> Python None

This follows Pyodide:
https://pyodide.org/en/stable/usage/type-conversions.html

Signed-off-by: Damien George <damien@micropython.org>
2024-05-13 11:53:10 +10:00
Damien George
a67e326cb9 webassembly/proxy_c: Ensure objs thrown into generators are exceptions.
This commit defines a new `JsException` exception type which is used on the
Python side to wrap JavaScript errors.  That's then used when a JavaScript
Promise is rejected, and the reason is then converted to a `JsException`
for the Python side to handle.

This new exception is exposed as `jsffi.JsException`.

Signed-off-by: Damien George <damien@micropython.org>
2024-05-13 11:52:17 +10:00
Damien George
3f34be69c7 webassembly/asyncio: Fix case where a Promise is resolved with no arg.
Signed-off-by: Damien George <damien@micropython.org>
2024-05-13 11:48:41 +10:00
Damien George
be1ecb54e6 webassembly/api: Resolve thenables returned from runPythonAsync.
JavaScript semantics are such that the caller of an async function does not
need to await that function for it to run to completion.  This commit makes
that behaviour also apply to top-level async Python code run via
`runPythonAsync()`.

Signed-off-by: Damien George <damien@micropython.org>
2024-05-07 11:33:05 +10:00
Damien George
c056840ee8 webassembly/objpyproxy: Implement JS iterator protocol for Py iterables.
This allows using JavaScript for..of on Python iterables.

Signed-off-by: Damien George <damien@micropython.org>
2024-05-07 00:20:56 +10:00
Damien George
9da63a343e webassembly/proxy_c: Reject promises with a PythonError instance.
The `reason` in a rejected promise should be an instance of `Error`.  That
leads to better error messages on the JavaScript side.

Signed-off-by: Damien George <damien@micropython.org>
2024-05-06 14:04:13 +10:00
Damien George
8a3546b3bd webassembly: Add JavaScript-based asyncio support.
This commit adds a significant portion of the existing MicroPython asyncio
module to the webassembly port, using parts of the existing asyncio code
and some custom JavaScript parts.

The key difference to the standard asyncio is that this version uses the
JavaScript runtime to do the actual scheduling and waiting on events, eg
Promise fulfillment, timeouts, fetching URLs.

This implementation does not include asyncio.run(). Instead one just uses
asyncio.create_task(..) to start tasks and then returns to the JavaScript.
Then JavaScript will run the tasks.

The implementation here tries to reuse as much existing asyncio code as
possible, and gets all the semantics correct for things like cancellation
and asyncio.wait_for.  An alternative approach would reimplement Task,
Event, etc using JavaScript Promise's.  That approach is very difficult to
get right when trying to implement cancellation (because it's not possible
to cancel a JavaScript Promise).

Signed-off-by: Damien George <damien@micropython.org>
2024-04-24 16:24:00 +10:00
Angus Gratton
6877987002 tests/cpydiff: Add a note about risk of resizing memoryview targets.
This a stop-gap until there is a proper fix for this.

This work was funded through GitHub Sponsors.

Signed-off-by: Angus Gratton <angus@redyak.com.au>
2024-04-22 11:51:18 +10:00
Angus Gratton
4bed614e70 py/objarray: Fix use-after-free if extending a bytearray from itself.
Two cases, one assigning to a slice.
Closes https://github.com/micropython/micropython/issues/13283

Second is extending a slice from itself, similar logic.

In both cases the problem occurs when m_renew causes realloc to move the
buffer, leaving a dangling pointer behind.

There are more complex and hard to fix cases when either argument is a
memoryview into the buffer, currently resizing to a new address breaks
memoryviews into that object.

Reproducing this bug and confirming the fix was done by running the unix
port under valgrind with GC-aware extensions.

Note in default configurations with GIL this bug exists but has no impact
(the free buffer won't be reused while the function is still executing, and
is no longer referenced after it returns).

Signed-off-by: Angus Gratton <angus@redyak.com.au>
2024-04-22 11:50:52 +10:00
Damien George
5114f2c1ea webassembly/proxy_js: Allow a Python proxy of a function to be undone.
This optimises the case where a Python function is, for example, stored to
a JavaScript attribute and then later retrieved from Python.  The Python
function no longer needs to be a proxy with double proxying needed for the
call from Python -> JavaScript -> Python.

Signed-off-by: Damien George <damien@micropython.org>
2024-03-30 13:13:51 +11:00
Damien George
7c62fbe3f2 webassembly/proxy_js: Promote Python thenable to a Promise.
Signed-off-by: Damien George <damien@micropython.org>
2024-03-30 13:13:51 +11:00
Damien George
3997532186 webassembly/proxy_c: Ensure return value of async fun is passed to JS.
Signed-off-by: Damien George <damien@micropython.org>
2024-03-30 13:13:51 +11:00
Damien George
20a86eff53 tests/net_inet: Add simpler tls sites test, and skip existing on axtls.
Ports that use axtls cannot run the `test_tls_sites.py` test because the
sites it connects to use advanced ciphers.  So skip this test on such
ports, and add a new, simpler test that doesn't require certificate
verification and works with axtls.

Signed-off-by: Damien George <damien@micropython.org>
2024-03-29 10:46:09 +11:00
Damien George
bdbc869f9e py/persistentcode: Bump .mpy sub-version to 6.3.
This is required because the .mpy native ABI was changed by the
introduction of `mp_proto_fun_t`, see commits:
- 416465d81e
- 5e3006f117
- e2ff00e811

And three `mp_binary` functions were added to `mp_fun_table` in
commit d2276f0d41.

Signed-off-by: Damien George <damien@micropython.org>
2024-03-28 16:18:26 +11:00
Damien George
c1513a078d tests/ports/webassembly: Add webassembly JS tests.
Signed-off-by: Damien George <damien@micropython.org>
2024-03-22 14:31:25 +11:00
Damien George
e41b571a29 tests/run-tests.py: Support running webassembly tests via node.
This allows running tests with a .js/.mjs suffix, and also .py tests using
node and the webassembly port.

Signed-off-by: Damien George <damien@micropython.org>
2024-03-22 14:31:25 +11:00
Damien George
9d27183bde tests/float/float_struct_e.py: Add specific test for struct 'e' type.
Signed-off-by: Damien George <damien@micropython.org>
2024-03-20 14:13:49 +11:00
Matthias Urlichs
e520fa2e0f py/binary: Support half-float 'e' format in struct pack/unpack.
This commit implements the 'e' half-float format: 10-bit mantissa, 5-bit
exponent.  It uses native _Float16 if supported by the compiler, otherwise
uses custom bitshifting encoding/decoding routines.

Signed-off-by: Matthias Urlichs <matthias@urlichs.de>
Signed-off-by: Damien George <damien@micropython.org>
2024-03-20 14:13:49 +11:00
Damien George
3c445f6636 py/emitnative: Implement viper unary ops positive, negative and invert.
Signed-off-by: Damien George <damien@micropython.org>
2024-03-19 10:31:36 +11:00
Damien George
c9016b4979 tests/basics: Split MicroPython-specific deque tests to separate file.
So that the MicroPython-specific behaviour can be isolated, and the CPython
compatible test don't need a .exp file.

Signed-off-by: Damien George <damien@micropython.org>
2024-03-19 10:08:32 +11:00
Dash Peters
7dff38fdc1 py/objdeque: Expand implementation to be doubly-ended and support iter.
Add `pop()`, `appendleft()`, and `extend()` methods, support iteration
and indexing, and initializing from an existing sequence.

Iteration and indexing (subscription) have independent configuration flags
to enable them.  They are enabled by default at the same level that
collections.deque is enabled (the extra features level).

Also add tests for checking new behavior.

Signed-off-by: Damien George <damien@micropython.org>
2024-03-18 14:10:14 +11:00
Angus Gratton
7fd8a6d4bc stm32/dma: Add D-cache protection for DMA RX operations, including SPI.
This new DMA API corrects possible cache coherency issues on chips with
D-Cache, when working with buffers at arbitrary memory locations (i.e.
supplied by Python code).

The API is used by SPI to fix an issue with corrupt data when reading from
SPI using DMA in certain cases.  A regression test is included (it depends
on external hardware connection).

Explanation:

1) It's necessary to invalidate D-Cache after a DMA RX operation completes
   in case the CPU reads (or speculatively reads) from the DMA RX region
   during the operation.  This seems to have been the root cause of issue
   #13471 (only when src==dest for this case).

2) More generally, it is also necessary to temporarily mark the first and
   last cache lines of a DMA RX operation as "uncached", in case the DMA
   buffer shares this cache line with unrelated data.  The CPU could
   otherwise write the other data at any time during the DMA operation (for
   example from an interrupt handler), creating a dirty cache line that's
   inconsistent with the DMA result.

Fixes issue #13471.

This work was funded through GitHub Sponsors.

Signed-off-by: Angus Gratton <angus@redyak.com.au>
2024-03-08 12:19:48 +11:00
Phil Howard
dda9b9c6da all: Prune trailing whitespace.
Prune trailing whitespace across the whole project (almost), done
automatically with:

    grep -IUrl --color "[[:blank:]]$" --exclude-dir=.git --exclude=*.exp |\
        xargs sed -i 's/[[:space:]]*$//'

Exceptions:
- Skip third-party code in lib/ and drivers/cc3100/
- Skip generated code in bluetooth_init_cc2564C_1.5.c
- Preserve command output whitespace in docs, eg:
  docs/esp8266/tutorial/repl.rst

Signed-off-by: Phil Howard <phil@gadgetoid.com>
2024-03-07 16:25:17 +11:00
Damien George
56f9dcb580 tests/multi_bluetooth: Move ble_deepsleep to stress_deepsleep_reconnect.
This test cannot run on boards that have a native USB REPL, so rename it to
indicate that its "special".  This makes it easier to run a subset of
tests, for example:

    ./run-multitests.py multi_bluetooth/ble*.py
    ./run-multitests.py multi_bluetooth/perf*.py
    ./run-multitests.py multi_bluetooth/stress*.py

Signed-off-by: Damien George <damien@micropython.org>
2024-02-29 14:16:05 +11:00
Damien George
8692d2602a extmod/asyncio: Make current_task raise exception when there is no task.
Matches CPython behaviour.

Fixes issue #11530.

Signed-off-by: Damien George <damien@micropython.org>
2024-02-28 15:48:51 +11:00
Felix Dörre
8547a78275 extmod/modwebsocket: Fix websocket to send correct close frame.
When the websocket closes currently, it does not send a proper
"close"-frame, but rather encodes the 0x8800-sequence inside a binary
packet, which is wrong.  The close packet is a different kind of websocket
frame, according to https://www.rfc-editor.org/rfc/rfc6455.

This change resolves an error in Firefox when the websocket closes.

Signed-off-by: Felix Dörre <felix@dogcraft.de>
2024-02-26 12:20:40 +11:00
Trent Warlaven
3deeabe6e8 tests/cpydiff: Add new CPy diff test for class name mangling.
Adds new tests/documentation for missing name mangling for private class
members.

Signed-off-by: Trent Warlaven <trwbox@gmail.com>
2024-02-21 11:42:35 +11:00
Damien George
3db29103a4 py/builtinevex: Fix setting globals for native functions in compile().
Signed-off-by: Damien George <damien@micropython.org>
2024-02-20 12:12:28 +11:00
Damien George
0c7ccb8807 py/objfun: Support __name__ on native functions and generators.
This is now easy to support, since the first machine-word of a native
function tells how to find the prelude, from which the function name can be
extracted in the same way as for bytecode.

Signed-off-by: Damien George <damien@micropython.org>
2024-02-20 10:56:24 +11:00
Damien George
5a3dd8c791 tests/ports/unix: Add coverage test for frozen functions and generators.
Signed-off-by: Damien George <damien@micropython.org>
2024-02-16 14:17:01 +11:00
Damien George
866fc3447c tests/multi_bluetooth/ble_irq_calls.py: Enhance test to test recursion.
Signed-off-by: Damien George <damien@micropython.org>
2024-02-16 10:20:28 +11:00
Damien George
abe43fe687 extmod/btstack: Reset pending_value_handle before calling read-done cb.
Similar to the previous commit but for MP_BLUETOOTH_IRQ_GATTC_READ_DONE:
the pending_value_handle needs to be reset before calling
mp_bluetooth_gattc_on_read_write_status(), which will call the Python IRQ
handler, which may in turn call back into BTstack to perform an action like
a write.  In that case the pending_value_handle will need to be available
for the write/read/etc to proceed.

Fixes issue #13634.

Signed-off-by: Damien George <damien@micropython.org>
2024-02-12 11:26:27 +11:00
Damien George
b4f59984f7 extmod/btstack: Reset pending_value_handle before calling write-done cb.
The pending_value_handle needs to be freed and reset before calling
mp_bluetooth_gattc_on_read_write_status(), which will call the Python IRQ
handler, which may in turn call back into BTstack to perform an action like
a write.  In that case the pending_value_handle will need to be available
for the write/read/etc to proceed.

Fixes issue #13611.

Signed-off-by: Damien George <damien@micropython.org>
2024-02-09 11:44:28 +11:00
Damien George
b87bbaeb43 tests: Use vfs module instead of os.
Signed-off-by: Damien George <damien@micropython.org>
2024-02-07 13:25:09 +11:00
Damien George
e7020463f1 extmod/modvfs: Add new "vfs" module with mount/umount and Vfs classes.
They have been moved from the "os" module.

Signed-off-by: Damien George <damien@micropython.org>
2024-02-07 13:25:08 +11:00