Damien George 388d419ba3 py/makecompresseddata.py: Make compression deterministic. ...

Error string compression is not deterministic in certain cases: it depends
on the Python version (whether dicts are ordered by default or not) and
probably also the order files are passed to this script, leading to a
difference in which words are included in the top 128 most common.

The changes in this commit use OrderedDict to keep parsed lines in a known
order, and, when computing how many bytes are saved by a given word, it
uses the word itself to break ties (which would otherwise be "random").

2020-04-20 10:32:49 +10:00

6.6 KiB

Raw Blame History

View Raw