Skip to content

Make fast compression the default #24

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
alvinwan wants to merge 1 commit into from
Closed

Make fast compression the default #24

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 14 additions & 1 deletion README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,13 @@ Bundle mode emits one file:
pymini bundle src -o out/bundle.py
```

The default profile now favors speed. To run the slower full pipeline with
extra compression passes, use:

```bash
pymini package src --slow -o out
```

You can also use the Python API directly:

```python
Expand All @@ -36,10 +43,16 @@ sources, modules = minify(
)
```

To use the slower profile through the API:

```python
sources, modules = minify(..., fast=False)
```

# Compression

Representative compression results, using
`--rename-modules --rename-global-variables --rename-arguments`:
`--slow --rename-modules --rename-global-variables --rename-arguments`:

![Benchmark summary chart comparing minify-only minification and minify-plus-wheel compression across packages](./benchmarks/summary.svg)

Expand Down
59 changes: 34 additions & 25 deletions benchmarks/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,16 @@ the benchmark harness used to reproduce them.

![Benchmark summary chart comparing minify-only minification and minify-plus-wheel compression across packages](./summary.svg)

## Profiles

`pymini` now defaults to the faster profile. The slower full pipeline is still
available with `pymini ... --slow` or `minify(..., fast=False)`.

Those extra hoisting and aliasing passes are input-sensitive: they help some
inputs, do nothing on some, and can slightly increase output on others. The
package size tables below keep using the slower profile so they stay directly
comparable with earlier revisions.

## Compression

Compression multipliers below are all relative to the original raw Python
Expand Down Expand Up @@ -78,7 +88,8 @@ repo's actual packaging metadata.
| pyminifier | 79,693 | 61,088 | 64,019 | 76,101 |
| rich | 310,458 | 183,326 | failed | 266,399 |

`pymini` is benchmarked with `--rename-modules --rename-global-variables --rename-arguments`.
The `pymini` compression tables above use
`--slow --rename-modules --rename-global-variables --rename-arguments`.
The `pymini` rows use package mode. The baselines minify each file
independently in the preserved package tree. For wheel rows, each tool first
rewrites the package tree, then `python -m build --wheel` runs on that
Expand All @@ -95,36 +106,31 @@ Wheel-specific failures:

## Speed

| Input | pymini | pyminifier | python-minifier |
| --- | ---: | ---: | ---: |
| pyminifier.py | 3.0 ms | 0.8 ms | 2.6 ms |
| pyminify.py | 7.9 ms | 2.0 ms | 6.7 ms |
| click | 855.2 ms | failed | 823.9 ms |
| pytest | 3.258 s | failed | 3.156 s |
| TexSoup | 218.6 ms | failed | 251.1 ms |
| timefhuman | 441.5 ms | failed | 450.6 ms |
| pyminifier | 196.7 ms | 68.5 ms | 191.7 ms |
| rich | 3.226 s | failed | 3.080 s |

pyminifier minification fails on `__init__.py` with
`TypeError: 'NoneType' object is not subscriptable`.

The single-file rows come from [benchmark_speed.py](./benchmark_speed.py). The
package rows are averages of three fresh-process runs from the same
environment used for the compression comparison. `pymini` used package mode
with `--rename-modules --rename-global-variables --rename-arguments`, while
the baseline tools minified each file independently in the preserved package
tree. The `click`, `pytest`, and `pyminifier` rows use the checked-in fixtures
under `.bench-repos`; the other package rows use local package checkouts.
The current default favors speed. On the checked-in fixtures, the slower
profile adds about 25-65% runtime.

| Input | pymini | pymini --slow | pymini bytes | pymini --slow bytes |
| --- | ---: | ---: | ---: | ---: |
| pyminifier.py | 3.0 ms | 3.5 ms | 435 | 435 |
| pyminify.py | 5.3 ms | 7.9 ms | 1,242 | 935 |
| click | 677.8 ms | 913.7 ms | 120,579 | 121,176 |
| pytest | 2.797 s | 3.669 s | 503,996 | 508,888 |
| pyminifier | 142.9 ms | 194.8 ms | 31,481 | 31,332 |

The single-file rows come from [benchmark_speed.py](./benchmark_speed.py).
The package rows above use the checked-in fixtures under `.bench-repos`.

The slower extra passes are not monotonic: `pyminify.py` and the
`pyminifier` package shrink a bit more, while `click` and `pytest` end up
slightly larger.

# Reproduce

Recompute the speed measurements with:

```bash
python3 -m pip install -e ".[dev]" python-minifier
git clone https://github.com/liftoff/pyminifier /tmp/pyminifier
PYTHONPATH=. .venv/bin/python benchmarks/benchmark_speed.py --pyminifier-root /tmp/pyminifier
PYTHONPATH=. .venv/bin/python benchmarks/benchmark_speed.py
```

The larger package comparisons in this file were run against these checkouts:
Expand All @@ -141,6 +147,9 @@ The larger package comparisons in this file were run against these checkouts:
Each package result above was checked against the repo's own test suite using a
temporary minified package tree on `PYTHONPATH`.

These validation rows correspond to the same `pymini --slow` rewrites used for
the compression tables above.

| Package | pymini | pyminifier | python-minifier |
| --- | --- | --- | --- |
| TexSoup | 78 passed | 78 passed | 78 passed |
Expand All @@ -161,7 +170,7 @@ To reproduce that flow locally:
```bash
git clone https://github.com/alvinwan/TexSoup /tmp/texsoup
mkdir -p /tmp/texsoup-out/TexSoup
pymini package /tmp/texsoup/TexSoup -o /tmp/texsoup-out/TexSoup --rename-modules --rename-global-variables --rename-arguments
pymini package /tmp/texsoup/TexSoup -o /tmp/texsoup-out/TexSoup --slow --rename-modules --rename-global-variables --rename-arguments
cp -R /tmp/texsoup/tests /tmp/texsoup-tests
PYTHONPATH=/tmp/texsoup-out:/tmp/texsoup-tests python3 -m pytest /tmp/texsoup-tests/tests -o addopts=''
```
Expand Down
72 changes: 54 additions & 18 deletions benchmarks/benchmark_speed.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -18,12 +18,16 @@
ROOT = Path(__file__).resolve().parents[1]
EXAMPLE_DIR = ROOT / "tests" / "examples"
DEFAULT_TEXSOUP_ROOT = Path("/tmp/pymini-texsoup-repo/TexSoup")
DEFAULT_PYMINIFIER_ROOT = Path("/tmp/pymini-pyminifier-src/pyminifier-2.1")
PYMINI_AGGRESSIVE_OPTIONS = {
DEFAULT_PYMINIFIER_ROOT = ROOT / ".bench-repos" / "pyminifier-tool"
PYMINI_DEFAULT_OPTIONS = {
"keep_module_names": False,
"keep_global_variables": False,
"rename_arguments": True,
}
PYMINI_SLOW_OPTIONS = {
**PYMINI_DEFAULT_OPTIONS,
"fast": False,
}


def benchmark_transform(
Expand All @@ -50,13 +54,14 @@ def benchmark_transform(
}


def pymini_single_file_transform(path: Path):
def pymini_single_file_transform(path: Path, *, fast: bool = True):
def transform(source: str) -> str:
outputs, _ = minify(
source,
path.stem,
keep_global_variables=False,
rename_arguments=True,
fast=fast,
)
return outputs[0]

Expand Down Expand Up @@ -101,14 +106,16 @@ def benchmark_package_api(
*,
iterations: int,
warmup: int,
fast: bool = True,
) -> dict[str, float]:
paths, module_root = resolve_python_files(str(package_root))
sources, modules, _ = load_sources(paths, module_root=module_root)
options = PYMINI_DEFAULT_OPTIONS if fast else PYMINI_SLOW_OPTIONS
for _ in range(warmup):
minify(
sources,
modules,
**PYMINI_AGGRESSIVE_OPTIONS,
**options,
)
samples = []
outputs = None
Expand All @@ -117,7 +124,7 @@ def benchmark_package_api(
outputs, _ = minify(
sources,
modules,
**PYMINI_AGGRESSIVE_OPTIONS,
**options,
)
samples.append(perf_counter() - start)
raw_bytes = sum(len(source.encode()) for source in sources)
Expand All @@ -132,24 +139,25 @@ def benchmark_package_api(
}


def benchmark_package_cli(package_root: Path, *, iterations: int) -> dict[str, float]:
def benchmark_package_cli(package_root: Path, *, iterations: int, fast: bool = True) -> dict[str, float]:
samples = []
output_bytes = 0
for _ in range(iterations):
output_dir = Path(tempfile.mkdtemp(prefix="pymini-bench-"))
try:
start = perf_counter()
rc = cli_main(
[
"package",
str(package_root),
"--rename-modules",
"--rename-global-variables",
"--rename-arguments",
"-o",
str(output_dir),
]
)
argv = [
"package",
str(package_root),
"--rename-modules",
"--rename-global-variables",
"--rename-arguments",
"-o",
str(output_dir),
]
if not fast:
argv.insert(-2, "--slow")
rc = cli_main(argv)
samples.append(perf_counter() - start)
if rc != 0:
raise RuntimeError(f"pymini CLI returned {rc}")
Expand All @@ -166,7 +174,10 @@ def print_example_results(
warmup: int,
pyminifier_root: Path,
) -> None:
tool_factories = [("pymini", pymini_single_file_transform)]
tool_factories = [
("pymini", lambda path: pymini_single_file_transform(path, fast=True)),
("pymini-slow", lambda path: pymini_single_file_transform(path, fast=False)),
]
python_minifier = load_python_minifier()
if python_minifier is not None:
tool_factories.append(("python-minifier", python_minifier))
Expand Down Expand Up @@ -210,10 +221,23 @@ def print_package_results(
texsoup_root,
iterations=package_api_iterations,
warmup=warmup,
fast=True,
)
api_slow_result = benchmark_package_api(
texsoup_root,
iterations=package_api_iterations,
warmup=warmup,
fast=False,
)
cli_result = benchmark_package_cli(
texsoup_root,
iterations=package_cli_iterations,
fast=True,
)
cli_slow_result = benchmark_package_cli(
texsoup_root,
iterations=package_cli_iterations,
fast=False,
)

print()
Expand All @@ -227,10 +251,22 @@ def print_package_results(
f"{api_result['avg_ms']:.3f}\t"
f"{api_result['throughput_kb_s']:.1f}"
)
print(
f"TexSoup-api-slow\t"
f"{int(api_slow_result['files'])}\t"
f"{int(api_slow_result['bytes'])}\t"
f"{int(api_slow_result['output_bytes'])}\t"
f"{api_slow_result['avg_ms']:.3f}\t"
f"{api_slow_result['throughput_kb_s']:.1f}"
)
print(
f"TexSoup-cli\t-\t-\t{int(cli_result['output_bytes'])}\t"
f"{cli_result['avg_ms']:.3f}\t-"
)
print(
f"TexSoup-cli-slow\t-\t-\t{int(cli_slow_result['output_bytes'])}\t"
f"{cli_slow_result['avg_ms']:.3f}\t-"
)


def main() -> int:
Expand Down
35 changes: 32 additions & 3 deletions pymini/cli.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -36,6 +36,20 @@ def build_parser() -> ArgumentParser:
action='store_true',
help='Rename function and method arguments, including internal keyword call sites when safe.',
)
profile = parser.add_mutually_exclusive_group()
profile.add_argument(
'--fast',
dest='fast',
action='store_true',
help='Use the default faster profile.',
)
profile.add_argument(
'--slow',
dest='fast',
action='store_false',
help='Run the slower full pipeline with extra compression passes.',
)
parser.set_defaults(fast=True)
parser.add_argument('--single-file', action='store_true', help=SUPPRESS)
parser.add_argument('-o', '--output', help='Path to the output directory', default='./')
parser.add_argument('--version', action='version', version=f'%(prog)s {__version__}')
Expand All @@ -57,11 +71,18 @@ def effective_mode(args) -> str:
return BUNDLE_MODE if args.single_file else args.mode


def resolve_options(args) -> tuple[str, bool, bool, bool, bool]:
def resolve_options(args) -> tuple[str, bool, bool, bool, bool, bool]:
mode = effective_mode(args)
keep_module_names = not args.rename_modules
keep_global_variables = not args.rename_global_variables
return mode, keep_module_names, keep_global_variables, args.rename_arguments, mode == BUNDLE_MODE
return (
mode,
keep_module_names,
keep_global_variables,
args.rename_arguments,
args.fast,
mode == BUNDLE_MODE,
)


def resolve_python_files(path: str) -> tuple[list[Path], Optional[Path]]:
Expand Down Expand Up @@ -168,7 +189,14 @@ def write_outputs(
def main(argv: Optional[Sequence[str]] = None) -> int:
parser = build_parser()
args = parser.parse_args(normalize_argv(argv))
mode, keep_module_names, keep_global_variables, rename_arguments, output_single_file = resolve_options(args)
(
mode,
keep_module_names,
keep_global_variables,
rename_arguments,
fast,
output_single_file,
) = resolve_options(args)
paths, module_root = resolve_python_files(args.path)
if not paths:
parser.error(f"no Python files matched {args.path!r}")
Expand All @@ -185,6 +213,7 @@ def main(argv: Optional[Sequence[str]] = None) -> int:
keep_module_names=keep_module_names,
keep_global_variables=keep_global_variables,
rename_arguments=rename_arguments,
fast=fast,
output_single_file=output_single_file,
)
try:
Expand Down
Loading
Loading