I've disabled the use of unsafe features (Jinja extensions, migrations, and tasks) by default and added a new CLI switch `--UNSAFE` which enables them. Templates that don't use unsafe features are unaffected by this change. But Copier will raise an error for templates that do use unsafe features unless the `--UNSAFE` flag is passed.
I've not added an interactive prompt that asks for consent for using unsafe features because I think it's not clear how to distinguish between interactive prompting and raising an error when `--UNSAFE` is not passed. For this, I think Copier would need a switch that clearly states whether interactive or non-interactive mode is desired. Currently, `--defaults` implies this for questions.
Fixes https://github.com/copier-org/copier/issues/1137
BREAKING CHANGE: Copier raises an error when a template uses unsafe features unless the `--UNSAFE` switch is passed
This new command allows to reapply a template, keeping old answers but discarding subproject evolution.
It is useful when there are bugs replaying an old version of the template, or when the subproject has drifted too much from the template and you need to reset it.
BREAKING CHANGE: All CLI calls to Copier must now include the subcommand as the 1st argument. For example, `copier` must become now `copier update`; also `copier ./tpl ./dst` must become `copier copy ./tpl ./dst`.
BREAKING CHANGE: All flags must go after the subcommand now. For example, `copier -r HEAD update ./dst` must now become `copier update -r HEAD ./dst` or `copier update ./dst -r HEAD`.
BREAKING CHANGE: Automatic mode removed. Since now subcommands are required, the automatic mode is removed.
BREAKING CHANGE: Deprecated `copier.copy` function is removed. Use `copier.run_copy`, `copier.run_update` or `copier.run_recopy` explicitly as needed.
Fix https://github.com/copier-org/copier/issues/1081
Close https://github.com/copier-org/copier/issues/1082
* refactor(tests): add type hints and clean up
* fix: import `Protocol` from `typing-extensions` when using Python 3.7
* fix: import `Literal` from `typing-extensions` when using Python 3.7
* refactor(tests): create template and subproject directory using `tmp_path_factory` fixture
* refactor(tests): use OS-agnostic file path separator
* refactor(tests): use `/` separator for `skip_if_exists` paths
* style: fix formatting error
* refactor(tests): expect POSIX path for local repo URL
- Provide a dev shell.
- Provide a nix package.
- Provide a nix flake.
- Development environment based on direnv.
- Docs.
- Configure Gitpod to use direnv and nix.
- Configure Cachix out of the box, and document how to use it.
- Add direnv and nix to CI.
- Satisfy some linters that came from Precommix, even when Precommix was later discarded.
- Mark some tests as impure.
- Run only pure tests when building Copier with Nix.
- Add poetry loader to direnv.
- Update contribution guide.
* Refactor
* Fix#110.
* Rewrite test_config_exclude, test_config_exclude_overridden and test_config_include. These tests were badly designed, using a monkeypatch that would never happen in the real world, and actually producing false positives. I moved them to test_exclude.py and rewritten to test the what and not the how.
* Fix#214 by removing skip option. Relevant tests use the better skip_if_exists=["**"].
* Remove subdirectory flag from API/CLI. It was confusing and could lead to bad maintenance situations. Fixes#315.
* Remove extra_paths and fix#321
* Remember that you cannot use _copier_conf.src_path as a path now
* use dataclasses
* Create errors module, simplify some tests, fix many others
* Fix some tests, complete EnvOps removal
* Fix#214 and some tests related to it
* Reorder code
* Update docs and imports
* Modularize test_complex_questions
* Interlink worker and questionary a bit better
* Removal of Questionary class, which only had 1 meaningful method that is now merged into Worker to avoid circular dependencies.
* Fix#280 in a simple way: only user answers are type-casted inside API, and CLI transforms all `--data` using YAML always. Predictable.
* Use prereleases correctly.
* Reorder AnswersMap to have a more significative repr.
* Simpler cache for old `Question.get_choices()` method (renamed now).
* fix wrong test
* Fix test_subdirectory
* Fix test_tasks (and remove tests/demo_tasks)
* Fix path filter tests, and move it to test_exclude, where it belongs
* Make test_config pass
* Fix more wrongly designed tests
* Use cached_property backport if needed
* xfail test known to fail on mac
* force posix paths on windows
* Add typing_extensions for python < 3.8
* Sort dependencies in pyproject.toml
* Support python 3.6 str-to-datetime conversion
* Workaround https://bugs.python.org/issue43095
* xfail test_path_filter on windows
* Upgrade mkdocs and other dependencies to fix https://github.com/pawamoy/mkdocstrings/issues/222
* Add missing reference docs.
* Add workaround for https://github.com/pawamoy/mkdocstrings/pull/209
* Docs.
* Remove validators module
* Add workaround for https://github.com/pawamoy/mkdocstrings/issues/225
* Restore docs autorefs as explained in https://github.com/pawamoy/mkdocstrings/issues/226#issuecomment-775413562.
* Workaround https://github.com/pawamoy/pytkdocs/issues/86
* use a new toolkit for user prompting: questionary (patched).
* multiline questions.
* conditional questions.
* new toolkit for ui tests: pexpect.
* fix lots of tests.
* fix windows builds with newer poetry-dynamic-versioning.
* linters and mypy are now tests, to have faster ci.
* update deprecated ci commands.
* removed dependencies from old times.
* Remove toml 0.5+ syntax (dotted keys)
* Use powershell where syntax is compatible
* Change skip to xfail
* xfail pexpect tests on windows. I'm tired of trying to make it work
* more docs
* possibly something more.
Scenario:
1. Your project is in version `v1`.
2. You update ito `v2a1` using `--prerelease`.
3. Later, you just update without passing `--prerelease`.
Current result: Copier will try to downgrade your project to the latest normal release. Migrations do not work backwards, so this will leave your project broken.
Expected result: Copier forbids downgrading.
@Tecnativa TT23705
Before this patch, Copier didn't ignore prereleases when detecting latest template tag.
This is mostly a bug because there's no way to safely upgrade a template to the latest non-prerelease tag automatically.
This a behavioral change that probably didn't hit anybody out there, but enough to make a new big release.
- Ignore errors when executing `shutil.rmtree()`, because it seems like it's common to fail when deleting git repos on Windows, and since these are temp files, we don't really care that much there's garbage left. Any good OS should clean the temp folders automatically.
- Always find Jinja templates in `PosixPath` mode.
- Ignore `OSError` when trying to enter a possibly git root directory. This is yielded by Windows when the path is a URL and we don't really care about it.
- Fix some tests with non-windows hardcoded stuff.
- Fix a test that was using a Bash script. Modified to be Python, which should work fine cross-system.
- Remove external python dependencies (yaml, plumbum) from test task/migration files. These are available on Linux because it gets the python env from the venv, but on Windows, it uses the main python interpreter and breaks. After all, that's not very important here.
- Do not modify EOL in CI.
- Use python executable in tests instead of python3.
- Update pre-commit versions to include https://github.com/pre-commit/pre-commit-hooks/pull/509.
- Disable autorebasing.
- Disable pre-commit on CI on Windows.
- Require python 3.8 on Windows, where `tempfile` supports autoremoving directories with read-only files.
When migrating a template, it can happen that the answers spec is modified.
When that happens, the migration script should modify the answers file. Thus, the update process should reload answers after running pre-migrations.
That's what it is doing now with this patch.
@Tecnativa TT23705
The rendering context now includes `_copier_conf`, which is a copy of the raw `ConfigData` object, but removing anything not JSON-serializable (a.k.a. `data.now` and `data.make_secret`).
This is useful for a number of purposes:
1. Imagine you have migration scripts but you want to store them on your template and NOT in your copies. Then you can use `_copier_conf.src_path` to find the local clone and call anything from there. This fixes#95 and fixes#157.
2. In the execution environment, it includes `$STAGE` (can be `task`, `after` or `before`), and in the case of migrations also `$VERSION_{FROM,TO,CURRENT}` to let migration scripts know where they are in the moment they're called, so we can reuse migration scripts and unleash their full power.
Now you can write your scripts in any language you want, and use any task execution system you want. Tests include a bash script and a python script, and docs include how to integrate with Invoke.
@Tecnativa TT20357
