When composing templates, it's often needed to be able to load answers from other templates that you know are usually combined with yours. Or any other kind of external data.
@moduon MT-8282
This new command allows to reapply a template, keeping old answers but discarding subproject evolution.
It is useful when there are bugs replaying an old version of the template, or when the subproject has drifted too much from the template and you need to reset it.
BREAKING CHANGE: All CLI calls to Copier must now include the subcommand as the 1st argument. For example, `copier` must become now `copier update`; also `copier ./tpl ./dst` must become `copier copy ./tpl ./dst`.
BREAKING CHANGE: All flags must go after the subcommand now. For example, `copier -r HEAD update ./dst` must now become `copier update -r HEAD ./dst` or `copier update ./dst -r HEAD`.
BREAKING CHANGE: Automatic mode removed. Since now subcommands are required, the automatic mode is removed.
BREAKING CHANGE: Deprecated `copier.copy` function is removed. Use `copier.run_copy`, `copier.run_update` or `copier.run_recopy` explicitly as needed.
Fix https://github.com/copier-org/copier/issues/1081
Close https://github.com/copier-org/copier/issues/1082
* refactor(tests): add type hints and clean up
* fix: import `Protocol` from `typing-extensions` when using Python 3.7
* fix: import `Literal` from `typing-extensions` when using Python 3.7
* refactor(tests): create template and subproject directory using `tmp_path_factory` fixture
* refactor(tests): use OS-agnostic file path separator
* refactor(tests): use `/` separator for `skip_if_exists` paths
* style: fix formatting error
* refactor(tests): expect POSIX path for local repo URL
The test case checks that merging entries for the the settings
`skip_if_exists`, `exclude`, `jinja_extensions` and
`secret_questions` work as expected and entries are concatenated.
* allow overriden defaults in questionnaire
This works in conjunction with https://github.com/copier-org/copier/pull/476.
If partial answers are written out to a file, they can be re-loaded as
default overrides to whatever is specified in the template allowing one
to resume progress.
* fix failing test
* Refactor
* Fix#110.
* Rewrite test_config_exclude, test_config_exclude_overridden and test_config_include. These tests were badly designed, using a monkeypatch that would never happen in the real world, and actually producing false positives. I moved them to test_exclude.py and rewritten to test the what and not the how.
* Fix#214 by removing skip option. Relevant tests use the better skip_if_exists=["**"].
* Remove subdirectory flag from API/CLI. It was confusing and could lead to bad maintenance situations. Fixes#315.
* Remove extra_paths and fix#321
* Remember that you cannot use _copier_conf.src_path as a path now
* use dataclasses
* Create errors module, simplify some tests, fix many others
* Fix some tests, complete EnvOps removal
* Fix#214 and some tests related to it
* Reorder code
* Update docs and imports
* Modularize test_complex_questions
* Interlink worker and questionary a bit better
* Removal of Questionary class, which only had 1 meaningful method that is now merged into Worker to avoid circular dependencies.
* Fix#280 in a simple way: only user answers are type-casted inside API, and CLI transforms all `--data` using YAML always. Predictable.
* Use prereleases correctly.
* Reorder AnswersMap to have a more significative repr.
* Simpler cache for old `Question.get_choices()` method (renamed now).
* fix wrong test
* Fix test_subdirectory
* Fix test_tasks (and remove tests/demo_tasks)
* Fix path filter tests, and move it to test_exclude, where it belongs
* Make test_config pass
* Fix more wrongly designed tests
* Use cached_property backport if needed
* xfail test known to fail on mac
* force posix paths on windows
* Add typing_extensions for python < 3.8
* Sort dependencies in pyproject.toml
* Support python 3.6 str-to-datetime conversion
* Workaround https://bugs.python.org/issue43095
* xfail test_path_filter on windows
* Upgrade mkdocs and other dependencies to fix https://github.com/pawamoy/mkdocstrings/issues/222
* Add missing reference docs.
* Add workaround for https://github.com/pawamoy/mkdocstrings/pull/209
* Docs.
* Remove validators module
* Add workaround for https://github.com/pawamoy/mkdocstrings/issues/225
* Restore docs autorefs as explained in https://github.com/pawamoy/mkdocstrings/issues/226#issuecomment-775413562.
* Workaround https://github.com/pawamoy/pytkdocs/issues/86
When adding the `exclude` option from CLI/API, it extends the definitions found in `copier.yml`.
When adding it in `copier.yml`, it replaces the defaults.
Fixes#215, which explains that it was very unexpected that excluding some extra things included a lot of other things by accident.
Before this patch, using `!include` was a bit absurd because it would fail under any useful scenario:
- Including with a glob.
- Trying to include more than 1 file.
Now all those are supported, and they can coexist. Besides, the patch is quite simple, which makes it more attractive.
Fix#237.
- Ignore errors when executing `shutil.rmtree()`, because it seems like it's common to fail when deleting git repos on Windows, and since these are temp files, we don't really care that much there's garbage left. Any good OS should clean the temp folders automatically.
- Always find Jinja templates in `PosixPath` mode.
- Ignore `OSError` when trying to enter a possibly git root directory. This is yielded by Windows when the path is a URL and we don't really care about it.
- Fix some tests with non-windows hardcoded stuff.
- Fix a test that was using a Bash script. Modified to be Python, which should work fine cross-system.
- Remove external python dependencies (yaml, plumbum) from test task/migration files. These are available on Linux because it gets the python env from the venv, but on Windows, it uses the main python interpreter and breaks. After all, that's not very important here.
- Do not modify EOL in CI.
- Use python executable in tests instead of python3.
- Update pre-commit versions to include https://github.com/pre-commit/pre-commit-hooks/pull/509.
- Disable autorebasing.
- Disable pre-commit on CI on Windows.
- Require python 3.8 on Windows, where `tempfile` supports autoremoving directories with read-only files.
- `ConfigData` objects now know where do answers come from.
- Their `.data` attribute is now a computed property that merges all those answer sources in priority order.
- To merge dicts, `ChainMap` is used extensively.
- `query_user_data` gets both `last_answer_data` and `forced_answers_data` instead of the single `answers_data` argument it got before.
- By knowing where does an answer come from, now Copier avoids asking something if the user already answered that from other source. For example, if you use `copier -d some_question=some_answer copy $src $dst`, then Copier will not ask you about `some_question` again, because you already answered that. 🎉
- All answer sources are deep-copied to avoid modifying mutable dicts and affecting further copier executions.
- A minimal amount of tests got updated as they were now supposed to fail with new behavior.
- Ignore flake8 `E501 line too long` errors. We use black, that's just unnecessarily annoying.
@Tecnativa TT23705
After #163, a `copier.yml` file without a `_answers_file` definition would default to `None` instead of `".copier-answers.yml"` as expected.
I modified some tests to reproduce this failure, and fixed them.
@Tecnativa TT20357
With this enhacenment, template designers are able to choose a different default path for the `.copier-answers.yml` file, and users are still able to alter it via their API/CLI calls.
The main purpose for this is to let users use several templates to fill the same destination directory, and still be able to autoupdate all of them by just providing the answers file path to use.
@Tecnativa TT20357
Secrets questions are available in the jinja render context, but are not logged into `.copier-answers.yml` by default.
To mark a question as secret, just set `secret: true` into its definition. You can also pass a list of question keys in `_secret_questions` if you prefer.
@Tecnativa TT20357
This commit fixes#119. Summary:
- Depend on packaging to get a good implementation of PEP 440.
- Document new behaviors.
- Add support for migrations.
- Checkout by default the most modern git tag available in the template.
- Reuse `run_tasks` to run migrations too.
- Use `git describe --tags --always` to obtain template commit, to get a version parseable by PEP 440.
- Update `test_updatediff` to with new behaviors.
@Tecnativa TT20357
As explained in #110, the dangling parameters for methods keep on complicating development.
A full code refactoring is too much work for now, but I did a little refactoring which consists in:
- Merge `Flags` into `ConfigData`, and remove anything related exclusively to flags.
- Any method that uses arguments from `ConfigData` gets instead a `conf: ConfigData` argument.
- `get_jinja_renderer()` removed; it's not useful as its little work can easily be done by `Renderer` itself.
This way, whenever we need a new config, we can just add it to `ConfigData` and use it more widely.
A full refactoring is still needed IMHO, where we have a main `Copier` class which handles its state and everything it needs (or it could be `CopierApp` instead), but that's more a design decision to be taken by the project leader.
@Tecnativa TT20357
Fix#88 the easiest way possible. Changes summary:
- A new `--vcs-ref` flag that indicates which commit/ref you want to copy, and only applies when the source directory is git-versioned.
- A new `--no-diff` flag to `copier update` skips the smartypants diff behavior.
What it does?
- Checks you are updating a git-versioned destination from a git-versioned source.
- Clones the destination into temp dir.
- Does a `copier copy -f tmp_src tmp_dst` (yes, a temp dir also).
- Gets the git diff between `tmp_dst` and `dst_path`.
- Performs a normal `copier update dst_path`.
- Applies the git diff at the end, to try to respect downstream project evolution.
@Tecnativa TT20357
The `.tmpl` suffix doesn't seem to be a very widely adopted standard on the Jinja world. Others like `.jinja` or `.j2` usually have better IDE integration.
It's not like `.tmpl` is a wrong default, so here I'm leaving it as a default but letting template developers to change it to another one that fits better their environment.
Apart from this, if you really had to produce any `.tmpl` files in your copies, having files that ended with `.tmpl.tmpl` seemed a bit weird, while now you can change that suffix to whatever fits better your project.
@Tecnativa TT20357
OK, now this diff is very big, but it's actually modifying only style, not code functions themselves. From now on, this won't have to happen again.
Basically I just ran `pre-commit run --all-files` and committed that. I hope you like how it looks now!
This is a totally breaking PR. It had to come sooner or later...
Before this patch, copier had some ugly problems:
- If a file was called `--exclude`, you couldn't add it to `--extra-paths` due to using `nargs='*'` in an option. It is a weird thing that `ArgumentParser` allows to do, but can potentially lead to unfixable problems.
- Argument parsing was a little bit handycraft work.
Meet [Plumbum](https://plumbum.readthedocs.io/), the swiss army knife for CLI packages:
- CLI has been refactored as a plumbum application.
- Tests for CLI parsing removed, since they were incompatible and we are using a well-tested CLI parsing library now after all.
- The application now includes 2 subcommands: `copy` and `update`.
- If called without args, or with 1, it will use `update`.
- If called with 2 args, it will use `copy` (this preserves old usage unchanged, unless you are copying from a template called `copy` or `update`; in such corner case, just call it as `copier copy copy destination` instead).
- `copier --help` and `copier --help-all` are more useful now.
And you might be wondering... Adding such a big dependency just for that? Well, actually plumbum supports more than just CLI applications: supports sane execution of local & remote commands, sane output coloring, sane user prompting... Possibly it will make Copier go to the next level as we keep on using it more. Just take a look at its docs! You'll love it 😉
Of course, all that new CLI had to be reflected in API changes:
- `src_path` is now optional (`None` by default). In such case, copier will try to guess the `src_path` from the `.copier-answers.yml` file in the copy, or fail otherwise.
- `dst_path` is also now optional (`.` by default), meaning that without a `src_path`, you'll be trying to copy anything else to current `$PWD`.
- The builtin `_log` variable, which contains all necessary data for copy updates, now has also a `_src_path` key that points to the original source. This only happens when it was copied from a git repo or from an absolute path in the local disk, as relative paths are not easily reproducible in next updates (they depend on the user's CWD when running copier).
- A new `original_src_path` variable goes around, indicating the git repo or abs system path passed to `src_path` on the initial call. This is just to be stored in `_log` as explained.
```bash
copier gh:example/template destination
cd destination
echo my changes >> anywhere
git commit -m changing .
copier # Equals to `copier update .`
git commit -m 'copier update' .
```
🎉 You're updated!
This is an importan step to complete #88.
@Tecnativa TT20357
