1071 Commits

Author SHA1 Message Date
Yuya Nishihara
f14a6e6f19 git: remove PartialEq requirement from GitPushError
Non-trivial error types usually don't implement Eq/PartialEq.
2025-01-10 01:16:33 +00:00
Yuya Nishihara
49999a7507 local_working_copy: on snapshot, ignore submodule in ignored directories
Fixes #5246
2025-01-08 09:39:59 +09:00
Martin von Zweigbergk
4a7d9891e6 rewrite: back out 9d4a973
I think the test case added in the previous commit shows a clear case
for why commit 9d4a973 was a bad idea.
2025-01-07 09:08:46 -08:00
Martin von Zweigbergk
ada946db62 rewrite: add another test of rebasing with same change on both sides 2025-01-07 09:08:46 -08:00
Scott Taylor
42b390bbc4 git: use merged parent tree for git index
Instead of setting the index to match the tree of HEAD, we now set the
index to the merged parent tree of the working copy commit. This means
that if you edit a merge commit, it will make the Git index look like it
would in the middle of a `git merge` operation (with all of the
successfully-merged files staged in the index).

If there are any 2-sided conflicts in the merged parent tree, then they
will be added to the index as conflicts. Since Git doesn't support
conflicts with more than 2 sides, many-sided conflicts are staged as the
first side of the conflict. The following commit will improve this.
2025-01-06 19:17:51 -06:00
Scott Taylor
9cc8b35251 git: use gix instead of git2 to update index
This will give us more fine-grained control over what files we put in
the index, allowing us to create conflicted index states. We also still
need to use git2 to clean up the merge/rebase state, since gix doesn't
have any function for this currently.
2025-01-06 19:17:51 -06:00
Scott Taylor
8e3ec9c58c git: add tests for index in colocated repo
These tests should still pass after we switch to gix for resetting the
index, so we need to make sure they don't rely on the cached index from
the `git2::Repository` instance.
2025-01-06 19:17:51 -06:00
Yuya Nishihara
b8653989c1 tests: add convenient method to initialize TestWorkspace with test settings
Functions are renamed, and their arguments are reordered to be consistent with
the TestRepo API.
2025-01-06 22:37:33 +09:00
pylbrecht
87b38b3073 sign: provide display in TestSigningBackend
We need to provide a value for the `display` attribute to assert against
in upcoming tests, which verify signature templates.
2025-01-04 13:24:08 +01:00
pylbrecht
638f123459 sign: move TestSigningBackend to lib
We need to make `TestSigningBackend` available for cli tests, as we will
add cli tests for signing related functionality (templates for
displaying commit signatures, `jj sign`) in upcoming commits.

Co-authored-by: julienvincent <m@julienvincent.io>
2025-01-04 13:24:08 +01:00
Yuya Nishihara
cff73841ed repo: remove &UserSettings argument from new/rewrite_commit(), use self.settings 2024-12-31 10:51:57 +09:00
Yuya Nishihara
14b52205fb repo: remove &UserSettings argument from start_transaction(), use self.settings 2024-12-31 10:51:57 +09:00
Yuya Nishihara
57806ee8c2 test_commit_builder: reload repo per settings
I'm going to remove the settings argument from start_transaction(),
new_commit(), and rewrite_commit(), and these functions will use the settings
passed in to the constructor.
2024-12-31 10:51:57 +09:00
Yuya Nishihara
e4a350fcaa config: extract jj-lib's default values to embedded TOML file
It's nice that "jj config list --include-defaults" can show these default
values.

I just copied the jj-cli directory structure, but it's unlikely we'll add more
config/*.toml files.
2024-12-25 10:44:37 +09:00
Yuya Nishihara
4a69d0178c settings: cache commit and operation parameters by UserSettings
This helps propagate configuration error. RevsetParseContext is also updated
because it was easier to pass &str in to it.
2024-12-23 22:57:57 +09:00
Yuya Nishihara
6f00c565b2 graph: inline ReverseGraphIterator to callers 2024-12-23 09:28:03 +09:00
Yuya Nishihara
78b5766993 repo, workspace: use dunce::canonicalize() to normalize paths
These paths may be printed, compared with user inputs, or passed to external
programs. It's probably better to avoid unusual "\\?\C:\" paths on Windows.

Fixes #5143
2024-12-22 09:45:37 +09:00
Scott Taylor
6baa43624c local_working_copy: store materialized conflict marker length
Storing the conflict marker length in the working copy makes conflict
parsing more consistent, and it allows us to parse valid conflict hunks
even if the user left some invalid conflict markers in the file while
resolving the conflicts.
2024-12-21 11:36:30 -06:00
Scott Taylor
b11ce6bd28 conflicts: escape conflict markers by making them longer
If a file contains lines which look like conflict markers, then we need
to make the real conflict markers longer so that the materialized
conflicts can be parsed unambiguously.

When parsing the conflict, we require that the conflict markers are at
least as long as the materialized conflict markers based on the current
tree. This can lead to some unintuitive edge cases which will be solved
in the next commit.

For instance, if we have a file explaining the differences between
Jujutsu's conflict markers and Git's conflict markers, it could produce
a conflict with long markers like this:

```
<<<<<<<<<<< Conflict 1 of 1
%%%%%%%%%%% Changes from base to side #1
 Jujutsu uses different conflict markers than Git, which just shows the
-sides of a conflict without a diff.
+sides of a conflict without a diff:
+
+<<<<<<<
+left
+|||||||
+base
+=======
+right
+>>>>>>>
+++++++++++ Contents of side #2
Jujutsu uses different conflict markers than Git:

<<<<<<<
%%%%%%%
-base
+left
+++++++
right
>>>>>>>
>>>>>>>>>>> Conflict 1 of 1 ends
```

We should support these options for "git" conflict marker style as well,
since Git actually does support producing longer conflict markers in
some cases through .gitattributes:

https://git-scm.com/docs/gitattributes#_conflict_marker_size

We may also want to support passing the conflict marker length to merge
tools as well in the future, since Git supports a "%L" parameter to pass
the conflict marker length to merge drivers:

https://git-scm.com/docs/gitattributes#_defining_a_custom_merge_driver
2024-12-21 11:36:30 -06:00
Benjamin Tan
dddeb8b526 lib: fix various typos
This commit fixes typos unintentionally introduced in d9c68e08, when
renaming `jj branch` to `jj bookmark`.
2024-12-21 02:46:24 +08:00
Yuya Nishihara
6bbaf79ca5 settings: parse TOML date-time value as well as string timestamp
We can usually omit quotes in --config=NAME=VALUE, but an RFC3339 string is a
valid TOML date-time expression. It's weird that quoting is required to specify
a date-time value.
2024-12-18 09:51:56 +09:00
Yuya Nishihara
4f08c62fe5 settings: propagate error from UserSettings::from_config()
All variables parsed here are debug options, but it would be annoying if
timestamp options were silently ignored because of a typo.
2024-12-18 09:51:56 +09:00
Martin von Zweigbergk
b836e0ae95 docs/cli: update URLs to from martinvonz user to jj-vcs org
We just migrated to the jj-vcs GitHub org, so we should point to the
new GitHub URLs.
2024-12-17 12:44:44 -08:00
Yuya Nishihara
168c7979fe working_copy: on snapshot, warn new large files and continue
I think this provides a better UX than refusing any operation due to large
files. Because untracked files won't be overwritten, it's usually safe to
continue operation ignoring the untracked files. One caveat is that new large
files can become tracked if files of the same name checked out. (see the test
case)

FWIW, the warning will be printed only once if watchman is enabled. If we use
the snapshot stats to print untracked paths in "jj status", this will be a
problem.

Closes #3616, #3912
2024-12-11 20:19:51 +09:00
Yuya Nishihara
f4fdc19d9e working_copy: plumbing to propagate untracked paths to caller 2024-12-11 20:19:51 +09:00
Yuya Nishihara
8caec186c1 local_working_copy: filter deleted files per directory (or job)
This greatly reduces the amount of paths to be sent over the channel and the
strings to be hashed.

Benchmark:
1. original (omitted)
2. per-directory spawn (previous patch)
3. per-directory deleted files (this patch)
4. shorter path comparison (omitted)

gecko-dev (~357k files, ~25k dirs)
```
% JJ_CONFIG=/dev/null hyperfine --sort command --warmup 3 --runs 30 ..
Benchmark 2: target/release-with-debug/jj-2 -R ~/mirrors/gecko-dev debug snapshot
  Time (mean ± σ):     710.7 ms ±   9.1 ms    [User: 3070.7 ms, System: 2142.6 ms]
  Range (min … max):   695.9 ms … 740.1 ms    30 runs

Benchmark 3: target/release-with-debug/jj-3 -R ~/mirrors/gecko-dev debug snapshot
  Time (mean ± σ):     480.1 ms ±   8.8 ms    [User: 3190.5 ms, System: 2127.2 ms]
  Range (min … max):   471.2 ms … 509.8 ms    30 runs

Relative speed comparison
        1.76 ±  0.03  target/release-with-debug/jj-2 -R ~/mirrors/gecko-dev debug snapshot
        1.19 ±  0.03  target/release-with-debug/jj-3 -R ~/mirrors/gecko-dev debug snapshot
```

linux (~87k files, ~6k dirs)
```
% JJ_CONFIG=/dev/null hyperfine --sort command --warmup 3 --runs 30 ..
Benchmark 2: target/release-with-debug/jj-2 -R ~/mirrors/linux debug snapshot
  Time (mean ± σ):     242.3 ms ±   3.3 ms    [User: 656.8 ms, System: 538.0 ms]
  Range (min … max):   236.9 ms … 252.3 ms    30 runs

Benchmark 3: target/release-with-debug/jj-3 -R ~/mirrors/linux debug snapshot
  Time (mean ± σ):     204.2 ms ±   3.0 ms    [User: 667.3 ms, System: 545.6 ms]
  Range (min … max):   197.1 ms … 209.2 ms    30 runs

Relative speed comparison
        1.27 ±  0.03  target/release-with-debug/jj-2 -R ~/mirrors/linux debug snapshot
        1.07 ±  0.02  target/release-with-debug/jj-3 -R ~/mirrors/linux debug snapshot
```

nixpkgs (~45k files, ~31k dirs)
```
% JJ_CONFIG=/dev/null hyperfine --sort command --warmup 3 --runs 30 ..
Benchmark 2: target/release-with-debug/jj-2 -R ~/mirrors/nixpkgs debug snapshot
  Time (mean ± σ):     190.7 ms ±   4.1 ms    [User: 859.3 ms, System: 881.1 ms]
  Range (min … max):   184.6 ms … 202.4 ms    30 runs

Benchmark 3: target/release-with-debug/jj-3 -R ~/mirrors/nixpkgs debug snapshot
  Time (mean ± σ):     173.3 ms ±   6.7 ms    [User: 899.4 ms, System: 889.0 ms]
  Range (min … max):   166.5 ms … 197.9 ms    30 runs

Relative speed comparison
        1.18 ±  0.03  target/release-with-debug/jj-2 -R ~/mirrors/nixpkgs debug snapshot
        1.07 ±  0.04  target/release-with-debug/jj-3 -R ~/mirrors/nixpkgs debug snapshot
```

git (~4.5k files, 0.2k dirs)
```
% JJ_CONFIG=/dev/null hyperfine --sort command --warmup 30 --runs 50 ..
Benchmark 2: target/release-with-debug/jj-2 -R ~/mirrors/git debug snapshot
  Time (mean ± σ):      30.6 ms ±   1.1 ms    [User: 33.8 ms, System: 39.0 ms]
  Range (min … max):    29.0 ms …  35.0 ms    50 runs

Benchmark 3: target/release-with-debug/jj-3 -R ~/mirrors/git debug snapshot
  Time (mean ± σ):      28.8 ms ±   1.0 ms    [User: 33.0 ms, System: 37.6 ms]
  Range (min … max):    26.8 ms …  31.3 ms    50 runs

Relative speed comparison
        1.06 ±  0.05  target/release-with-debug/jj-2 -R ~/mirrors/git debug snapshot
        1.00          target/release-with-debug/jj-3 -R ~/mirrors/git debug snapshot
```
2024-12-10 10:51:04 +09:00
Yuya Nishihara
99d8703d3b local_working_copy: spawn snapshot job per directory with file count threshold
This change basically means two things:
 a. a directory scan isn't split into too many small jobs, and
 b. a directory scan isn't blocked by recursive visit_directory() calls.
Before, small jobs were created at each recursion depth, so there were silent
time slice before these jobs started producing work.

I don't know if this mitigates the issue #4508, but it's slightly faster on my
Linux machine.

matcher.visit(dir) is moved to caller because it's silly to spawn an empty job.
TreeState::snapshot() already checks that for the root path.

Benchmark:
1. original
2. per-directory spawn (this patch)
3. per-directory deleted files (omitted)
4. shorter path comparison (omitted)

gecko-dev (~357k files, ~25k dirs)
```
% JJ_CONFIG=/dev/null hyperfine --sort command --warmup 3 --runs 30 ..
Benchmark 1: target/release-with-debug/jj-1 -R ~/mirrors/gecko-dev debug snapshot
  Time (mean ± σ):     764.9 ms ±  16.7 ms    [User: 3274.7 ms, System: 2183.3 ms]
  Range (min … max):   731.9 ms … 814.2 ms    30 runs

Benchmark 2: target/release-with-debug/jj-2 -R ~/mirrors/gecko-dev debug snapshot
  Time (mean ± σ):     710.7 ms ±   9.1 ms    [User: 3070.7 ms, System: 2142.6 ms]
  Range (min … max):   695.9 ms … 740.1 ms    30 runs

Relative speed comparison
        1.89 ±  0.05  target/release-with-debug/jj-1 -R ~/mirrors/gecko-dev debug snapshot
        1.76 ±  0.03  target/release-with-debug/jj-2 -R ~/mirrors/gecko-dev debug snapshot
```

linux (~87k files, ~6k dirs)
```
% JJ_CONFIG=/dev/null hyperfine --sort command --warmup 3 --runs 30 ..
Benchmark 1: target/release-with-debug/jj-1 -R ~/mirrors/linux debug snapshot
  Time (mean ± σ):     268.2 ms ±  11.3 ms    [User: 636.6 ms, System: 518.5 ms]
  Range (min … max):   247.5 ms … 295.2 ms    30 runs

Benchmark 2: target/release-with-debug/jj-2 -R ~/mirrors/linux debug snapshot
  Time (mean ± σ):     242.3 ms ±   3.3 ms    [User: 656.8 ms, System: 538.0 ms]
  Range (min … max):   236.9 ms … 252.3 ms    30 runs

Relative speed comparison
        1.40 ±  0.06  target/release-with-debug/jj-1 -R ~/mirrors/linux debug snapshot
        1.27 ±  0.03  target/release-with-debug/jj-2 -R ~/mirrors/linux debug snapshot
```

nixpkgs (~45k files, ~31k dirs)
```
% JJ_CONFIG=/dev/null hyperfine --sort command --warmup 3 --runs 30 ..
Benchmark 1: target/release-with-debug/jj-1 -R ~/mirrors/nixpkgs debug snapshot
  Time (mean ± σ):     201.0 ms ±   8.5 ms    [User: 929.3 ms, System: 917.6 ms]
  Range (min … max):   170.3 ms … 218.5 ms    30 runs

Benchmark 2: target/release-with-debug/jj-2 -R ~/mirrors/nixpkgs debug snapshot
  Time (mean ± σ):     190.7 ms ±   4.1 ms    [User: 859.3 ms, System: 881.1 ms]
  Range (min … max):   184.6 ms … 202.4 ms    30 runs

Relative speed comparison
        1.24 ±  0.06  target/release-with-debug/jj-1 -R ~/mirrors/nixpkgs debug snapshot
        1.18 ±  0.03  target/release-with-debug/jj-2 -R ~/mirrors/nixpkgs debug snapshot
```

git (~4.5k files, 0.2k dirs)
```
% JJ_CONFIG=/dev/null hyperfine --sort command --warmup 30 --runs 50 ..
Benchmark 1: target/release-with-debug/jj-1 -R ~/mirrors/git debug snapshot
  Time (mean ± σ):      30.3 ms ±   1.1 ms    [User: 40.5 ms, System: 39.4 ms]
  Range (min … max):    28.3 ms …  35.7 ms    50 runs

Benchmark 2: target/release-with-debug/jj-2 -R ~/mirrors/git debug snapshot
  Time (mean ± σ):      30.6 ms ±   1.1 ms    [User: 33.8 ms, System: 39.0 ms]
  Range (min … max):    29.0 ms …  35.0 ms    50 runs

Relative speed comparison
        1.05 ±  0.05  target/release-with-debug/jj-1 -R ~/mirrors/git debug snapshot
        1.06 ±  0.05  target/release-with-debug/jj-2 -R ~/mirrors/git debug snapshot
```

- CPU: 8-core AMD Ryzen 7 PRO 4750U with Radeon Graphics (-MT MCP-)
- speed/min/max: 1600/1400/1700 MHz Kernel: 6.11.10-amd64 x86_64
- Filesystem: ext4
2024-12-10 10:51:04 +09:00
Yuya Nishihara
da3c75b3cb config: add convenient method to insert value into layer, migrate callers
The current implementation is silly, which will be reimplemented to be using
toml_edit::DocumentMut. "jj config set" will probably be ported to this API.
2024-12-07 11:06:21 +09:00
Essien Ita Essien
694dfb8722 jj-lib: abstract a lower-level api for fetching from git.
* GitFetch{} separates `fetch()` from `import_refs()`.
* This allows more control over the stages of the fetch so multiple fetches
  can happen before `import_refs` resolves the changes in the local jj repo.
* Implement `git::fetch` in terms of the new api.
* Add a test case for initial fetch from a repo without `HEAD` set. This tests
  the default branch retrieving behaviour.

Issue: #4923
2024-12-01 12:22:09 +00:00
Yuya Nishihara
4931b2ba04 merged_tree: remove redundant .clone() from TreeDiffStreamImpl::new() 2024-11-30 10:20:43 +09:00
Yuya Nishihara
c741e3db39 merged_tree: use Merge<Tree> to represent pending trees in TreeDiffStreamImpl
This seems a slightly better in that MergedTree no longer represent a subtree.
2024-11-30 10:20:43 +09:00
Yuya Nishihara
a3ca6c6f46 tests: do not pass in commit objects loaded from different store
For the same reason as 37c41d0eaf4d.
2024-11-30 10:20:43 +09:00
Yuya Nishihara
ecf25a1d9b tests: extract helper that instantiates StackedConfig with commit timestamp 2024-11-30 10:01:05 +09:00
Yuya Nishihara
da01734639 settings: own StackedConfig by UserSettings, migrate tests to use config layer
UserSettings::get_*() will be changed to look up a merged value from
StackedConfig, not from a merged config::Value. This will help migrate away
from the config crate.

Not all tests are ported to ConfigLayer::parse() because it seemed a bit odd
to format!() a TOML document and parse it to build a table of configuration
variables.
2024-11-30 10:01:05 +09:00
Martin von Zweigbergk
a5690beab5 test_merged_tree: avoid a temporary lifetime extension
I think it's just clearer to assign the owned value to the variable
than to assign a reference to a temporary value.
2024-11-27 18:53:28 -08:00
Martin von Zweigbergk
409be2e1c4 store: make get_tree() functions take owned repo path
The function needs an owned value, so we might as well pass it one and
avoid a few clone calls.
2024-11-27 18:53:28 -08:00
Martin von Zweigbergk
10c90a5099 merged_tree: propagate errors from conflict iterator 2024-11-23 13:53:04 -08:00
Scott Taylor
26f5d6150c conflicts: add "git" conflict marker style
Adds a new "git" conflict marker style option. This option matches Git's
"diff3" conflict style, allowing these conflicts to be parsed by some
external tools that don't support JJ-style conflicts. If a conflict has
more than 2 sides, then it falls back to the similar "snapshot" conflict
marker style.

The conflict parsing code now supports parsing Git-style conflict
markers in addition to the normal JJ-style conflict markers, regardless
of the conflict marker style setting. This has the benefit of allowing
the user to switch the conflict marker style while they already have
conflicts checked out, and their old conflicts will still be parsed
correctly.

Example of "git" conflict markers:

```
<<<<<<< Side #1 (Conflict 1 of 1)
fn example(word: String) {
    println!("word is {word}");
||||||| Base
fn example(w: String) {
    println!("word is {w}");
=======
fn example(w: &str) {
    println!("word is {w}");
>>>>>>> Side #2 (Conflict 1 of 1 ends)
}
```
2024-11-23 08:28:47 -06:00
Scott Taylor
d2b06b9cf9 conflicts: add "snapshot" conflict marker style
Adds a new "snapshot" conflict marker style which returns a series of
snapshots, similar to Git's "diff3" conflict style. The "snapshot"
option uses a subset of the conflict hunk headers as the "diff" option
(it just doesn't use "%%%%%%%"), meaning that the two options are
trivially compatible with each other (i.e. a file materialized with
"snapshot" can be parsed with "diff" and vice versa).

Example of "snapshot" conflict markers:

```
<<<<<<< Conflict 1 of 1
+++++++ Contents of side #1
fn example(word: String) {
    println!("word is {word}");
------- Contents of base
fn example(w: String) {
    println!("word is {w}");
+++++++ Contents of side #2
fn example(w: &str) {
    println!("word is {w}");
>>>>>>> Conflict 1 of 1 ends
}
```
2024-11-23 08:28:47 -06:00
Scott Taylor
e5cb9f94f6 conflicts: add "ui.conflict-marker-style" config
Adds a new "ui.conflict-marker-style" config option. The "diff" option
is the default jj-style conflict markers with a snapshot and a series of
diffs to apply to the snapshot. New conflict marker style options will
be added in later commits.

The majority of the changes in this commit are from passing the config
option down to the code that materializes the conflicts.

Example of "diff" conflict markers:

```
<<<<<<< Conflict 1 of 1
+++++++ Contents of side #1
fn example(word: String) {
    println!("word is {word}");
%%%%%%% Changes from base to side #2
-fn example(w: String) {
+fn example(w: &str) {
     println!("word is {w}");
>>>>>>> Conflict 1 of 1 ends
}
```
2024-11-23 08:28:47 -06:00
Scott Taylor
6e959fa12c conflicts: allow stripped trailing whitespace in diffs
Some editors strip trailing whitespace on save, which breaks any diffs
which have context lines, since the parsing function expects them to
start with a space. There's no visual difference between " \n" and "\n",
so it seems reasonable to accept both.
2024-11-22 18:00:05 -06:00
Scott Taylor
efacbcbd45 conflicts: demo failed parse of diff with empty line 2024-11-22 18:00:05 -06:00
Scott Taylor
9674852dc7 conflicts: allow CRLF line endings on conflict markers
Currently, conflict markers ending in CRLF line endings aren't allowed.
I don't see any reason why we should reject them, since some
editors/tools might produce CRLF automatically on Windows when saving
files, which would break the conflicts otherwise.
2024-11-22 18:00:05 -06:00
Scott Taylor
ee7f829d4c conflicts: demo failed parse of markers with CRLF 2024-11-22 18:00:05 -06:00
Yuya Nishihara
59a79fdcc0 conflicts: extract materialize_merge_result_to_bytes() helper
We have many callers of materialize_merge_result() who just want in-memory
buffer.
2024-11-21 10:50:37 +09:00
Yuya Nishihara
5cc0bd0950 rewrite: fix duplicated commits to be rebased onto destination
I believe this was an oversight. "jj duplicate" should duplicate commits (=
patches), not trees.

This patch adds a separate test file because test_rewrite.rs is pretty big, and
we'll probably want to migrate CLI tests to jj-lib.
2024-11-21 10:49:51 +09:00
Luke Randall
068fa0f37e revset: allow tags() to take a pattern for an argument
This makes it more consistent with `bookmarks()`.

Co-authored-by: Austin Seipp <aseipp@pobox.com>
2024-11-20 00:47:23 +00:00
Benjamin Tan
4db4f413a7 revset: add fork_point function
This can be used to find the fork point (best common ancestors) of a
revset with an arbitrary number of commits, which cannot be expressed
currently in the revset language.
2024-11-16 04:08:01 +08:00
Martin von Zweigbergk
de6da1a088 transaction: propagate errors from commit() 2024-11-13 23:05:24 -08:00
Yuya Nishihara
7be4904982 tests: fix flakiness in shallow Git repo test
This test reliably failed if I dropped tv_nsec part from statx().

Since we reload the repo now, several assertions get "fixed". I've added
index().has_id() test to clarify that it's still broken.
2024-11-12 20:27:51 +09:00