3762 Commits

Author SHA1 Message Date
Yuya Nishihara
a1fe2dc95e git: trim trailing space in sideband messages received from git subprocess
Since git CLI appends "        " to sideband messages to overwrite the previous
line, we need to undo that. Our GitSidebandProgressMessageWriter expects
original sideband messages.
2025-02-12 01:25:53 +00:00
Emily
77f54a267e git: enable subprocessing by default
Given the previously‐stated intention of making this default
for the 0.27 release, prepare for that decision ahead of time by
enabling subprocessing by default on trunk. This will help surface
any regressions and workflow incompatibilities and therefore give
us more information to decide whether to keep or revert this commit,
without inconveniencing any users who haven’t already opted in to
the bleeding edge.

Please feel free to revert without hesitation if any major issues
arise; this is not intended as a strong commitment to enable this
option for the next stable release if it turns out to not be ready. In
that case, it’s better that we learn that early on in the cycle,
rather than having to revert at the last minute or, worse, cutting
a stable release that we later find contains a serious regression.
2025-02-10 22:10:22 +00:00
Jacob Hayes
6aefb58ed8
sign: Support ~/ path expansion for allowed-signers
Fixes #5625
2025-02-10 16:50:39 +07:00
Evan Mesterhazy
aee05b9ea0 repo: Use thiserror #[from] with EditCommitError::RewriteRootCommit
See the discussion in #5623. This prevents future accidents if member variables
are added to RewriteRootCommit which need to be propagated to EditCommitError
during conversion.
2025-02-10 04:23:04 +00:00
Evan Mesterhazy
4422e8cea6 cli: Fix single-character string constant used as pattern clippy warning
For further information visit https://rust-lang.github.io/rust-clippy/master/index.html#single_char_pattern

Apparently this got moved to "pedantic" in 1.80, but I ran clippy 1.79 and saw
it. Seems harmless to fix though.

#cleanup
2025-02-09 13:23:41 -05:00
Evan Mesterhazy
6d6f2c6dec repo: Change name for unused closure argument
The previous name violated the "snake case" convention for variable names. I
suspect the intention was to use `RewriteRootCommit` as a type declaration, not
the argument name. Since the argument isn't used, we can use an underscore as
the name and leave the type declaration.

#cleanup
2025-02-08 15:25:44 +00:00
avmikhailov
a16555f9af index: Return Result from heads
there are many ways to implement `heads` for your custom backend:
one involves calling `self.evaluate_revset` with a proper revset. 
However, it can return an error, but `heads` does not allow it.

In our implementation of the index backend we do exactly the above 
and we ended up with several unwraps which we are trying to avoid 
as it could potentially crash our prod jobs.

P.S. The same logic applies to many methods in this trait, but I am doing one at a time.
2025-02-07 13:45:47 +01:00
Baltasar Dinis
0c446c9069 git: skip running pre-push hooks when subprocessing to match git2 behaviour 2025-02-06 22:43:06 +00:00
Yuya Nishihara
6574f12173 local_working_copy: scan directory entries excluded by auto-track pattern
Since we need to scan directory entries recursively in order to detect new
untracked paths, it doesn't make sense to reuse the .gitignore code path.

This change means all untracked file paths are listed in "jj status" even if
the whole directory is untracked. It might be a bit verbose, but should be
okay. Directories like node_modules should be excluded by .gitignore, not by
auto-track pattern.

Fixes #5389
2025-02-06 18:04:57 +09:00
Yuya Nishihara
1faea485ca git: enable sideband progress on subprocess push
The parsing function is adjusted in case stderr contained unparsed progress
messages.
2025-02-06 01:05:48 +00:00
Yuya Nishihara
e091172b63 git: force git subprocess to not localize error messages
Since we parse error messages, we need to disable translation at all.
2025-02-05 16:20:13 +09:00
Yuya Nishihara
e85a5c44cd rewrite: move description handling from squash_commits() to caller
This allows caller to reborrow tx.repo() to render description template. It
also matches the documented behavior.
2025-02-04 06:05:58 +00:00
Yuya Nishihara
7898eb9f82 git: split function that queries remote default branch, call only when needed
With git.subprocess = true, it's more important to skip unneeded remote
operations. Each remote command may involve user intervention if authentication
requires manual step.

This change also means that the remote connection is no longer reused in git2
impl. I think the added cost is acceptable. The git2 impl will hopefully be
removed soon, and the remote branch name is needed only when cloning new repo.
2025-02-03 03:43:11 +00:00
Yuya Nishihara
241b8873ba revset: escape symbols in "did you mean" hint
The hint should include expressions that can be copied to -r argument.
2025-02-03 01:31:04 +00:00
Yuya Nishihara
fc7ac4c0bf revset: extract helper that formats remote symbol
I'll add a few more callers.
2025-02-03 01:31:04 +00:00
Yuya Nishihara
5be25bc8cf parser: use backtick to quote name or expression in error message
I don't have any preference about quoting styles, but it looks weird if an
escaped symbol is surrounded by double quotes.
2025-02-03 01:31:04 +00:00
Yuya Nishihara
7e55dd2294 git: check remote existence without using git2::Repository 2025-02-01 11:04:38 +00:00
Yuya Nishihara
bba6f15dfe git: initialize subprocess context without using git2::Repository 2025-02-01 11:04:38 +00:00
Yuya Nishihara
d4023d873d git: instantiate GitSubprocessContext by push_updates() for consistency 2025-02-01 11:04:38 +00:00
Yuya Nishihara
8cad3ac95a git: extract GitFetchImpl that holds implementation-specific objects
This could be a trait object, but we only need a single fetch() function per
implementation.
2025-02-01 11:04:38 +00:00
Yuya Nishihara
6e3c7fcf2d git: extract inner fetch functions out of GitFetch struct
I'm going to add enum that holds either git2::Repository or subprocess context.
"&mut self" will be useless there.
2025-02-01 11:04:38 +00:00
Yuya Nishihara
658cd3e0fd git: resolve git2::Repository internally in GitFetch::new()
git2::Repository will be removed from the subprocess code path.
2025-02-01 11:04:38 +00:00
Yuya Nishihara
9bf29accc2 git: resolve git2::Repository internally in push_updates()
git2::Repository will be removed from the subprocess code path.
2025-02-01 11:04:38 +00:00
Yuya Nishihara
27e7672a3b git: store bad pattern object in GitFetchError::InvalidBranchPattern
This helps add From<GitFetchError> impl.
2025-02-01 11:04:38 +00:00
Yuya Nishihara
67bb2809ed git: test that local progress is emitted through callback 2025-02-01 03:30:39 +00:00
Baltasar Dinis
b35d503bf7 git: show fetch progress with git.subprocess = true 2025-02-01 01:46:31 +00:00
Scott Taylor
66808e5dc1 conflicts: use clearer wording for missing newline comment
The current comment uses `[noeol]`, which can be difficult to
understand. In this commit, the brackets are changed to parentheses to
make it clear that there is no semantic meaning to the comment, and the
wording is changed to be more clear to the user.
2025-01-31 23:38:42 +00:00
Scott Taylor
0b719332aa conflicts: don't mark missing newline in Git-style conflicts
Git doesn't show this information in their "diff3" style either, and it
looks a bit strange having two different bracketed sections of text next
to each other, so I think it would be better to remove it.
2025-01-31 23:38:42 +00:00
Martin von Zweigbergk
4ac970fcc5 cleanup: prefer MutableRepo::repo() over repo_mut()
When we don't need a mutable reference, we should be using `repo()`.
2025-01-31 18:03:01 +00:00
Martin von Zweigbergk
ef20b7b7ab revset: add a function for escaping a symbol
When we have e.g. a bookmark name or a remote name and we want to
produce a revset from it, we may need to quote and escape it. This
patch implements a function for that.

Perhaps we'll want to more generally be able to format a whole
`RevsetExpression` later.
2025-01-31 15:54:47 +00:00
Yuya Nishihara
b637e98127 git: enable sideband message callback globally, disable on fetch without tty
With this and "git --progress" PR #5519, remote git progress will be displayed.

The sideband callback is disabled in git2 code path if progress is not enabled.
If this were enabled, most "git fetch" tests would fail with git2 due to remote
progress messages. Since there's no git2 API to turn the remote progress off,
and the git2 code path is supposed to be phased out, I just inserted ad-hoc
workaround.
2025-01-31 01:47:34 +00:00
Yuya Nishihara
cdbfd6bc51 git: reimplement get_all_remotes() by using gix API 2025-01-31 00:52:23 +00:00
Yuya Nishihara
7a1cd6a0f6 git: factor out common get_git_backend/repo() helper, simplify error handling 2025-01-31 00:52:23 +00:00
Yuya Nishihara
d3f56c0acd git: forward "remote: " lines when they arrive
This patch adds thread-based implementation. On Unix, we can poll pipes without
using thread, but I don't think platform-dependent optimization is needed.
2025-01-30 07:06:42 +00:00
Baltasar Dinis
120b9cc766 git: forward remote messages to output 2025-01-28 21:56:04 +00:00
Baltasar Dinis
e8620c31ae git: describe codepaths that require git2 2025-01-28 18:29:01 +00:00
Baltasar Dinis
1be574c219 git: update jj git clone|fetch to use new GitFetch api directly.
* Make the new `GitFetch` api public.
* Move `git::fetch` to `lib/tests/test_git.rs` as `git_fetch`, to minimize
  churn in the tests. Update test call sites to use `git_fetch`
* Delete the `git::fetch` from `lib/src/git.rs`.
* Update `jj git clone` and `git_fetch` in `cli/src/git_utils.rs` to use
  the new api directly. Removing one redundant layer of indirection.
* This fixes #4920 as it first fetches from all remotes before `import_refs()`
  is called, so there is no race condition if the same commit is treated
  differently in different remotes specified in the same command.

Original commit by @essiene
2025-01-28 18:29:01 +00:00
Scott Taylor
66faeb4487 conflicts: show "noeol" state in conflict marker comment
This isn't strictly necessary since it doesn't affect parsing, but it
should make it more understandable for the user.
2025-01-27 22:59:06 +00:00
Scott Taylor
326c453064 conflicts: allow missing newlines at end of file
A missing newline would cause the following conflict markers to become
invalid when materializing the conflict, so we add a newline to prevent
this from happening. When parsing, if we parsed a conflict at the end of
the file which ended in a newline, we can check whether the file
actually did end in a newline, and then remove the newline if it didn't.
2025-01-27 22:59:06 +00:00
Scott Taylor
352807c050 conflicts: remove support for non-simplified file conflicts
File conflicts have been simplified during materialization since 0.19.0,
so it should be safe to remove support for unsimplified conflicts now.
This will make implementing the next commit easier, since it won't have
to support unsimplified conflicts.
2025-01-27 22:59:06 +00:00
Yuya Nishihara
9985dce0ab revset: inline reversed() iterator to caller
The implementation isn't specific to revset, and is short.
2025-01-27 01:35:36 +00:00
Scott Taylor
00069119d8 tests: keep user's $PATH while running jj commands
Currently, the Git subprocess tests only work on Linux due to a default
path being used for the `git` executable when `$PATH` is unset. This can
break if Git isn't installed at the expected path. Also, I believe it is
currently necessary to set the `$TEST_GIT_EXECUTABLE_PATH` environment
variable on Windows for tests to pass. Instead, we should use the user's
`$PATH` to locate the `git` executable, as well as any other executables
that are needed. This also makes `$TEST_GIT_EXECUTABLE_PATH` no longer
necessary, so it can be removed.
2025-01-26 17:16:59 +00:00
Bryce Berger
cbb743cfb5 config: add --when.command to scope resolution
Closes #5217

Motivating use case:

    [[--scope]]
    --when.command = ["log"]
    [--scope.ui]
    pager = "less"

This adds a new (optional) field to `--when` config conditions, to
inspect the current command.

`--when.commands` is a list of space-separated values that matches a
space-separated command. To be specific:

    --when.command = ["file"]        # matches `jj file show`, `jj file list`, etc
    --when.command = ["file show"]   # matches `jj file show`, but *NOT* `jj file list`
    --when.command = ["file", "log"] # matches `jj file` *OR* `jj log` (or subcommand of either)

When both `--when.commands` and `--when.repositories` are set, the
intersection is used.
2025-01-24 05:30:07 +00:00
bsdinis
35440ce1bd git: spawn a separate git process for network operations
Reasoning:

`jj` fails to push/fetch over ssh depending on the system.
Issue #4979 lists over 20 related issues on this and proposes spawning
a `git` subprocess for tasks related to the network (in fact, just push/fetch
are enough).

This PR implements this.

Implementation Details:

This PR implements shelling out to `git` via `std::process::Command`.
There are 2 sharp edges with the patch:
 - it relies on having to parse out git errors to match the error codes
   (and parsing git2's errors in one particular instance to match the
   error behaviour). This seems mostly unavoidable

 - to ensure matching behaviour with git2, the tests are maintained across the
   two implementations. This is done using test_case, as with the rest
   of the codebase

Testing:

Run the rust tests:
```
$ cargo test
```

Build:
```
$ cargo build
```

Clone a private repo:
```
$ path/to/jj git clone --config='git.subprocess=true' <REPO_SSH_URL>
```

Create new commit and push
```
$ echo "TEST" > this_is_a_test_file.txt
$ path/to/jj describe -m 'test commit'
$ path/to/jj git push --config='git.subprocess=true' -b <branch>
```


Issues Closed

With a grain of salt, but most of these problems should be fixed (or at least checked if they are fixed). They are the ones listed in #4979 .

SSH:
- https://github.com/jj-vcs/jj/issues/63
- https://github.com/jj-vcs/jj/issues/440
- https://github.com/jj-vcs/jj/issues/1455
- https://github.com/jj-vcs/jj/issues/1507
- https://github.com/jj-vcs/jj/issues/2931
- https://github.com/jj-vcs/jj/issues/2958
- https://github.com/jj-vcs/jj/issues/3322
- https://github.com/jj-vcs/jj/issues/4101
- https://github.com/jj-vcs/jj/issues/4333
- https://github.com/jj-vcs/jj/issues/4386
- https://github.com/jj-vcs/jj/issues/4488
- https://github.com/jj-vcs/jj/issues/4591
- https://github.com/jj-vcs/jj/issues/4802
- https://github.com/jj-vcs/jj/issues/4870
- https://github.com/jj-vcs/jj/issues/4937
- https://github.com/jj-vcs/jj/issues/4978
- https://github.com/jj-vcs/jj/issues/5120
- https://github.com/jj-vcs/jj/issues/5166

Clone/fetch/push/pull:
- https://github.com/jj-vcs/jj/issues/360
- https://github.com/jj-vcs/jj/issues/1278
- https://github.com/jj-vcs/jj/issues/1957
- https://github.com/jj-vcs/jj/issues/2295
- https://github.com/jj-vcs/jj/issues/3851
- https://github.com/jj-vcs/jj/issues/4177
- https://github.com/jj-vcs/jj/issues/4682
- https://github.com/jj-vcs/jj/issues/4719
- https://github.com/jj-vcs/jj/issues/4889
- https://github.com/jj-vcs/jj/discussions/5147
- https://github.com/jj-vcs/jj/issues/5238

Notable Holdouts:
 - Interactive HTTP authentication (https://github.com/jj-vcs/jj/issues/401, https://github.com/jj-vcs/jj/issues/469)
 - libssh2-sys dependency on windows problem (can only be removed if/when we get rid of libgit2): https://github.com/jj-vcs/jj/issues/3984
2025-01-23 16:50:53 +00:00
bsdinis
c773c0296f git: test git fetch with more than two remotes and branches
In some cases, there are non trivial codepaths for fetching multiple
branches explicitly. In particular, it might be the case that fetching
works for n = 2 but not n = 3.

This commit changes a cli test to have 3 remotes and 3 branches, and a
lib test with 3 branches, only one of them succeds.
2025-01-23 16:50:53 +00:00
bsdinis
717ae02f62 lib: add type support for refspecs
Fully qualified git refs (e.g., `+refs/head/main:refs/remotes/origin/main`)
are commonly used throughout the git glue code.

It is laborious and error prone to keep parsing refspecs when we need
only a segment of it.

This commits adds a type, `RefSpec`, which does exactly that
2025-01-23 16:50:53 +00:00
Yuya Nishihara
721aecb4cc config: add public ConfigLayer::ensure_table() function
It's not uncommon to insert new table at a certain path. This could be modelled
as set_table(name, new_table), but the caller might want to merge items with
the existing table if any. This function can be used for both cases.
2025-01-22 01:01:04 +00:00
Yuya Nishihara
d9e0fb7974 config: add migration type that runs user-specified function
This was needed in order to migrate fix.tool-command to [fix.tools] table. We
don't have other use cases right now, but there will be something in future.
2025-01-22 01:01:04 +00:00
Yuya Nishihara
ffaaf89f05 config: add migration type that renames and updates value
This will be used in order to migrate boolean value to enum, for example.
2025-01-22 01:01:04 +00:00
Yuya Nishihara
9d77ad5594 config: add separate type for migration error, attach source path
Migration can fail because of unexpected value type, etc.
2025-01-22 01:01:04 +00:00