228 Commits

Author SHA1 Message Date
Yuya Nishihara
658cd3e0fd git: resolve git2::Repository internally in GitFetch::new()
git2::Repository will be removed from the subprocess code path.
2025-02-01 11:04:38 +00:00
Yuya Nishihara
9bf29accc2 git: resolve git2::Repository internally in push_updates()
git2::Repository will be removed from the subprocess code path.
2025-02-01 11:04:38 +00:00
Martin von Zweigbergk
4ac970fcc5 cleanup: prefer MutableRepo::repo() over repo_mut()
When we don't need a mutable reference, we should be using `repo()`.
2025-01-31 18:03:01 +00:00
Baltasar Dinis
1be574c219 git: update jj git clone|fetch to use new GitFetch api directly.
* Make the new `GitFetch` api public.
* Move `git::fetch` to `lib/tests/test_git.rs` as `git_fetch`, to minimize
  churn in the tests. Update test call sites to use `git_fetch`
* Delete the `git::fetch` from `lib/src/git.rs`.
* Update `jj git clone` and `git_fetch` in `cli/src/git_utils.rs` to use
  the new api directly. Removing one redundant layer of indirection.
* This fixes #4920 as it first fetches from all remotes before `import_refs()`
  is called, so there is no race condition if the same commit is treated
  differently in different remotes specified in the same command.

Original commit by @essiene
2025-01-28 18:29:01 +00:00
Scott Taylor
00069119d8 tests: keep user's $PATH while running jj commands
Currently, the Git subprocess tests only work on Linux due to a default
path being used for the `git` executable when `$PATH` is unset. This can
break if Git isn't installed at the expected path. Also, I believe it is
currently necessary to set the `$TEST_GIT_EXECUTABLE_PATH` environment
variable on Windows for tests to pass. Instead, we should use the user's
`$PATH` to locate the `git` executable, as well as any other executables
that are needed. This also makes `$TEST_GIT_EXECUTABLE_PATH` no longer
necessary, so it can be removed.
2025-01-26 17:16:59 +00:00
bsdinis
35440ce1bd git: spawn a separate git process for network operations
Reasoning:

`jj` fails to push/fetch over ssh depending on the system.
Issue #4979 lists over 20 related issues on this and proposes spawning
a `git` subprocess for tasks related to the network (in fact, just push/fetch
are enough).

This PR implements this.

Implementation Details:

This PR implements shelling out to `git` via `std::process::Command`.
There are 2 sharp edges with the patch:
 - it relies on having to parse out git errors to match the error codes
   (and parsing git2's errors in one particular instance to match the
   error behaviour). This seems mostly unavoidable

 - to ensure matching behaviour with git2, the tests are maintained across the
   two implementations. This is done using test_case, as with the rest
   of the codebase

Testing:

Run the rust tests:
```
$ cargo test
```

Build:
```
$ cargo build
```

Clone a private repo:
```
$ path/to/jj git clone --config='git.subprocess=true' <REPO_SSH_URL>
```

Create new commit and push
```
$ echo "TEST" > this_is_a_test_file.txt
$ path/to/jj describe -m 'test commit'
$ path/to/jj git push --config='git.subprocess=true' -b <branch>
```


Issues Closed

With a grain of salt, but most of these problems should be fixed (or at least checked if they are fixed). They are the ones listed in #4979 .

SSH:
- https://github.com/jj-vcs/jj/issues/63
- https://github.com/jj-vcs/jj/issues/440
- https://github.com/jj-vcs/jj/issues/1455
- https://github.com/jj-vcs/jj/issues/1507
- https://github.com/jj-vcs/jj/issues/2931
- https://github.com/jj-vcs/jj/issues/2958
- https://github.com/jj-vcs/jj/issues/3322
- https://github.com/jj-vcs/jj/issues/4101
- https://github.com/jj-vcs/jj/issues/4333
- https://github.com/jj-vcs/jj/issues/4386
- https://github.com/jj-vcs/jj/issues/4488
- https://github.com/jj-vcs/jj/issues/4591
- https://github.com/jj-vcs/jj/issues/4802
- https://github.com/jj-vcs/jj/issues/4870
- https://github.com/jj-vcs/jj/issues/4937
- https://github.com/jj-vcs/jj/issues/4978
- https://github.com/jj-vcs/jj/issues/5120
- https://github.com/jj-vcs/jj/issues/5166

Clone/fetch/push/pull:
- https://github.com/jj-vcs/jj/issues/360
- https://github.com/jj-vcs/jj/issues/1278
- https://github.com/jj-vcs/jj/issues/1957
- https://github.com/jj-vcs/jj/issues/2295
- https://github.com/jj-vcs/jj/issues/3851
- https://github.com/jj-vcs/jj/issues/4177
- https://github.com/jj-vcs/jj/issues/4682
- https://github.com/jj-vcs/jj/issues/4719
- https://github.com/jj-vcs/jj/issues/4889
- https://github.com/jj-vcs/jj/discussions/5147
- https://github.com/jj-vcs/jj/issues/5238

Notable Holdouts:
 - Interactive HTTP authentication (https://github.com/jj-vcs/jj/issues/401, https://github.com/jj-vcs/jj/issues/469)
 - libssh2-sys dependency on windows problem (can only be removed if/when we get rid of libgit2): https://github.com/jj-vcs/jj/issues/3984
2025-01-23 16:50:53 +00:00
bsdinis
c773c0296f git: test git fetch with more than two remotes and branches
In some cases, there are non trivial codepaths for fetching multiple
branches explicitly. In particular, it might be the case that fetching
works for n = 2 but not n = 3.

This commit changes a cli test to have 3 remotes and 3 branches, and a
lib test with 3 branches, only one of them succeds.
2025-01-23 16:50:53 +00:00
Yuya Nishihara
5d1f3d006d repo: pass &Commit to record_abandoned_commit() to simplify error handling 2025-01-17 00:28:25 +00:00
Yuya Nishihara
f14a6e6f19 git: remove PartialEq requirement from GitPushError
Non-trivial error types usually don't implement Eq/PartialEq.
2025-01-10 01:16:33 +00:00
Scott Taylor
42b390bbc4 git: use merged parent tree for git index
Instead of setting the index to match the tree of HEAD, we now set the
index to the merged parent tree of the working copy commit. This means
that if you edit a merge commit, it will make the Git index look like it
would in the middle of a `git merge` operation (with all of the
successfully-merged files staged in the index).

If there are any 2-sided conflicts in the merged parent tree, then they
will be added to the index as conflicts. Since Git doesn't support
conflicts with more than 2 sides, many-sided conflicts are staged as the
first side of the conflict. The following commit will improve this.
2025-01-06 19:17:51 -06:00
Scott Taylor
9cc8b35251 git: use gix instead of git2 to update index
This will give us more fine-grained control over what files we put in
the index, allowing us to create conflicted index states. We also still
need to use git2 to clean up the merge/rebase state, since gix doesn't
have any function for this currently.
2025-01-06 19:17:51 -06:00
Scott Taylor
8e3ec9c58c git: add tests for index in colocated repo
These tests should still pass after we switch to gix for resetting the
index, so we need to make sure they don't rely on the cached index from
the `git2::Repository` instance.
2025-01-06 19:17:51 -06:00
Yuya Nishihara
cff73841ed repo: remove &UserSettings argument from new/rewrite_commit(), use self.settings 2024-12-31 10:51:57 +09:00
Yuya Nishihara
14b52205fb repo: remove &UserSettings argument from start_transaction(), use self.settings 2024-12-31 10:51:57 +09:00
Martin von Zweigbergk
b836e0ae95 docs/cli: update URLs to from martinvonz user to jj-vcs org
We just migrated to the jj-vcs GitHub org, so we should point to the
new GitHub URLs.
2024-12-17 12:44:44 -08:00
Essien Ita Essien
694dfb8722 jj-lib: abstract a lower-level api for fetching from git.
* GitFetch{} separates `fetch()` from `import_refs()`.
* This allows more control over the stages of the fetch so multiple fetches
  can happen before `import_refs` resolves the changes in the local jj repo.
* Implement `git::fetch` in terms of the new api.
* Add a test case for initial fetch from a repo without `HEAD` set. This tests
  the default branch retrieving behaviour.

Issue: #4923
2024-12-01 12:22:09 +00:00
Martin von Zweigbergk
de6da1a088 transaction: propagate errors from commit() 2024-11-13 23:05:24 -08:00
Yuya Nishihara
7be4904982 tests: fix flakiness in shallow Git repo test
This test reliably failed if I dropped tv_nsec part from statx().

Since we reload the repo now, several assertions get "fixed". I've added
index().has_id() test to clarify that it's still broken.
2024-11-12 20:27:51 +09:00
dploch
41631bc0e6 test_git: fix some clippy ref errors 2024-11-08 13:59:37 -05:00
Yuya Nishihara
739bf8decf repo_path: add stub for checked to_fs_path(), rename unchecked functions
I'm going to add "checked" version of to_fs_path(), but all callers can't be
migrated to it. For example, an error message should be produced even if the
path is malformed.

This patch also adds error variants to propagate InvalidRepoPathError. They
don't use ::Other { .. } so the errors can be distinguished in tests.
2024-11-06 15:03:41 -08:00
Yuya Nishihara
d4786a3256 testutils: move load_repo_at_head() to TestEnvironment
It will depend on the TestBackendData mapping.
2024-11-02 08:39:02 +09:00
Lukas Wirth
9f16419202 git clone: Add depth argument 2024-10-14 20:01:08 +02:00
Lukas Wirth
802e3db27e git_backend: Support shallow git repositories 2024-10-14 20:01:08 +02:00
Martin von Zweigbergk
1aa2aec141 bookmarks: update some leftover uses of the word "branch" 2024-09-11 19:19:31 -07:00
Philip Metzger
d9c68e08b1 everything: Rename branches to bookmarks
Jujutsu's branches do not behave like Git branches, which is a major
hurdle for people adopting it from Git. They rather behave like
Mercurial's (hg) bookmarks. 

We've had multiple discussions about it in the last ~1.5 years about this rename in the Discord, 
where multiple people agreed that this _false_ familiarity does not help anyone. Initially we were 
reluctant to do it but overtime, more and more users agreed that `bookmark` was a better for name 
the current mechanism. This may be hard break for current `jj branch` users, but it will immensly 
help Jujutsu's future, by defining it as our first own term. The `[experimental-moving-branches]` 
config option is currently left alone, to force not another large config update for
users, since the last time this happened was when `jj log -T show` was removed, which immediately 
resulted in breaking users and introduced soft deprecations.

This name change will also make it easier to introduce Topics (#3402) as _topological branches_ 
with a easier model. 

This was mostly done via LSP, ripgrep and sed and a whole bunch of manual changes either from
me being lazy or thankfully pointed out by reviewers.
2024-09-11 18:54:45 +02:00
Martin von Zweigbergk
19f383ffdd tests: avoid ReadonlyRepo::repo_path() 2024-09-07 12:20:02 -07:00
Martin von Zweigbergk
8d090628c3 transaction: rename mut_repo() to idiomatic repo_mut()
We had both `repo()` and `mut_repo()` on `Transaction` and I think it
was easy to get confused and think that the former returned a
`&ReadonlyRepo` but both of them actually return a reference to
`MutableRepo` (the latter obviously returns a mutable reference). I
hope that renaming to the more idiomatic `repo_mut()` will help
clarify.

We could instead have renamed them to `mut_repo()` and
`mut_repo_mut()` but that seemed unnecessarily long. It would better
match the `mut_repo` variables we typically use, though.
2024-09-07 10:51:43 -07:00
Martin von Zweigbergk
cc15ecf7c7 op log: change "resolve concurrent" to "reconcile divergent"
"Concurrent" operations are not necessarily actually concurrent, so
"divergent" seems like a better name. And "reconcile" seems like a
better term for merging them, though we also sometimes use "merge".
2024-08-30 21:56:11 -07:00
Matt Kulukundis
8ead72e99f formatting only: switch to Item level import ganularity 2024-08-22 14:52:54 -04:00
Yuya Nishihara
bafb357209 git: on abandoning unreachable commits, don't count HEAD ref
This basically reverts 20eb9ecec10f "git: don't abandon HEAD commit when it
loses a branch." I think the new behavior is more consistent because the Git
HEAD is equivalent to @- in jj, so it shouldn't be considered a named ref.

Note that we've made old HEAD branch not considered at 92cfffd84377 "git: on
external HEAD move, do not abandon old branch."

#4108
2024-07-24 21:22:26 +09:00
Yuya Nishihara
c240313c4b view: remove has_branch() which is called only from tests
Since we've split (local, remotes) branches to (locals, remotes { branches }),
.has_branch() API no longer makes much sense. Callers often need to check if
a remote branch is tracked.
2024-06-28 10:29:06 +09:00
Yuya Nishihara
5e7cb3435e git: unset unborn HEAD ref on export
Otherwise, newly created default branch would be re-imported as a new Git HEAD.
This could be addressed by cmd_git_init(), but the same situation can be
crafted by using "git checkout -b".
2024-06-01 11:01:16 +09:00
Ilya Grigoriev
12081166f6 test_git.rs: delete now-useless test 2024-05-28 21:38:26 -07:00
Ilya Grigoriev
777da99533 jj git push: always force-push, all safety logic now in push_negotiation
This should be a no-op, though that is not necessarily obvious in corner
cases.

Note that libgit2 already performs the push negotiation even when
pushing without force (without `+` in the refspec).
2024-05-28 21:38:26 -07:00
Ilya Grigoriev
8d3dd17b51 jj git push: safety checks in push negotiation, "force-with-lease"
As explained in the commit, our logic is a bit more complicated than
that of `git push --force-with-lease`. This is to match the behavior of
`jj git fetch` and branch conflict resolution rules.
2024-05-28 21:38:26 -07:00
Ilya Grigoriev
b0306eb742 jj git push: no-op push negotiation, plumb data to it
Hopefully, splitting the no-op portion will make the following commits
easier to review.
2024-05-28 21:38:26 -07:00
Ilya Grigoriev
b54d962d58 test_git.rs: add two more commits to PushSetup, rename them
The new names are a bit clearer. Soem tests already use a
`sideways_commit`, `parent_of_main_commit` will be needed in
an upcoming test.
2024-05-28 21:38:26 -07:00
mlcui
a075a5c6ca git: Only reset Git index if non-empty
In chromium/src.git, this gives an approximate ~0.83s speedup for
commands if the Git index is empty, and a ~0.14s slowdown if the Git
index is non-empty.

As most users using jj will likely be using jj to write to a colocated
repo - and therefore avoid modifying the Git index - this should be a
general speedup for most colocated checkouts.
2024-05-27 11:35:13 +10:00
Yuya Nishihara
aaa2025dfc git: on fetch, pin visible untracked remote refs
This implements the other workaround described in 57167cefda93 "git: on
import_refs(), don't abandon ancestors of newly fetched refs":

> I think there are two ways to fix the problem:
>  a. pin non-tracking remote branches just like local refs
>  b. pin newly fetched refs in addition to local refs
> This patch implements (b) because it's simpler and more obvious that the
> fetched commits would never be abandoned immediately.

The idea of (a) is that untracked remote branches are independent read-only
refs, and read-only branches shouldn't be rewritten implicitly. Once the
branch gets rewritten or abandoned by user, these remote refs will be hidden,
and won't be pinned anymore.

Since (a) effectively supersedes (b), this patch also removes the original
workaround.

Fixes #3495
2024-04-14 11:38:21 +09:00
Yuya Nishihara
5eea88d26a tests: fix concurrent git read/write test to retry on ref lock contention
Apparently, gix has 100ms timeout. Since this test tries to create contended
situation, it's possible that the ref lock can't be acquired. I've added
upper bound to the retry loop at b37293fa683c "tests: add upper bound to
test_concurrent_read_write_commit() loop", so ignoring arbitrary errors
should be okay.

The problem can be reproduced on my Linux machine by inserting 10ms sleep() to
gix and increasing the concurrency.

Fixes #3069
2024-02-17 15:09:27 +09:00
Yuya Nishihara
fc114ef217 git: extract Git HEAD handling bits from import_some_refs()
I'm going to make WorkspaceCommandHelper::maybe_snapshot() snapshot the working
copy before importing refs. git::import_some_refs() can rebase the working copy
branch and therefore @ can be moved. git::import_head() doesn't, and it should
be invoked before snapshotting.

git::import_head() is inserted to some of the git:import_refs() callers where
HEAD seems to matter. I feel it's a bit odd that the HEAD ref is imported to
non-colocated repo, but "jj init --git-repo" relies on that, and I think the
existence of HEAD@git is harmless. It's merely a ref to the revision checked
out somewhere else.
2024-01-27 00:01:59 +09:00
Yuya Nishihara
d5a98df046 git_backend: teach "format.tree-level-conflicts" config by constructor
Since GitBackend constructors now depend on &UserSettings, it makes sense to
initialize the formatting options there.
2024-01-10 08:57:51 +09:00
Yuya Nishihara
fa5e40719c object_id: extract ObjectId trait and macros to separate module
I'm going to add a prefix resolution method to OpStore, but OpStore is
unrelated to the index. I think ObjectId, HexPrefix, and PrefixResolution can
be extracted to this module.
2024-01-05 10:20:57 +09:00
Yuya Nishihara
55b4f69fb6 repo: propagate store error from add_heads() 2023-12-24 00:22:30 +09:00
Yuya Nishihara
6971ec239a tests: set git_settings.auto_local_branch where it matters 2023-12-17 08:30:24 +09:00
Martin von Zweigbergk
60fae3114e transaction: take description at end instead of start
It seems better to have the caller pass the transaction description
when we finish the transaction than when we start it. That way we have
all the information we want to include more readily available.
2023-12-13 08:12:49 -08:00
Yuya Nishihara
2d76907048 git: unimplement PartialEq on FailedRefExportReason
Gitoxide errors don't implement PartialEq. We could instead stringify the
errors, but there aren't many callers who expect FailedRefExportReason to
be comparable.
2023-12-09 15:18:19 +09:00
Yuya Nishihara
9f8831e825 git: unimplement PartialEq on GitExportError
Gitoxide errors don't implement PartialEq, and I don't think it makes sense
to test equality of InternalGitError objects.
2023-12-09 15:18:19 +09:00
Yuya Nishihara
a77eed648b git: have export_refs() obtain git2::Repository instance from store 2023-12-09 15:18:19 +09:00
Yuya Nishihara
77c811163f tests: make sure to specify external git repository path including ".git" 2023-12-07 08:43:49 +09:00