The first thing I would try is disabling repository size reporting and the activity
graph:

web.showRepositorySizes = false
web.generateActivityGraph = false

If those don't help (especially size), I would build from master and play with JGit's
runtime parameters which have been exposed as settings (issue 389).

Actually, maybe you should just skip to building from master.

Thanks for quick reply.

>web.showRepositorySizes = false
>web.generateActivityGraph = false

did not help

Have you built from master and tweaked the JGit parameters?

nope, used release 0.9.3 and tweaked gitblit.properties
what Jgit params should be tweaked and where?
Thanks

To tweak JGit's parameters you will have to build from master; this is a new feature
in the upcoming 1.0.0 release.

1.0.0 release is imminent - Thursday or Friday - in case you have not yet built from
source.

Hi, I will continue with Mike integration of gitBlit in our development environment.

First of all, I've upgraded version of gitBlit from 0.9.3 to 1.0.0.
When I start the gitBlit  the process hang, I see it in the system call:
#strace -p 5072
Process 5072 attached - interrupt to quit
futex(0x7f810161d9d0, FUTEX_WAIT, 5073, NULL
And browser is hanging too.

I try to twink JGit params and set them x4. After restarting I see same problem, process
of java hanging and no system calls.

Do you have some recommendation about gitBlit tuning for "big dev environments ?"


Thank you. 

Sorry, I don't have any recommendations/experience with big dev environments; they were
not my focus.

I think you should clone the Gitblit source and run it in Eclipse to diagnose where
the trouble lies.  Otherwise we are just guessing.

Hi All,
I found the root issue of our environment.

 We have more then hundred git repository on one volume, few are in root of git repository
( ../../git/some_repo.git ) , but more in subfolder ( ../../git/some_folder/some_repo.git
or ../../git/some_folder/../some_repo.git ).
When I start gitblit with git.searchRepositoriesSubfolders = false; gitblit starts
to work after 1 min. But when git.searchRepositoriesSubfolders = true, it isn't start
after 4 hours.

How I can limit gitblit searching of repositories, for example not deeper then 3 folders,
under git.repositoriesFolder ?

2-nd issue, gitblit starts to search repos in invisible folder: .snapshot. Of course
I don't have links to files under .snapshot and this folder is from NetApp.
How I can exclude the .snapshot from git repository searching.

Thank you.

At the moment you will have to hack on Gitblit to support either of these scenarios.

Hi James,
I attach the patch, that fix issue of "Excluded folders" and limitation of searching
for repositories under subfolders. Thanks to Mike for the patch.

Delimiter for "excluded folders" is a space.
For using those features, required to add 2 parameters to gitblit.properties, example
:
git.repositoriesFolderExclude = .snapshot
git.repositoriesFolderDepth = 2

-------------------------------------------------------------------
Now we have other issue, when I click on "repositories" link, server gitblit starts
anew scanning of git repositories tree, and the answer can be derived in 3-5 min. and
every clicking on this link and answer, take long time.

We try to solve this issue, and add some cache to gitblit. Can you provide me, which
java class calls when clicking on "repositories" link in the top.


Thank you.

3-5 mins?!  That is crazy.  Are you sure you have disabled web.showRepositorySizes??

Take a look at GitBlit.getRepositoryModels(UserModel user).

I would write a unit test that calls JGitUtils.getRepositoryList and time it's execution
to get an idea if the bottleneck is locating the repositories on disk.

Have you adjusted the JGit parameters according to the 1.0.0 docs?

Thanks for the contribution.  I have committed a variation of your suggested patch and
will be in the next release.

The attached patch does not actually exclude fragment matches, I believe it only excludes
exact matches.  I usually steer clear of string.matches("") because it's operation
is counter-intuitive.

Keep me informed on what you discover concerning the performance bottleneck.

btw, String.matches() does use of regex for matches, so it should be fine.
http://www.exampledepot.com/egs/java.lang/hassubstr.html

I think, performance-wise, the following code at JGitUtils.java:322-328 is redundant
and introduces performance problem: 

1. it will compile same exclude list for every recursion call - you can compile it
once (in line 285) and pass to the recursive function or use String.matches() which
should be optimal.

2. Please keep in mind the performance/scalability issues of this code when number
of subfolders of repository root dir can be big.

my 2 cents :)

I agree, moving pattern compilation to line 285 is an improvement.

But I don't see how you can claim String.matches() is optimal or an improvement since
it would execute this for every folder at every level:

String.matches() {
  return Pattern.matches(regex, this) {
    Pattern p = Pattern.compile(regex);
    Matcher m = p.matcher(input);
    return m.matches();
  }
}

Thanks for the expressions link.  I'm not a regex guy.  I wanted to write something
like: (jgit)+ to exclude any folder with jgit in the name.  Matches will not match
that, this is why I went with find.  I will review that further.

Have you discovered your real problem yet?

I hate to be a broken record but... have you discovered anything useful about Gitblit
or your server that may shed light on this issue?

I have added a repository list caching mechanism which is currently in testing.  This
should reduce disk io significantly _after_ the first enumeration (performed on startup).
 I suspect your issue is more complex than just the overhead of Gitblit recursing the
folder tree but this is what I can do for you based on the information I have.

Fix/change released in 1.1.0.