pdsh is slow to generate the list of hosts when using a few thousand hosts selected via genders. I first noticed this on zin, but it is easy to recreate with a simple genders file like "foo[1-3000] bar". The results for pdsh-2.30 are:

max = 1000 : foo[1-1000] real 0m0.790s user 0m0.778s sys 0m0.010s max = 2000 : foo[1-2000] real 0m3.136s user 0m3.110s sys 0m0.019s max = 3000 : foo[1-3000] real 0m7.043s user 0m6.980s sys 0m0.045s max = 4000 : foo[1-4000] real 0m12.558s user 0m12.472s sys 0m0.057s max = 5000 : foo[1-5000] real 0m19.728s user 0m19.579s sys 0m0.104s max = 6000 : foo[1-6000] real 0m28.414s user 0m28.238s sys 0m0.112s max = 7000 : foo[1-7000] real 0m38.689s user 0m38.411s sys 0m0.190s max = 8000 : foo[1-8000] real 0m50.599s user 0m50.265s sys 0m0.220s max = 9000 : foo[1-9000] real 1m3.908s user 1m3.293s sys 0m0.470s max = 10000 : foo[1-10000] real 1m19.859s user 1m19.083s sys 0m0.594s

While the results for pdsh-2.20 are:

max = 1000 : foo[1-1000] real 0m0.009s user 0m0.007s sys 0m0.001s max = 2000 : foo[1-2000] real 0m0.015s user 0m0.011s sys 0m0.005s max = 3000 : foo[1-3000] real 0m0.022s user 0m0.017s sys 0m0.005s max = 4000 : foo[1-4000] real 0m0.035s user 0m0.027s sys 0m0.004s max = 5000 : foo[1-5000] real 0m0.038s user 0m0.032s sys 0m0.007s max = 6000 : foo[1-6000] real 0m0.051s user 0m0.039s sys 0m0.010s max = 7000 : foo[1-7000] real 0m0.055s user 0m0.048s sys 0m0.008s max = 8000 : foo[1-8000] real 0m0.067s user 0m0.049s sys 0m0.016s max = 9000 : foo[1-9000] real 0m0.079s user 0m0.061s sys 0m0.017s max = 10000 : foo[1-10000] real 0m0.090s user 0m0.076s sys 0m0.013s

Note how for v2.30, 3000 hosts takes 7.043s while for v2.20 it takes 0.022s.

This issue was updated by revision 3c17946e6211.

Add a test to the testsuite to ensure that a genders -g query on the order of 10000 hosts returns in reasonable time (<1s)

This issue was closed by revision e149fe78edea.