Export to GitHub

snakeyaml - issue #121

Blocks of consecutive tests fail with "too many open files" ... looks like file handle leaks


Posted on Jun 9, 2011 by Massive Cat

What steps will reproduce the problem? 1. Build with mock on Fedora 15 (unfortunately the issue is not systematically reproducible - it appears on few computers only - probably depends on their ulimit configuration? not sure)

What is the expected output? No errors.

What do you see instead? Look at the attachaments ... 1.) workstation 2.) build server

What version of SnakeYAML are you using? On what Java version? 1.8

Attachments

Comment #1

Posted on Jun 9, 2011 by Massive Rhino

The problem is not related to SnakeYAML. It can indicate a bug in Java or a configuration problem for the operating system. There is nothing SnakeYAML can do to solve the problem.

Comment #2

Posted on Jun 9, 2011 by Helpful Monkey

I think it needs more investigation. We may leak file handles because of not closing input streams. Brief look gives - seems like we do not close input streams... :(

But as I said - need more investigation.

Comment #3

Posted on Jun 9, 2011 by Massive Cat

Hi py4fun.

Thanks for the answer. Are You really sure, that this can't be caused by forgotten close calls? As forgotten close calls have exactly this kind of symptoms. File handles can get depleted when there's a missing close call in methods which are called multiple times and the differencies in the behavior are usually caused by race conditions related to the current system state (number of concurrent processes, etc.)

Please, try to reconsider that. Thank You.

Regards, Jaromir.

Comment #4

Posted on Jun 9, 2011 by Massive Cat

I'm gonna try the findbugs tool in order to find possible leaks and send patches if I find any ...

Comment #5

Posted on Jun 9, 2011 by Massive Cat

The following methods were marked as suspicious for failing to clean up stream or resources:

examples.LoadExampleTest.testLoadFromStream() examples.LoadExampleTest.testLoadManyDocuments() org.yaml.snakeyaml.issue99.YamlBase64Test.testLocalBinaryTag() org.yaml.snakeyaml.issue99.YamlBase64Test.testRedefineBinaryTag() org.yaml.snakeyaml.issue99.YamlBase64Test.testYamlBase64Loading() org.yaml.snakeyaml.issue99.YamlBase64Test.testYamlBase64LoadingLiteral()

Comment #6

Posted on Jun 9, 2011 by Massive Rhino

We are now closing more input streams. All the mentioned and many more others are now closing within the test. Give it a try.

Comment #7

Posted on Jun 9, 2011 by Massive Cat

Thank You.

I'm going to merge the following two commits into a patch file against 1.8 and test these changes on our build system.

I'm unsure if they can be applied directly on the 1.8 since there could be more changes dependent on changes in different files which won't be propagated with this patch.


4d1e0ed27f01:

/src/test/java/examples/LoadExampleTest.java /src/test/java/org/yaml/snakeyaml/issues/issue99/YamlBase64Test.java

a4b407aba3a5:

/src/changes/changes.xml /src/test/java/org/pyyaml/AnInstance.java /src/test/java/org/pyyaml/PyCanonicalTest.java /src/test/java/org/pyyaml/PyEmitterTest.java /src/test/java/org/pyyaml/PyErrorsTest.java /src/test/java/org/pyyaml/PyImportTest.java /src/test/java/org/pyyaml/PyReaderTest.java /src/test/java/org/pyyaml/PyStructureTest.java

/src/test/java/org/pyyaml/PyTokensTest.java

Comment #8

Posted on Jun 9, 2011 by Massive Cat

The following patch (including changes from commits 4d1e0ed27f01 & a4b407aba3a5) against the 1.8 solved the problem.

http://jcapik.fedorapeople.org/files/snakeyaml/4/snakeyaml-issue121-file-handle-leaks.patch

The package has been build successfully.

I believe this issue can be marked as fixed.

Comment #9

Posted on Jun 9, 2011 by Massive Rhino

(No comment was entered for this change.)

Status: Fixed

Labels:
Type-Defect Priority-Medium