Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

OutOfMemoryErrors (memory leak) when using latest versions of Java 6 and 7 #42

Closed
GoogleCodeExporter opened this issue Jul 25, 2015 · 14 comments

Comments

@GoogleCodeExporter
Copy link

What steps will reproduce the problem?
1. Unzip the attached maven project
2. cd thumbnailator-test
3. mvn test 

Maven runs the test with -Xms256m -Xmx256m. It resizes a png file 100 times, 
then resizes a jpg around 40 times before throwing an OOME. You can also build 
and run an executable jar with 
mvn -DskipTests=true package
java -Xms256m -Xmx256m -jar target/jpegleaker.jar

This fails on Ubuntu, java-version
java version "1.7.0_21"
Java(TM) SE Runtime Environment (build 1.7.0_21-b11)
Java HotSpot(TM) 64-Bit Server VM (build 23.21-b01, mixed mode)

This works on Ubuntu java -version
java version "1.7.0_17"
Java(TM) SE Runtime Environment (build 1.7.0_17-b02)
Java HotSpot(TM) 64-Bit Server VM (build 23.7-b01, mixed mode)



Original issue reported on code.google.com by will.t...@xtremelabs.com on 16 May 2013 at 12:58

Attachments:

@GoogleCodeExporter
Copy link
Author

After some investigation it looks like dispose() needs to be called on 
ImageReaders and ImageWriters after using them. The methods affected would be 
net.coobird.thumbnailator.tasks.io.InputStreamImageSource.read()
net.coobird.thumbnailator.tasks.io.OutputStreamImageSink.write(BufferedImage)
net.coobird.thumbnailator.tasks.io.FileImageSource.read()
net.coobird.thumbnailator.tasks.io.FileImageSink.write(BufferedImage)

Original comment by will.t...@xtremelabs.com on 16 May 2013 at 2:19

@GoogleCodeExporter
Copy link
Author

Hi,

Thank you for taking your time to report this issue.

It does appear like there are many instances of people encountering 
`OutOfMemoryError`s when using Thumbnailator with Java 7 Update 21.

The need to address this issue has been acknowledged.

Original comment by coobird...@gmail.com on 20 May 2013 at 1:47

  • Changed state: Accepted
  • Added labels: Priority-Critical
  • Removed labels: Priority-Medium

@GoogleCodeExporter
Copy link
Author

Issue 44 appears to be the same issue.

Original comment by coobird...@gmail.com on 20 May 2013 at 1:48

@GoogleCodeExporter
Copy link
Author

I've also gotten a report of a memory leak via an e-mail from Vladimir Shomin 
on April 24.

Quote:

> I have discovered some memory problem due exploiting system, when thousands of
> images have queued for thumbnailation - application server tomcat-7.0.39
> under jdk-1.7.0_21 crushes with OutOfMemoryError.
>
> Exploring heap state with VisualVM tool, I have found that memory is filled
> by megabyte-sized byte[] objects that linked as root by
> com.sun...JPEGImageReader . Forums recommend to use BufferedImage.dispose()
> and ImageReader.flush() methods beyond image processing cause its releases
> low-level resources used within image manipulations.

Again, this appears to occur when using Java 7 Update 21, which is the most 
up-to-date version of Java available as of the time of writing this comment.

Therefore, this issue should be given the highest priority.

Original comment by coobird...@gmail.com on 20 May 2013 at 1:59

@GoogleCodeExporter
Copy link
Author

Issue 44 has been merged into this issue.

Original comment by coobird...@gmail.com on 20 May 2013 at 2:22

@GoogleCodeExporter
Copy link
Author

Reproducing the Problem
-------------------------

In an effort to reproduce the problem, I ran a variation of the following code:

  Thumbnails.of(new File("/path/to/images").listFiles()
    .scale(0.1)
    .toFiles(new ConsecutivelyNumberedFilenames(new File("/path/for/thumbnails")));

I set the JVM's maximum heap settings to `-Xmx96m`, as that was the easiest 
value to see the Java 1.7.0_21 JVM to throw an `OutOfMemoryError` while 
processing images.

Using Thumbnailator 0.4.3 with the above settings, Java 1.7.0_21 would throw an 
`OutOfMemoryError` after processing 15 JPEGs which had the dimensions of 3648 x 
2736 pixels.

In contrast, when running with the same settings on Java 1.7.0_4, an 
`OutOfMemoryError` did not occur.


Attempting a Fix
-------------------------

As pointed out by the issue reporters, the problem probably lies in the lack of 
disposal code for the `ImageReader`s and `ImageWriter`s.

First, the disposal of the `ImageReader`s in the `FileImageSource` and 
`InputStreamImageSource` classes were added.
However, after running the test code, it encountered an `OutOfMemoryError` at 
15 images; the same as last time.

Next, the disposal of the `ImageWriters`s in the `FileImageSink` and 
`InputStreamImageSink` classes were added.

The results this time was that the test code would run without encountered an 
`OutOfMemoryError`.
(The code was subsequently changed to output to PNG, and the results were the 
same -- no `OutOfMemoryError`.)

As an added measure, a call to `BufferedImage.flush` has been added to the 
`Thumbnailator.createThumbnail(ThumbnailTask<?, ?>)` method, which should free 
up some resources as well.


Current Status
-------------------------

Thumbnailator 0.4.4 which addresses this `OutOfMemoryError` issue should be 
ready (hopefully) by the end of the week.

The changes has been pushed to the repository under the `0.4.4` branch:
http://code.google.com/p/thumbnailator/source/browse/?name=0.4.4

All unit tests are passing at this point, but I'll perform some manual testing 
before release.

Original comment by coobird...@gmail.com on 21 May 2013 at 4:16

  • Changed title: OutOfMemoryErrors (memory leak) when using latest version of Java 7
  • Changed state: Started
  • Added labels: Milestone-0.4.4

@GoogleCodeExporter
Copy link
Author

Issue 46 is similar issue (though that may not mean the cause is similar). 
Please take a look as well.

Original comment by jain...@gmail.com on 22 May 2013 at 3:33

@GoogleCodeExporter
Copy link
Author

Issue 46 is for jdk 1.6 build 45.

Original comment by jain...@gmail.com on 22 May 2013 at 3:34

@GoogleCodeExporter
Copy link
Author

I have reproduced the problem with Thumbnailator 0.4.3 running on Java 6 Update 
45.

When running the candidate for Thumbnailator 0.4.4 on Java 6 Update 45, it does 
not exhibit the `OutOfMemoryError` problem.

I'll merge Issue 46 into this issue.

Original comment by coobird...@gmail.com on 23 May 2013 at 12:23

  • Changed title: OutOfMemoryErrors (memory leak) when using latest versions of Java 6 and 7

@GoogleCodeExporter
Copy link
Author

Issue 46 has been merged into this issue.

Original comment by coobird...@gmail.com on 23 May 2013 at 12:25

@GoogleCodeExporter
Copy link
Author

Issue 45 has been merged into this issue.

Original comment by coobird...@gmail.com on 23 May 2013 at 2:45

@GoogleCodeExporter
Copy link
Author

This issue has been resolved in Thumbnailator 0.4.4.

I'd like to thank everyone involved in resolving this issue for contributing 
your time and effort! :)

Original comment by coobird...@gmail.com on 23 May 2013 at 3:49

  • Changed state: Fixed

@GoogleCodeExporter
Copy link
Author

was this a bug in a particular jdk version? was there a commit that fixed it? 
just wondering

Original comment by rogerpack2005 on 7 Jan 2015 at 12:19

@GoogleCodeExporter
Copy link
Author

No bug in the jdk, just a bug in Thumbnailator where resources weren't being 
cleaned up. It only seemed to matter in more recent jdk versions which were 
doing some optimizations by re-using resources (assuming they'd be disposed of 
when necessary)

Original comment by wt...@pivotal.io on 7 Jan 2015 at 12:39

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant