menu
more_vert

Weblogic OutOfMemory exception的误解 -- thread limitation

作者:dayu

[ERROR][thread ] Could not start thread Timer-72025. Resource temporarily unavailable Exception in thread "Timer-72024" java.lang.OutOfMemoryError: Resource temporarily unavailable in tsStartJavaThread (lifecycle.c:1097). Attempting to allocate 2G bytes There is insufficient native memory for the Java Runtime Environment to continue. Possible reasons: The system is out of physical RAM or swap space In 32 bit mode, the process size limit was hit Possible solutions: Reduce memory load on the system Increase physical memory or swap space Check if swap backing store is full Use 64 bit Java on a 64 bit OS Decrease Java heap size (-Xmx/-Xms) Decrease number of Java threads Decrease Java thread stack sizes (-Xss) Disable compressed references (-XXcompressedRefs=false) at java.lang.Thread.start0(Native Method) at java.lang.Thread.start(Thread.java:640) at java.util.Timer.<init>(Timer.java:137) at java.util.Timer.<init>(Timer.java:106) at ...... at java.util.TimerThread.mainLoop(Timer.java:512) at java.util.TimerThread.run(Timer.java:462)
第一反应以为是内存不足,于是dump了heap回来看。发现系统仅仅占用了80M的内存,远远系统内存绰绰有余,那么为啥会出现这样的情况呢?查看了出错的代码。发现代码启动了非常多的线程,遂想起曾经遇到过类似的问题,weblogic里面启动的线程太多。超出了linux的最大限制,也是会出这个看起来内存泄漏的日志。查看系统的限制:

sc-1@vECE-222 limits.d # ulimit -a
core file size          (blocks, -c) 0
data seg size           (kbytes, -d) unlimited
scheduling priority             (-e) 0
file size               (blocks, -f) unlimited
pending signals                 (-i) 38610
max locked memory       (kbytes, -l) 64
max memory size         (kbytes, -m) unlimited
open files                      (-n) 8192
pipe size            (512 bytes, -p) 8
POSIX message queues     (bytes, -q) 819200
real-time priority              (-r) 0
stack size              (kbytes, -s) 10240
cpu time               (seconds, -t) unlimited
max user processes              (-u) 1024
virtual memory          (kbytes, -v) unlimited
file locks                      (-x) unlimited
发现最大的进程数是1024,注意:linux里面计算线程和进程是用一样计算的,这里说的max user processes是包含线程的。

使用下面代码重现问题:

public class ThreadExample {

    public static void main(String[] args){
        System.out.println(Thread.currentThread().getName());
        for(int i=0; i< 2048; i++){
            new Thread("" + i){
                public void run(){
                    System.out.println(Thread.currentThread().getName() + " running");

                    try {
                        Thread.sleep(60000);
                    } catch (InterruptedException e) {
                        e.printStackTrace();
                    }
                    System.out.println(Thread.currentThread().getName() + " ending");
                }
            }.start();
        }
    }
}
运行和输出:

sc-1@ECE30CP3_Z1 etmdaha # su -p occas -c "/opt/jrockit-jdk/bin/java ThreadExample"
Main Thread
0 running
5 running
//
836 running
[ERROR][thread ] Could not start thread 837. Resource temporarily unavailable
Exception in thread "Main Thread" java.lang.OutOfMemoryError: Resource temporarily unavailable in tsStartJavaThread (lifecycle.c:1096).

Attempting to allocate 3G bytes

There is insufficient native memory for the Java
Runtime Environment to continue.

Possible reasons:
  The system is out of physical RAM or swap space
  In 32 bit mode, the process size limit was hit

Possible solutions:
  <strong>Reduce memory load on the system
  Increase physical memory or swap space
  Check if swap backing store is full
  Use 64 bit Java on a 64 bit OS
  Decrease Java heap size (-Xmx/-Xms)
  Decrease number of Java threads
  Decrease Java thread stack sizes (-Xss)
  Disable compressed references (-XXcompressedRefs=false)
</strong>
        at java.lang.Thread.start0(Native Method)
        at java.lang.Thread.start(Thread.java:640)
        at ThreadExample.main(ThreadExample.java:6)

linux系统资源有各种限制:内存限制。栈限制,文件数目限制,线程限制。要小心陷进这些问题。

# ulimit -a
core file size          (blocks, -c) 0
data seg size           (kbytes, -d) unlimited
scheduling priority             (-e) 0
file size               (blocks, -f) unlimited
pending signals                 (-i) 79010
max locked memory       (kbytes, -l) 64
max memory size         (kbytes, -m) unlimited
open files                      (-n) 1024
pipe size            (512 bytes, -p) 8
POSIX message queues     (bytes, -q) 819200
real-time priority              (-r) 0
stack size              (kbytes, -s) 10240
cpu time               (seconds, -t) unlimited
max user processes              (-u) 1024
virtual memory          (kbytes, -v) unlimited
file locks                      (-x) unlimited

这里有两篇非常好的文章:

http://people.redhat.com/alikins/system_tuning.html#threads

http://kavassalis.com/2011/03/linux-and-the-maximum-number-of-processes-threads/


Weblogic OutOfMemory exception的误解 -- thread limitation

原文地址:http://www.cnblogs.com/gccbuaa/p/7249584.html