java - How to know the reason for JVM crashing with Segfault?

Question

We are seeing the JVM getting crashed at times with segfault. The only error we see in logs is as below.

Anyone can suggest something by looking at the below error trace.

---

# A fatal error has been detected by the Java Runtime Environment:
#
#  SIGSEGV (0xb) at pc=0x00007fef7f1d3eb0, pid=42623, tid=0x00007feea62c8700
#
# JRE version: OpenJDK Runtime Environment (8.0_222-b10) (build 1.8.0_222-b10)
# Java VM: OpenJDK 64-Bit Server VM (25.222-b10 mixed mode linux-amd64 compressed oops)
# Problematic frame:
# J 62683 C2 org.apache.ignite.internal.marshaller.optimized.OptimizedObjectOutputStream.writeObject0(Ljava/lang/Object;)V (331 bytes) @ 0x00007fef7f1d3eb0 [0x00007fef7f1d3e00+0xb0]
#
# Failed to write core dump. Core dumps have been disabled. To enable core dumping, try "ulimit -c unlimited" before starting Java again
#
# An error report file with more information is saved as:
# /tmp/hsperfdata_pvappuser/hs_err_pid42623.log
#
# If you would like to submit a bug report, please visit:
#   http://bugreport.java.com/bugreport/crash.jsp

---

While trying to understand the reason for this crash Oracle JVM docs https://docs.oracle.com/javase/8/docs/technotes/guides/troubleshoot/crashes001.html ,this looks to be the case of 5.1.2 Crash in Compiled Code as the problematic frame is java frame(has a "J")

Though could not get much further from it, we also not sure when it comes, the only probale pattern is it comes when JVM is running for 5-6 days so usually on Friday. We are using openjdk-8 ("1.8.0_232") distribution provided by RedHat running on RHEL 6.10.

Looking forward to get any leading point in tracing this error.

Answer 1

The current stack frame has writeObject0 as the last called method. There is a naming convention that native method's names end with 0. Therefore check whether that method is indeed native.

If it is, it is probably written in C, an ancient unsafe language whose programs tend to crash in an uncontrolled way. This often leads to SIGSEGV.

In this case, that method is written in Java though.

As you were told in the error message, read hs_err_pid42623.log for further details. In that file you will find the registers and a few machine instructions around the code that crashed.