客服电话
APP下载
迪恩网络APP
随时随地掌握行业动态
官方微信
扫描二维码
关注迪恩网络微信公众号
|
项目的缓存中用到了rocksdb，实例化时报错了：
问题现象：类加载失败引起的一系列后续问题：类找不到、实例加载失败、容器加载失败。 问题定位：代码跟踪： public class Options extends RocksObject
implements DBOptionsInterface<Options>, ColumnFamilyOptionsInterface<Options>,
MutableColumnFamilyOptionsInterface<Options> {
static {
RocksDB.loadLibrary();
}
进入到RocksDB类，实例化rocksDB，先读取是否已加载标志位，没加载过才会去加载: /**
* Loads the necessary library files.
* Calling this method twice will have no effect.
* By default the method extracts the shared library for loading at
* java.io.tmpdir, however, you can override this temporary location by
* setting the environment variable ROCKSDB_SHAREDLIB_DIR.
*/
public static void loadLibrary() {
if (libraryLoaded.get() == LibraryState.LOADED) {
return;
}
if (libraryLoaded.compareAndSet(LibraryState.NOT_LOADED,
LibraryState.LOADING)) {
final String tmpDir = System.getenv("ROCKSDB_SHAREDLIB_DIR");
// loading possibly necessary libraries.
for (final CompressionType compressionType : CompressionType.values()) {
try {
if (compressionType.getLibraryName() != null) {
System.loadLibrary(compressionType.getLibraryName());
}
} catch (UnsatisfiedLinkError e) {
// since it may be optional, we ignore its loading failure here.
}
}
try {
NativeLibraryLoader.getInstance().loadLibrary(tmpDir);
} catch (IOException e) {
libraryLoaded.set(LibraryState.NOT_LOADED);
throw new RuntimeException("Unable to load the RocksDB shared library"
+ e);
}
libraryLoaded.set(LibraryState.LOADED);
return;
}
while (libraryLoaded.get() == LibraryState.LOADING) {
try {
Thread.sleep(10);
} catch(final InterruptedException e) {
//ignore
}
}
}
注意一点，临时目录是从环境变量ROCKSDB_SHAREDLIB_DIR读取的，所以可以通过设置该环境变量改变临时目录路径（日志里的C:\Users\Administrator\AppData\Local\Temp\librocksdbjni3696928169151614297.dll）。接着进入到NativeLibraryLoader看RocksDB的实例化详情： /** * Firstly attempts to load the library from <i>java.library.path</i>, * if that fails then it falls back to extracting * the library from the classpath * {@link org.rocksdb.NativeLibraryLoader#loadLibraryFromJar(java.lang.String)} * * @param tmpDir A temporary directory to use * to copy the native library to when loading from the classpath. * If null, or the empty string, we rely on Java's * {@link java.io.File#createTempFile(String, String)} * function to provide a temporary location. * The temporary file will be registered for deletion * on exit. * * @throws java.io.IOException if a filesystem operation fails. */ public synchronized void loadLibrary(final String tmpDir) throws IOException { try { System.loadLibrary(sharedLibraryName); } catch(final UnsatisfiedLinkError ule1) { try { System.loadLibrary(jniLibraryName); } catch(final UnsatisfiedLinkError ule2) { loadLibraryFromJar(tmpDir); } } } /** * Attempts to extract the native RocksDB library * from the classpath and load it * * @param tmpDir A temporary directory to use * to copy the native library to. If null, * or the empty string, we rely on Java's * {@link java.io.File#createTempFile(String, String)} * function to provide a temporary location. * The temporary file will be registered for deletion * on exit. * * @throws java.io.IOException if a filesystem operation fails. */ void loadLibraryFromJar(final String tmpDir) throws IOException { if (!initialized) { System.load(loadLibraryFromJarToTemp(tmpDir).getAbsolutePath()); initialized = true; } } File loadLibraryFromJarToTemp(final String tmpDir) throws IOException { final File temp; if (tmpDir == null || tmpDir.isEmpty()) { temp = File.createTempFile(tempFilePrefix, tempFileSuffix); } else { temp = new File(tmpDir, jniLibraryFileName); if (temp.exists() && !temp.delete()) { throw new RuntimeException("File: " + temp.getAbsolutePath() + " already exists and cannot be removed."); } if (!temp.createNewFile()) { throw new RuntimeException("File: " + temp.getAbsolutePath() + " could not be created."); } } if (!temp.exists()) { throw new RuntimeException("File " + temp.getAbsolutePath() + " does not exist."); } else { temp.deleteOnExit(); } // attempt to copy the library from the Jar file to the temp destination try (final InputStream is = getClass().getClassLoader(). getResourceAsStream(jniLibraryFileName)) { if (is == null) { throw new RuntimeException(jniLibraryFileName + " was not found inside JAR."); } else { Files.copy(is, temp.toPath(), StandardCopyOption.REPLACE_EXISTING); } } return temp; } 先从sharedLibraryName（rocksdbjni）实例化，再从jniLibraryName（rocksdbjni-win64）实例化，最后从临时目录实例化。第一步是从JDK环境变量（即java.library.path，本机是D:\Dev\Java\jdk1.8.0_102\jre\bin）加载rocksdbjni，失败再从类路径加载rocksdbjni-win64，最后从本地临时目录加载librocksdbjni-win64.dll。这三部曲是rocksDB实例化的精髓。而最后一步是最复杂的，需要创建一个临时文件，然后从rocksdbjni这个jar包中把librocksdbjni-win64.dll复制到本地C:\Users\Administrator\AppData\Local\Temp\librocksdbjni3696928169151614297.dll，最后用org.rocksdb.NativeLibraryLoader这个类加载器来加载librocksdbjni3696928169151614297.dll得到rocksdb的实例，然后把这个DLL文件加入到加载类库中，后续容器重复加载时无需再次复制DLL文件。 上面的三部曲都需要利用java.lang包的工具来进行类加载，用的到类有System、Runtime和ClassLoader： /**
* Loads the native library specified by the filename argument. The filename
* argument must be an absolute path name.
*
* If the filename argument, when stripped of any platform-specific library
* prefix, path, and file extension, indicates a library whose name is,
* for example, L, and a native library called L is statically linked
* with the VM, then the JNI_OnLoad_L function exported by the library
* is invoked rather than attempting to load a dynamic library.
* A filename matching the argument does not have to exist in the
* file system.
* See the JNI Specification for more details.
*
* Otherwise, the filename argument is mapped to a native library image in
* an implementation-dependent manner.
*
* <p>
* The call <code>System.load(name)</code> is effectively equivalent
* to the call:
* <blockquote><pre>
* Runtime.getRuntime().load(name)
* </pre></blockquote>
*
* @param filename the file to load.
* @exception SecurityException if a security manager exists and its
* <code>checkLink</code> method doesn't allow
* loading of the specified dynamic library
* @exception UnsatisfiedLinkError if either the filename is not an
* absolute path name, the native library is not statically
* linked with the VM, or the library cannot be mapped to
* a native library image by the host system.
* @exception NullPointerException if <code>filename</code> is
* <code>null</code>
* @see java.lang.Runtime#load(java.lang.String)
* @see java.lang.SecurityManager#checkLink(java.lang.String)
*/
@CallerSensitive
public static void load(String filename) {
Runtime.getRuntime().load0(Reflection.getCallerClass(), filename);
}
/**
* Loads the native library specified by the filename argument. The filename
* argument must be an absolute path name.
* (for example
* <code>Runtime.getRuntime().load("/home/avh/lib/libX11.so");</code>).
*
* If the filename argument, when stripped of any platform-specific library
* prefix, path, and file extension, indicates a library whose name is,
* for example, L, and a native library called L is statically linked
* with the VM, then the JNI_OnLoad_L function exported by the library
* is invoked rather than attempting to load a dynamic library.
* A filename matching the argument does not have to exist in the file
* system. See the JNI Specification for more details.
*
* Otherwise, the filename argument is mapped to a native library image in
* an implementation-dependent manner.
* <p>
* First, if there is a security manager, its <code>checkLink</code>
* method is called with the <code>filename</code> as its argument.
* This may result in a security exception.
* <p>
* This is similar to the method {@link #loadLibrary(String)}, but it
* accepts a general file name as an argument rather than just a library
* name, allowing any file of native code to be loaded.
* <p>
* The method {@link System#load(String)} is the conventional and
* convenient means of invoking this method.
*
* @param filename the file to load.
* @exception SecurityException if a security manager exists and its
* <code>checkLink</code> method doesn't allow
* loading of the specified dynamic library
* @exception UnsatisfiedLinkError if either the filename is not an
* absolute path name, the native library is not statically
* linked with the VM, or the library cannot be mapped to
* a native library image by the host system.
* @exception NullPointerException if <code>filename</code> is
* <code>null</code>
* @see java.lang.Runtime#getRuntime()
* @see java.lang.SecurityException
* @see java.lang.SecurityManager#checkLink(java.lang.String)
*/
@CallerSensitive
public void load(String filename) {
load0(Reflection.getCallerClass(), filename);
}
synchronized void load0(Class<?> fromClass, String filename) {
SecurityManager security = System.getSecurityManager();
if (security != null) {
security.checkLink(filename);
}
if (!(new File(filename).isAbsolute())) {
throw new UnsatisfiedLinkError(
"Expecting an absolute path of the library: " + filename);
}
ClassLoader.loadLibrary(fromClass, filename, true);
}
// Invoked in the java.lang.Runtime class to implement load and loadLibrary.
static void loadLibrary(Class<?> fromClass, String name,
boolean isAbsolute) {
ClassLoader loader =
(fromClass == null) ? null : fromClass.getClassLoader();
if (sys_paths == null) {
usr_paths = initializePath("java.library.path");
sys_paths = initializePath("sun.boot.library.path");
}
if (isAbsolute) {
if (loadLibrary0(fromClass, new File(name))) {
return;
}
throw new UnsatisfiedLinkError("Can't load library: " + name);
}
if (loader != null) {
String libfilename = loader.findLibrary(name);
if (libfilename != null) {
File libfile = new File(libfilename);
if (!libfile.isAbsolute()) {
throw new UnsatisfiedLinkError(
"ClassLoader.findLibrary failed to return an absolute path: " + libfilename);
}
if (loadLibrary0(fromClass, libfile)) {
return;
}
throw new UnsatisfiedLinkError("Can't load " + libfilename);
}
}
for (int i = 0 ; i < sys_paths.length ; i++) {
File libfile = new File(sys_paths[i], System.mapLibraryName(name));
if (loadLibrary0(fromClass, libfile)) {
return;
}
libfile = ClassLoaderHelper.mapAlternativeName(libfile);
if (libfile != null && loadLibrary0(fromClass, libfile)) {
return;
}
}
if (loader != null) {
for (int i = 0 ; i < usr_paths.length ; i++) {
File libfile = new File(usr_paths[i],
System.mapLibraryName(name));
if (loadLibrary0(fromClass, libfile)) {
return;
}
libfile = ClassLoaderHelper.mapAlternativeName(libfile);
if (libfile != null && loadLibrary0(fromClass, libfile)) {
return;
}
}
}
// Oops, it failed
throw new UnsatisfiedLinkError("no " + name + " in java.library.path");
}
private static boolean loadLibrary0(Class<?> fromClass, final File file) {
// Check to see if we're attempting to access a static library
String name = findBuiltinLib(file.getName());
boolean isBuiltin = (name != null);
if (!isBuiltin) {
boolean exists = AccessController.doPrivileged(
new PrivilegedAction<Object>() {
public Object run() {
return file.exists() ? Boolean.TRUE : null;
}})
!= null;
if (!exists) {
return false;
}
try {
name = file.getCanonicalPath();
} catch (IOException e) {
return false;
}
}
ClassLoader loader =
(fromClass == null) ? null : fromClass.getClassLoader();
Vector<NativeLibrary> libs =
loader != null ? loader.nativeLibraries : systemNativeLibraries;
synchronized (libs) {
int size = libs.size();
for (int i = 0; i < size; i++) {
NativeLibrary lib = libs.elementAt(i);
if (name.equals(lib.name)) {
return true;
}
}
synchronized (loadedLibraryNames) {
if (loadedLibraryNames.contains(name)) {
throw new UnsatisfiedLinkError
("Native Library " +
name +
" already loaded in another classloader");
}
/* If the library is being loaded (must be by the same thread,
* because Runtime.load and Runtime.loadLibrary are
* synchronous). The reason is can occur is that the JNI_OnLoad
* function can cause another loadLibrary invocation.
*
* Thus we can use a static stack to hold the list of libraries
* we are loading.
*
* If there is a pending load operation for the library, we
* immediately return success; otherwise, we raise
* UnsatisfiedLinkError.
*/
int n = nativeLibraryContext.size();
for (int i = 0; i < n; i++) {
NativeLibrary lib = nativeLibraryContext.elementAt(i);
if (name.equals(lib.name)) {
if (loader == lib.fromClass.getClassLoader()) {
return true;
} else {
throw new UnsatisfiedLinkError
("Native Library " +
name +
" is being loaded in another classloader");
}
}
}
NativeLibrary lib = new NativeLibrary(fromClass, name, isBuiltin);
nativeLibraryContext.push(lib);
try {
lib.load(name, isBuiltin);
} finally {
nativeLibraryContext.pop();
}
if (lib.loaded) {
loadedLibraryNames.addElement(name);
libs.addElement(lib);
return true;
}
return false;
}
}
}
这里拿三部曲最后一部举例说明下：参数fromClass就是调用System.load方法所在的类，这里是rocksDB的加载库类org.rocksdb.NativeLibraryLoader，另一个参数File就是C:\Users\Administrator\AppData\Local\Temp\librocksdbjni3696928169151614297.dll，从NativeLibraryLoader类加载librocksdbjni5581206381055061043.dll后，把这个DLL文件放入loadedLibraryNames这个Vector，后续重复加载直接从该Vector取，无需再从rocksdbjni的jar包中复制DLL文件。 跟踪到这里，问题代码已经浮出水面： lib.load(name, isBuiltin); 这个load方法是ClassLoader类的内部类NativeLibrary的原生方法，动态链接库加载就是从这里进入DLL文件里具体加载方法的 native void load(String name, boolean isBuiltin); 从这个load出来就是UnsatisfiedLinkError。所以问题的焦点转移到了DLL文件本身，用Dependency Walker打开一看，有错误提示： Error: Modules with different CPU types were found. 继续跟进，发现其中一个文件叫IEShims.dll显示是X86，而我本机是X64的。替换该文件，Dependency Walker不再报错，但跑代码发现依然报错。 问题解决：捕获rockdbs异常，记录日志，跳过，继续启动加载spring容器。该DLL的加载涉及太多因素，无法一一排查，只能吃掉异常来规避，rocksdb加载失败对主业务无太大影响。 |
2023-10-27
2022-08-15
2022-08-17
2022-09-23
2022-08-13
六六分期app的软件客服如何联系？不知道吗？加qq群【895510560】即可！标题：六六分期
今天小编告诉大家如何处理win10系统火狐flash插件总是崩溃的问题，可能很多用户都不知
今天小编告诉大家如何对win10系统删除桌面回收站图标进行设置，可能很多用户都不知道
今天小编告诉大家如何对win10系统电脑设置节能降温的设置方法，想必大家都遇到过需要
我们在使用xp系统的过程中,经常需要对xp系统无线网络安装向导设置进行设置，可能很多
今天小编告诉大家如何处理win7系统玩cf老是与主机连接不稳定的问题，可能很多用户都不
电脑对日常生活的重要性小编就不多说了，可是一旦碰到win7系统设置cf烟雾头的问题，很
我们在日常使用电脑的时候，有的小伙伴们可能在打开应用的时候会遇见提示应用程序无法
今天小编告诉大家如何对win7系统打开vcf文件进行设置，可能很多用户都不知道怎么对win
今天小编告诉大家如何对win10系统s4开启USB调试模式进行设置，可能很多用户都不知道怎
请发表评论