java - Multiple Xodus app to access/share single directory

Question

According to the comment from this post

"I have xodus directory I wan to share between two applications.. But one with read /write access and other with read-only access.. Is the any other way to create Environment apart from Environments.newInstance("xodusDir"); for read-only access to xodus database.. I need only read only access to xodus from other application.."

Answered with

"This would be possible with upcoming version 1.3.0."

Does it mean that it is possible to have multiple processes to create Environments / Persistent Storage pointing to the same directory:

  Environment environment = getEnvironment(xodusRoot, dir);
  final PersistentEntityStore store = PersistentEntityStores.newInstance(environment);

With this topology:

Is this possible or is there a workaround to do this?

From what I know, the xd.lck prevents two instance from accessing the same Xodus directory.

Answer 1

As of version 1.3.0, you can open a database in read-only mode ignoring the lock:

final EnvironmentConfig config = new EnvironmentConfig().
            setLogDataReaderWriterProvider("jetbrains.exodus.io.WatchingFileDataReaderWriterProvider").
            setLogCacheShared(false).
            setMemoryUsagePercentage(1);
final Environment env = Environments.newInstance(dir, config);
final PersistentEntityStore store = PersistentEntityStores.newInstance(env);

Not only it ignores the lock, but it also automatically fetches new data using java.nio.file.WatchService.

One instance Environment/EntityStore can be opened as usually (primary one) and several instances can be opened as above (secondary ones). Your proxy server should correctly distribute traffic so that secondary instances wouldn't get write requests. Obviously, your application have to resolve eventual consistency issues that are unavoidable with such architecture.

Secondary instances can be opened in any JVM. If you open secondary instances in the same JVM which is used to open primary instance, then keep an eye on memory usage since secondary instances don't use shared log cache. Primary instance (if you don't use setMemoryUsagePercentage) can consume 50% of maximum heap memory. It's good if overall memory usage percentage doesn't exceed 50-55% (though this depends on what GC and what GC settings you use).

If you open secondary instances in separate JVM(s) then skip setLogCacheShared and setMemoryUsagePercentage settings in the code above. The only thing you have to configure is watching DataReaderWriterProvider.

With the version 1.3.91, you can even run JVM(s) with secondary instances on different hosts. Each host should have a copy of the database on its filesystem, it can be opened as follows:

final EnvironmentConfig config = new EnvironmentConfig().
            setLogDataReaderWriterProvider("jetbrains.exodus.io.WatchingFileDataReaderWriterProvider");
final Environment env = Environments.newInstance(dir, config);
final PersistentEntityStore store = PersistentEntityStores.newInstance(env);

The only extra thing you have to do with your infrastructure is to provide periodical syncing of database changes. Version 1.3.91 is tested and works fine with rsync. Syncing each 5-10 seconds would be ok in most cases, though high writing workloads would result in some syncing delays.