java - Slow SecureRandom initialization

Question

Suppose you do simple thing:

public class Main {
    public static void main(String[] args) {
        long started = System.currentTimeMillis();
        try {
            new URL(args[0]).openConnection();
        } catch (Exception ignore) {
        }
        System.out.println(System.currentTimeMillis() - started);
    }
}

Now run it with http://localhost as args[0]

It takes ~100 msec to complete.

Now try https://localhost

It takes 5000+ msec.

Now run the same thing on linux or in docker:

• http: ~100 msec
• https: ~350 msec

Why is this? Why such a huge difference between platforms? What can you do about it?

For long-running application servers and applications with their own long and heavy initialization sequence, these 5 seconds may not matter.

However, there are plenty of applications where this initial 5sec "hang" matters and may become frustrating...

Answer 1

(Note: see also latest updates at the end of this answer)

Explanation

Reason for this is default SecureRandom provider.

On Windows, there are 2 SecureRandom providers available:

- provider=SUN, type=SecureRandom, algorithm=SHA1PRNG
- provider=SunMSCAPI, type=SecureRandom, algorithm=Windows-PRNG

On Linux (tested in Alpine docker with Oracle JDK 8u162):

- provider=SUN, type=SecureRandom, algorithm=NativePRNG
- provider=SUN, type=SecureRandom, algorithm=SHA1PRNG
- provider=SUN, type=SecureRandom, algorithm=NativePRNGBlocking
- provider=SUN, type=SecureRandom, algorithm=NativePRNGNonBlocking

These are specified in jre/lib/security/java.security file.

security.provider.1=sun.security.provider.Sun
...
security.provider.10=sun.security.mscapi.SunMSCAPI

By default, first SecureRandom provider is used. On Windows, the default one is sun.security.provider.Sun, and this implementation reports following when JVM is run with -Djava.security.debug="provider,engine=SecureRandom":

Provider: SecureRandom.SHA1PRNG algorithm from: SUN
provider: Failed to use operating system seed generator: java.io.IOException: Required native CryptoAPI features not  available on this machine
provider: Using default threaded seed generator

And the default threaded seed generator is very slow.

You need to use SunMSCAPI provider.

Solution 1: Configuration

Reorder providers in configuration:

Edit jre/lib/security/java.security:

security.provider.1=sun.security.mscapi.SunMSCAPI
...
security.provider.10=sun.security.provider.Sun

I am not aware this can be done via system properties.

Or maybe yes, using-Djava.security.properties (untested, see this)

Solution 2: Programmatic

Reorder providers programmatically:

Optional.ofNullable(Security.getProvider("SunMSCAPI")).ifPresent(p->{
    Security.removeProvider(p.getName());
    Security.insertProviderAt(p, 1);
});

JVM now reports following (-Djava.security.debug="provider,engine=SecureRandom"):

Provider: SecureRandom.Windows-PRNG algorithm from: SunMSCAPI

Solution 3: Programmatic v2

Inspired by this idea, following piece of code inserts only a single SecureRandom service, configured dynamically from existing SunMSCAPI provider without the explicit reliance on sun.* classes. This also avoids the potential risks associated with indiscriminate prioritization of all services of SunMSCAPI provider.

public interface WindowsPRNG {

    static void init() {
        String provider = "SunMSCAPI"; // original provider
        String type = "SecureRandom"; // service type
        String alg = "Windows-PRNG"; // algorithm
        String name = String.format("%s.%s", provider, type); // our provider name
        if (Security.getProvider(name) != null) return; // already registered
        Optional.ofNullable(Security.getProvider(provider)) // only on Windows
                .ifPresent(p-> Optional.ofNullable(p.getService(type, alg)) // should exist but who knows?
                        .ifPresent(svc-> Security.insertProviderAt( // insert our provider with single SecureRandom service
                                new Provider(name, p.getVersion(), null) {{
                                    setProperty(String.format("%s.%s", type, alg), svc.getClassName());
                                }}, 1)));
    }

}

Performance

<140 msec (instead of 5000+ msec)

Details

There is a call to new SecureRandom() somewhere down the call stack when you use URL.openConnection("https://...")

It calls getPrngAlgorithm() (see SecureRandom:880)

And this returns first SecureRandom provider it finds.

For testing purposes, call to URL.openConnection() can be replaced with this:

new SecureRandom().generateSeed(20);

Disclaimer

I am not aware of any negative side effects caused by providers reordering. However, there may be some, especially considering default provider selection algorithm.

Anyway, at least in theory, from functional point of view this should be transparent to application.

Update 2019-01-08

Windows 10 (version 1803): Cannot reproduce this issue anymore on any of latest JDKs (tested all from old oracle 1.7.0_72 up to openjdk "12-ea" 2019-03-19).

It looks like it was Windows issue, fixed in latest OS updates. Related updates may or may not have taken place in recent JRE releases, too. However, I cannot reproduce the original issue even with my oldest JDK 7 update 72 installation which was definitelly affected, and definitelly not patched in any way.

There are still minor performance gains when using this solution (cca 350 msec on average) but the default behavior no longer suffers the intolerable 5+ seconds penalty.