groovy - Spring Boot + Eureka Server + Hystrix with Turbine: empty turbine.stream

Question

I'm trying to run Spring Boot (with Spring Cloud) + Eureka Server + Hystrix Dashboard and Turbine stream, but I run into a problem I couldn't find any solution so far. I use Spring Boot 1.2.1.RELEASE and Spring Cloud 1.0.0.RC2. Here is what I have:

The first instance is running Eureka server and Hystrix dashboard:

@Configuration
@EnableAutoConfiguration
@EnableEurekaServer
@EnableHystrixDashboard
@EnableDiscoveryClient
class Application {

    public static void main(String[] args) {
        SpringApplication.run Application, args
    }
}

Here you can find build.gradle for that instance - https://gist.github.com/wololock/570272ad7cf2d14a4d3c

Eureka server is running ok, I can see registered instances on eureka server dashboard, I can also use LoadBalancer to get the URL to registered instance using its id. So far everything is ok.

I have a few instances that are run with @EnableHystrix annotation and use @HystrixCommand to define which methods have to be monitored by Hystrix. When I pass URL to hystrix.stream of single instance to Hystrix dashboard, I can see it running with no problem.

I have also separate Turbine server, not complicated one:

@EnableAutoConfiguration
@EnableTurbine
@Configuration
@EnableDiscoveryClient
class Application {

    public static void main(String[] args) {
        SpringApplication.run Application, args
    }
}

Here you can find build.gradle for Turbine server instance - https://gist.github.com/wololock/ff0d855b8a890232851e

It uses very simple configuration, build mostly on the one provided by sample turbine app - https://github.com/spring-cloud-samples/turbine

info:
  component: Turbine

endpoints:
  restart:
    enabled: true
  shutdown:
    enabled: true

turbine:
  appConfig: pdf-creator-service

InstanceDiscovery:
  impl: io.spring.platform.netflix.turbine.EurekaInstanceDiscovery

server:
  port: 8989

management:
  port: 8990

eureka:
  instance:
    leaseRenewalIntervalInSeconds: 10
  client:
    serviceUrl:
      defaultZone: ${vcap.services.${PREFIX:}eureka.credentials.uri:http://user:password@localhost:8761}/eureka/

After running those instances in order:

1. eureka server
2. turbine server
3. discover client instance,

I have 2nd and 3rd instance registered in eureka server, turbine server log says, that there is one instance up:

[2015-02-06 12:35:04.162] boot - 20495  INFO [Timer-0] --- EurekaInstanceDiscovery: Fetching instance list for apps: [pdf-creator-service]
[2015-02-06 12:35:04.162] boot - 20495  INFO [Timer-0] --- EurekaInstanceDiscovery: Fetching instances for app: pdf-creator-service
[2015-02-06 12:35:04.162] boot - 20495  INFO [Timer-0] --- EurekaInstanceDiscovery: Received instance list for app: pdf-creator-service = 1
[2015-02-06 12:35:04.162] boot - 20495  INFO [Timer-0] --- InstanceObservable: Retrieved hosts from InstanceDiscovery: 1
[2015-02-06 12:35:04.162] boot - 20495  INFO [Timer-0] --- InstanceObservable: Found hosts that have been previously terminated: 0
[2015-02-06 12:35:04.162] boot - 20495  INFO [Timer-0] --- InstanceObservable: Hosts up:1, hosts down: 0
[2015-02-06 12:36:04.162] boot - 20495  INFO [Timer-0] --- EurekaInstanceDiscovery: Fetching instance list for apps: [pdf-creator-service]
[2015-02-06 12:36:04.162] boot - 20495  INFO [Timer-0] --- EurekaInstanceDiscovery: Fetching instances for app: pdf-creator-service
[2015-02-06 12:36:04.162] boot - 20495  INFO [Timer-0] --- EurekaInstanceDiscovery: Received instance list for app: pdf-creator-service = 1
[2015-02-06 12:36:04.162] boot - 20495  INFO [Timer-0] --- InstanceObservable: Retrieved hosts from InstanceDiscovery: 1
[2015-02-06 12:36:04.162] boot - 20495  INFO [Timer-0] --- InstanceObservable: Found hosts that have been previously terminated: 0
[2015-02-06 12:36:04.162] boot - 20495  INFO [Timer-0] --- InstanceObservable: Hosts up:1, hosts down: 0

Calling hystrix.stream from a single instance works, e.g. curl http://localhost:8885/hystrix.stream returns:

data: {"type":"HystrixCommand","name":"post","group":"PdfController","currentTime":1423223614259,"isCircuitBreakerOpen":false,"errorPercentage":0,"errorCount":0,"requestCount":0,"rollingCountCollapsedRequests":0,"rollingCountExceptionsThrown":0,"rollingCountFailure":0,"rollingCountFallbackFailure":0,"rollingCountFallbackRejection":0,"rollingCountFallbackSuccess":0,"rollingCountResponsesFromCache":0,"rollingCountSemaphoreRejected":0,"rollingCountShortCircuited":0,"rollingCountSuccess":0,"rollingCountThreadPoolRejected":0,"rollingCountTimeout":0,"currentConcurrentExecutionCount":0,"latencyExecute_mean":0,"latencyExecute":{"0":0,"25":0,"50":0,"75":0,"90":0,"95":0,"99":0,"99.5":0,"100":0},"latencyTotal_mean":0,"latencyTotal":{"0":0,"25":0,"50":0,"75":0,"90":0,"95":0,"99":0,"99.5":0,"100":0},"propertyValue_circuitBreakerRequestVolumeThreshold":20,"propertyValue_circuitBreakerSleepWindowInMilliseconds":5000,"propertyValue_circuitBreakerErrorThresholdPercentage":50,"propertyValue_circuitBreakerForceOpen":false,"propertyValue_circuitBreakerForceClosed":false,"propertyValue_circuitBreakerEnabled":true,"propertyValue_executionIsolationStrategy":"THREAD","propertyValue_executionIsolationThreadTimeoutInMilliseconds":8000,"propertyValue_executionIsolationThreadInterruptOnTimeout":true,"propertyValue_executionIsolationThreadPoolKeyOverride":null,"propertyValue_executionIsolationSemaphoreMaxConcurrentRequests":10,"propertyValue_fallbackIsolationSemaphoreMaxConcurrentRequests":10,"propertyValue_metricsRollingStatisticalWindowInMilliseconds":10000,"propertyValue_requestCacheEnabled":true,"propertyValue_requestLogEnabled":true,"reportingHosts":1}

data: {"type":"HystrixCommand","name":"generate","group":"WkHtmlToPdfGenerator","currentTime":1423223614259,"isCircuitBreakerOpen":false,"errorPercentage":0,"errorCount":0,"requestCount":0,"rollingCountCollapsedRequests":0,"rollingCountExceptionsThrown":0,"rollingCountFailure":0,"rollingCountFallbackFailure":0,"rollingCountFallbackRejection":0,"rollingCountFallbackSuccess":0,"rollingCountResponsesFromCache":0,"rollingCountSemaphoreRejected":0,"rollingCountShortCircuited":0,"rollingCountSuccess":0,"rollingCountThreadPoolRejected":0,"rollingCountTimeout":0,"currentConcurrentExecutionCount":0,"latencyExecute_mean":0,"latencyExecute":{"0":0,"25":0,"50":0,"75":0,"90":0,"95":0,"99":0,"99.5":0,"100":0},"latencyTotal_mean":0,"latencyTotal":{"0":0,"25":0,"50":0,"75":0,"90":0,"95":0,"99":0,"99.5":0,"100":0},"propertyValue_circuitBreakerRequestVolumeThreshold":20,"propertyValue_circuitBreakerSleepWindowInMilliseconds":5000,"propertyValue_circuitBreakerErrorThresholdPercentage":50,"propertyValue_circuitBreakerForceOpen":false,"propertyValue_circuitBreakerForceClosed":false,"propertyValue_circuitBreakerEnabled":true,"propertyValue_executionIsolationStrategy":"THREAD","propertyValue_executionIsolationThreadTimeoutInMilliseconds":8000,"propertyValue_executionIsolationThreadInterruptOnTimeout":true,"propertyValue_executionIsolationThreadPoolKeyOverride":null,"propertyValue_executionIsolationSemaphoreMaxConcurrentRequests":10,"propertyValue_fallbackIsolationSemaphoreMaxConcurrentRequests":10,"propertyValue_metricsRollingStatisticalWindowInMilliseconds":10000,"propertyValue_requestCacheEnabled":true,"propertyValue_requestLogEnabled":true,"reportingHosts":1}

data: {"type":"HystrixThreadPool","name":"PdfController","currentTime":1423223614259,"currentActiveCount":0,"currentCompletedTaskCount":4,"currentCorePoolSize":10,"currentLargestPoolSize":4,"currentMaximumPoolSize":10,"currentPoolSize":4,"currentQueueSize":0,"currentTaskCount":4,"rollingCountThreadsExecuted":0,"rollingMaxActiveThreads":0,"propertyValue_queueSizeRejectionThreshold":5,"propertyValue_metricsRollingStatisticalWindowInMilliseconds":10000,"reportingHosts":1}

But when I attach turbine.stream to hystrix dashboard, I get nothing. Logs say:

[2015-02-06 12:42:48.922] boot - 24816  INFO [Timer-0] --- EurekaInstanceDiscovery: Received instance list for app: pdf-creator-service = 1
[2015-02-06 12:42:48.922] boot - 24816  INFO [Timer-0] --- InstanceObservable: Retrieved hosts from InstanceDiscovery: 1
[2015-02-06 12:42:48.922] boot - 24816  INFO [Timer-0] --- InstanceObservable: Found hosts that have been previously terminated: 0
[2015-02-06 12:42:48.922] boot - 24816  INFO [Timer-0] --- InstanceObservable: Hosts up:1, hosts down: 0
[2015-02-06 12:43:26.237] boot - 24816  INFO [XNIO-2 task-4] --- TurbineStreamServlet: FilterCriteria: []
[2015-02-06 12:43:26.237] boot - 24816  INFO [XNIO-2 task-4] --- TurbineStreamServlet: StatsType filters: []
[2015-02-06 12:43:26.237] boot - 24816  INFO [XNIO-2 task-4] --- TurbineStreamingConnection: Relevance config: []
[2015-02-06 12:43:26.237] boot - 24816  INFO [XNIO-2 task-4] --- TurbineStreamingConnection: Relevance metrics config: {}
[2015-02-06 12:43:26.237] boot - 24816  INFO [XNIO-2 task-4] --- ClusterMonitor: Registering event handler for cluster monitor: StreamingHandler_f1308dda-58c5-47a5-b1e2-5a0bea32226b
[2015-02-06 12:43:26.237] boot - 24816  INFO [XNIO-2 task-4] --- TurbineDataDispatcher: 

Just added and starting handler tuple: StreamingHandler_f1308dda-58c5-47a5-b1e2-5a0bea32226b
[2015-02-06 12:43:26.238] boot - 24816  INFO [XNIO-2 task-4] --- AggDataFromCluster: Per handler dispacher started for: StreamingHandler_f1308dda-58c5-47a5-b1e2-5a0bea32226b
[2015-02-06 12:43:26.238] boot - 24816  INFO [XNIO-2 task-4] --- ClusterMonitor: All event handlers for cluster monitor: [StreamingHandler_637572ab-acda-4bf4-81cd-6a658adb73eb, StreamingHandler_f1308dda-58c5-47a5-b1e2-5a0bea32226b, StaticListener_For_Aggregator, StreamingHandler_5ec12ee8-3fcd-4a6f-9006-d2a6ecc309d0, StreamingHandler_72d7b9e2-ad98-42a0-9ac3-abe4aa57cc7a]
[2015-02-06 12:43:26.238] boot - 24816  INFO [XNIO-2 task-4] --- ClusterMonitor: Starting up the cluster monitor for default_agg
</cod

Answer 1

It works for me if I add some configuration for the cluster, e.g.

turbine:
  appConfig: customers,stores
  clusterNameExpression: new String('default')

Turbine has to know how to construct the "cluster" name (an aggregation key for sets of applications). The default is to use the appname, so if you don't set the clusterNameExpression you need to use a query param in the stream URL, e.g. /turbine.stream?cluster=CUSTOMERS (uppercased appname).