In this post we will describe what is needed to get started with managing your EAP 6 logs with ElasticSearch, Logstash and Kibana. There are several reasons why you would want to collect your logging output in a central place.
- Aggregate (output from multiple applications / hosts)
- Correlate events in different systems
- Analyze (more than grep)
- Integrate into monitoring
- Gather statistics
A common solution that supports all this use cases is provided by the ELK stack. It consists of ElasticSearch (ES), Logstash and Kibana. ElasticSearch provides persistence and analytics, Logstash provides the pipeline that brings your Logs into ES and Kibana provides a GUI for querying and dashboards.
In a recent blog-post Clustering in JBoss AS7/EAP 6 we showed how basic clustering in the new EAP 6 and JBoss AS 7 can be used. The EAP 6 is basically an AS 7 with official RedHat-support. Our cluster we described in that post was small and simple. This post will cover much more complex cluster structures, how to build them and how we can utilize the new domain-mode for our clusters. There are multiple ways to build and manage bigger JBoss cluster environments. We will describe two ways to do so: One using separating techniques also applicable to older JBoss versions and the other way using an Infinispan feature called distribution.
Scalability vs. Availability
The main challenge when building a cluster is to make it both highly available and scalable.
Availability for a cluster means: If one node fails, all the sessions on that node will be seamlessly served by another node. This can be achieved through session-replication. Session-replication is preconfigured and enabled in the
ha profile in the
domain.xml. Flat replication means that all sessions are copied to all other nodes: If you have got four nodes with 1GB memory for each of them, your cluster can only use 1GB of memory because basically all nodes store copies from each other. I. e. your cluster will not have 4*1GB=4GB memory. If you would add more nodes to this cluster you would not get more memory, you will even lose some memory due to overhead for replication. But you will get more availability and more important more network traffic due to replication overhead (all changes need to be redistributed to all other nodes). Let us call this cluster topology full-replication.
The ability to combine different servers to a cluster that hides its internal servers from the clients and offers a virtual platform for an application is important for enterprise applications. It can be used to provide
- high scalability by adding cheap computational resources to the cluster on demand or
- high availability by using a transparent failover that hides faults within single servers.
Usually high scalability limits high availability and vice versa, but it is also possible to get both. The JBoss application server can be configured to support both features.
This post is the first one of a series about clustering with the JBoss AS 7. Here, we focus on the basic concepts behind JBoss AS 7 clustering and show you how to setup a basic clustered environment with a simple Java EE application.
In the series, we concentrate on the JBoss AS 7 respectively the EAP 6, which is the Red Hat-supported version of the JBoss application server. Future posts will be about particular subsystems of the JBoss AS, such as HornetQ or Infinispan.