For this tutorial we are using 3 VMs, with IPs and hostnames – one Ansible controller/manager (ansible-controller) which will be doing the provisioning on the two remote servers i.e. ansible-node1 and ansible-node2:
‘ansible-controller‘ is the manger node, the one performing the provisioning on the rest of the hosts i.e. on ansible-controller we’ll be installing and configuring Ansible.
Continue reading “Install Ansible on CentOS 7”
What is Ansible ?
Ansible is an open-source configuration management
tool i.e. for automating environment setups development or production, cloud provisioning, change management across multiple nodes. The thing which makes Ansible better then the other popular configuration management tools, like Puppet [link], Chef, etc is it’s agentless architecture. In the formal (Chef ad Puppet) you are required to install the daeman/agent on all the nodes i.e. the controller/master node, as well as the nodes you need to manage (can be 100s or 1000s). In case of Ansible all you need to do is install the Ansible on the controller/master node, and it’ll ssh into all the ‘to be managed’ nodes (may require you to add the keys). This architecture not only makes the setup easy, but also reduces the network overhead, due to the continuous polling of controller/master node by the client agents/nodes. Ansible was initially supported and sponsored by Ansible, Inc (originally AnsibleWorks, Inc) – in October 2015 it was acquired by Red Hat.
Continue reading “Getting started with Ansible”
Step 0. Prelimenaries
Run yum update
Stop the firewall
To make the ports accessible i.e. for clustering nodes use 5672, 4369, and 25672.
$ sudo systemclt stop firewall-cmd
The above command will disable SELinux for the session i.e. until next reboot – to permanently disable it set SELINUX=disabled in /etc/selinux/config file.
Set host names
Doing so later will most probably break the installation.
We’ll name the nodes/VMs as rabbit1 (192.168.40.192) and rabbit2 (192.168.40.193) – the default is localhost.
Continue reading “Rabbitmq Cluster on CentOS 7”
What is the ELK Stack?
ELK is an acronym from the first letter of three open-source products — Elasticsearch, Logstash, and Kibana— from Elastic
. The 3 products are used collectively (though can be used separately) mainly for centralizing and visualizing logs from multiple servers (as much as you want).
- Elasticsearch is basically a distributed, NoSQL data store, that uses on the Lucene search capabilities.
- Logstash is a log collection pipeline tool that accepts inputs from various sources (log forwarder), executes different filtering and formatting, and writes the data to Elasticsearch.
- Kibana is a graphical-user-interface (GUI) for visualization of Elasticsearch data.
The ELK Stack is the most widely used log analytics solution, beating Splunk’s enterprise software, which had long been the market leader. The ELK Stack is downloaded 500,000 times every month, making it the world’s most popular log management platform. In contrast, Splunk — the historical leader in the space — self-reports 10,000 total customers.
This tutorial is a guide to set up ELK stack and Filebeat as log-forwarder to gather syslogs of a remote machine (or as many servers as you want).
Continue reading “Install ELK stack on CentOS 7 to centralize logs analytics”
Elasticsearch is a distributed storage and real-time search engine.
- Distributed storage – you just need to setup and add Elasticsearch nodes, it’ll keep the data distributed on the cluster nodes. The distributed-ness makes data durable and highly-available too.
- Real-time search engine – You can get to query the data the moment it’s been written.
Due to the above 2 attributes you have been listening and reading about Elasticsearch, wherever there’s a discussion of real-time data analysis. It’d not be an overstatement to say technologies like Elasticsearch set the foundation for any efficient and reliable search engine.
Continue reading “Install Elasticsearch 5 on CentOS 7.x”
If you have a fresh installation, it is recommended to run the update first
Usually Java comes installed on CentOS 7 (Everything), for CentOS 7 minimal you may need to install Java for various setups. On a CentOS 7 Everything, you can verify it by simply checking the version:
# java -version
openjdk version "1.8.0_111"
OpenJDK Runtime Environment (build 1.8.0_111-b15)
OpenJDK 64-Bit Server VM (build 25.111-b15, mixed mode)
Java 1.8.0_111-b15 is the latest available, ‘1.8.0’ states it is Java 8, ‘u111’ means update 111, and ‘b15’means build 15.
For CentOS 7 minimal, the same command will give a different output:
# java -version
-bash: java: command not found
Which means Java is not installed. The latest Java version currently is Java 8.
Continue reading “Install Java 8 on CentOS/RHEL 7.x”
is a configuration management
tool for Unix-like and Microsoft Windows systems – which is basically provisioning automation i.e. the steps you want to preform on your freshly spawned virtual machine. Puppet uses declarative language for specification, and these configuration declaration files are termed as “Puppet manifests”. Puppet treats anything configurable as a “resource” i.e. file, service, package, user, cron, etc. The Puppet manifest is about describing the resources and their required states. Puppet gets the dynamic data (i.e. OS dependent) using Facter utility, for example Apache web server package is named ‘apache2’ in Ubunut, and ‘httpd’ for CentOS systems, Puppet allows variables in manifest (.pp file) to get such info on the fly, and set the right installation command.
Continue reading “Install Puppet Master-Agent on CentOS 7”
For various cluster or distributed application setups it’s an explicit requirement to have date and timezone (TZ) on each node synced. The difference in date or TZ may result in serious issues, for example in case of Puppet Master/Agent setup it’s a must requirement to have the data and TZ synced across all the nodes, if not, Puppet master server being the certificate authority may issue agent certificate from the distant past or future, which other nodes will treat as expire.
For syncing data and timezone across all the nodes, for whatever reason the tool at hand is ntpd.
The Network Time Protocol daemon (ntpd) is an operating system program that maintains the system time in synchronization with time servers using the Network Time Protocol (NTP).
Continue reading “Syncing Date and Timezone – Install and configure ntpd”
Puppet is an open-source configuration management
tool – for infrastructure orchestration, or automated provisioning, or configuration automation, and lot more. The simplest use case is for automated provisioning i.e. the tasks we need to perform once our machine/VM comes up for the first time (or even after it), like installing webserver, DB server, etc. Instead of manually performing the tasks/running the tasks we can use any of the available configuration management tools (like Puppet) to automate the boring repetitive tasks, and also making configuration consistent across all the servers.
Continue reading “Installing open-source standalone Puppet on CentOS 7”
First of all I want to address the biggest confusion here i.e. DevOps is not any tool, or technology, or some product one can use to make and do things better. DevOps is an idea, a management and operations approach – emphasizing on cohesiveness between development and operations teams. In simplest words, it’s about gluing the development and the IT operation hence the name DevOps:
- Dev – comes from development (developer/software engineers), people who make the system/software, and update it during it’s lifetime.
- Ops – from IT operations (sysadmins), who take care of the system once it’s developed, i.e. in production.
Continue reading “What is DevOps and Configuration Management”