Maria-db(mysql) two node installation with galera data replication


This is the first step toward unsupported installation of the NDC, New Data Cloud. To get the commercial version with full support, click here.

This document takes you through the installation and creation of a two node database installation. This install was done in google compute engine but with the right band width will work between data centers. It is an Enterprise class solution that has absolutely no license fees whatsoever. This document is provided without support or warranty.

The base configuration:

2 nodes 1 VCPU 3.7 GB running CentOS 7. This does not use the Maria-db that ships with CentOS 7, which is the free version of Red Hat Enterprise Linux.

instance-4 is primary node

instance-5 is failover node (replication target)

Install Mariadb repos. Cut and paste the repo text from the mariadb site.

install repos













This procedure is for version 10.1 (stable). Execute on both nodes.

cd /etc/my.cnf.d

modify server.cnf











# These groups are read by MariaDB server.
# Use it for options that only the server (but not clients) should see
# See the examples of server my.cnf files in /usr/share/mysql/

# this is read by the standalone daemon and embedded servers

# this is only for the mysqld standalone daemon

# * Galera-related settings
# Mandatory settings

## Galera Cluster Configuration
## Galera Synchronization Configuration
## Galera Node Configuration
# this is only for embedded server

# This group is only read by MariaDB servers, not by MySQL.
# If you use the same .cnf file for MySQL and MariaDB,
# you can put MariaDB-only options here

# This group is only read by MariaDB-10.1 servers.
# If you use the same .cnf file for MariaDB of different versions,
# use this group for options that older servers don’t understand

On failover node edit the same file




# These groups are read by MariaDB server.
# Use it for options that only the server (but not clients) should see
# See the examples of server my.cnf files in /usr/share/mysql/

# this is read by the standalone daemon and embedded servers

# this is only for the mysqld standalone daemon

# * Galera-related settings











# Mandatory settings
# Allow server to accept connections on all interfaces.
# Optional setting
# this is only for embedded server

# This group is only read by MariaDB servers, not by MySQL.
# If you use the same .cnf file for MySQL and MariaDB,
# you can put MariaDB-only options here

# This group is only read by MariaDB-10.1 servers.
# If you use the same .cnf file for MariaDB of different versions,
# use this group for options that older servers don’t understand

On active node:


echo $?


## Zero return code indicates successful new cluster starrt

systemctl status mariadb.service











On passive or replication node:

systemctl start mariadb.service

systemctl status mariadb.service






Check the logs:

tail -f /var/log/mariadb.log












To check galleria cluster status see this page.

You have an  Enterprise class active-passive database cluster.

You paid Oracle $0.

Use TCP wrappers with care


Real life story.


DMZ based server dedicated to SFTP was configured with sshd rules in /etc/hosts.allow
sshd : ALL@16.89.97.*:ALLOW
sshd : ALL@14.251.*:ALLOW
sshd : AAL@208.94.61.*:ALLOW

Should have been:

sshd : ALL@16.89.97.*:ALLOW
sshd : ALL@14.251.*:ALLOW
sshd : ALL@208.94.61.*:ALLOW

That network was the firewall to the outside world.

The end users were inconvenienced and the firewall team wasted a lot of time reviewing rues and looking at logs.

Quick and dirty autofs script share


I run a large unix and linux server farm on the west coast. In the old days we had problems with scrip versions. The problem was we pushed scripts from a central server and inevitably due to network or space issues the updates did not happen reliably.

In 2012 we opened up our unix management server to NFS. We mounted /opt/scripts for scripts access and /var/rep to allow central report writing by scripts. Worked well, but some of the servers were in remote locations and stuff hung when network issues arose.

So we wanted to make the system on demand.

So we used autofs.

On the management server:

/sbin/service autofs stop

/sbin/service nfs stop

/sbin/service nfslock stop

cd /opt

mv scripts shared

cd /var

mv rep shared


/etc/auto.master configuration (comments snipped)

/- /etc/

/etc/ configuration (comments snipped)

/opt/scripts -ro,soft,intr
/var/rep -rw,soft,intr

/etc/exports configuration

/opt/shared *(ro,sync,no_root_squash)
/var/shared *(rw,sync,no_root_squash)

Start nfs,nfslock,autofs

Push and auto.master to the entire environment and restart autofs.

Next we will see if it works with HP-UX (other website)

Even works on local scripts


Making sure what is configured to mount is mounted


Here is a script that uses some advanced awk commands to check that what is configured to be mounted in /etc/fstab is actually mounted.

# fsrootreserve
# Load common environment
. /opt/scripts/env/.scriptenv.linux
# set total variable for reserve blocks
        echo “. Checking mount status of all filesystems defined in /etc/fstab”
if [ “$1” = “-y” ];then
function fixmount {
  mount $mtname
  if [ $rc -eq 0 ]
      echo ”      pass – ${mpn} is defined in /etc/fstab and currently mounted”
      echo ” FAIL – Remediate manually. The script can not mount ${mpn}.”
#      echo ” I’m sorry Dave I am afraid I can’t do that. HAL-9000.”
# The next line removes blank lines lines beginning with hash and some faux filesystems from the analysis. This is more efficient than piping to grep -v
awk ‘/./ && !/#/ && !/\/tmpfs/ && !/tmpfs/ && !/\/sys/ && !/swap/ && !/\/proc/ { print $2 }’ /etc/fstab  | while read -r mpn
echo “checking filesystem $mpn”
mt=$(grep “$mpn ” /proc/mounts |awk ‘!/rootfs/ {print $1}’ | wc -l);
if (($mt != 1 ));then
        if (($CHANGES));then
                fixmount $mpn
                echo ” FAIL      – file system ${mpn} is defined in /etc/fstab and NOT mounted. (-y will attempt to mount).”
                echo ”      pass – ${mpn} is defined in /etc/fstab and currently mounted”

Network install point httpd with Centos or RHEL6 using kickstart


Why set up a network install point? So you can do consistent Linux installations. Really so you can do the same installation over and over again and come out with consistent results. Today’s article is how to set up the install point and have it work. There is credit for the source article below but by itself that article will not work for Centos 6.

The chosen install path is /var/www/html/centos/6.2

When it is released Centos 6.3 will go in a directory named 6.3

My source iso images are in a bootable partition, not an lvm mount called /iso

Partial ll listing:

-rw-r–r–. 1 root   root   4423129088 Dec 15 20:50 CentOS-6.2-x86_64-bin-DVD1.iso
-rw-r–r–. 1 root   root   1317967872 Dec 15 20:50 CentOS-6.2-x86_64-bin-DVD2.iso


mount -o loop /iso/CentOS-6.2-x86_64-bin-DVD1.iso /mnt
# The file name does not matter a bit. Just use the one you have downloaded.
# Make sure nothing is already mounted on /mnt folder

Copy the install point to your target directory.

cp -rvf /mnt/* /var/www/html/centos/6.2

This will miss two files, .discinfo and .treeinfo which kickstart needs to work right.

cp /mnt/.discinfo /var/www/html/centos/6.2 
cp /mnt/.treeinfo /var/www/html/centos/6.2 What the install depot should look like:
 umount and repeat for DVD2 ls -lart /var/www/html/centos/6.2

[root@solaria 6.2]# ls -lart /var/www/html/centos/6.2
total 536
drwxr-xr-x. 3 root root   4096 Feb 21 20:52 EFI
-rw-r--r--. 1 root root  18009 Feb 21 20:52 GPL
drwxr-xr-x. 3 root root   4096 Feb 21 20:52 images
drwxr-xr-x. 2 root root   4096 Feb 21 20:52 isolinux
-rw-r--r--. 1 root root   1354 Feb 21 20:53 RELEASE-NOTES-en-US.html
-rw-r--r--. 1 root root     14 Feb 21 20:54 CentOS_BuildTag
-rw-r--r--. 1 root root    212 Feb 21 20:54 EULA
drwxr-xr-x. 2 root root 450560 Feb 21 20:55 Packages
-rw-r--r--. 1 root root   1706 Feb 21 20:55 RPM-GPG-KEY-CentOS-6
-rw-r--r--. 1 root root   1730 Feb 21 20:55 RPM-GPG-KEY-CentOS-Debug-6
-rw-r--r--. 1 root root   1730 Feb 21 20:55 RPM-GPG-KEY-CentOS-Security-6
-rw-r--r--. 1 root root   1734 Feb 21 20:55 RPM-GPG-KEY-CentOS-Testing-6
-r--r--r--. 1 root root   2056 Feb 21 20:55 TRANS.TBL
drwxr-xr-x. 3 root root   4096 Feb 22 21:51 ..
-rw-r--r--. 1 root root     31 Feb 23 22:22 .discinfo
-rw-r--r--. 1 root root    338 Feb 23 22:22 .treeinfo
-rw-r--r--. 1 root root   2952 Feb 28 20:05 mars-ks.cfg
-rw-r--r--. 1 root root   2975 Feb 28 20:05 columbia-ks.cfg
-rw-r--r--. 1 root root   2975 Feb 28 20:33 pacifica-ks.cfg
drwxr-xr-x. 7 root root   4096 Feb 28 20:33 .
drwxr-xr-x. 3 root root   4096 Feb 29 20:12 repodata

for the install from kickstart to work, you will need to update the repodata and create the group information to avoid some nasty install issues that will frustrate you a lot.
 cd /var/www/html/centos/6.2


createrepo -u -g /var/www/html/centos/6.2/repodata/


[root@solaria 6.2]# createrepo -u -g /var/www/html/centos/6.2/repodata/

Saving Primary metadata
Saving file lists metadata
Saving other metadata


Source material:

DNS configuration checklist


BIND has always been a dark art. Recent configuration nuances made in the name of improving security have made things all the more fun.

Here is a simple checklist that can avoid trouble and downed websites. Note that on RHEL6/CentOS/ and clones the /var/named/chroot structure has been made obsolete.

  • Before starting cp /etc/named.conf /root  # pick any location but not /tmp because Linux cron cleans that up.
  • After adding new zones, run named-checkconf on your newly edited file. Do NOT edit the original.
  • Copy the names of new zone files into the named.conf file using cut and paste to avoid spelling errors.
  • chown named:named <filename> newly created zone files. Failure to do so will result in a completely meaningless and impossible to fathom error message when you restart the named daemon.
  • Use named-checkzone to check syntax in manually edited zone files. Consider using ISPCONFIG3 GUI to maintain these records.
  • Use the same back up policy as above for named.conf when working on zone files.
  • Update serial number in the zone file to insure fast propagation of DNS changes.

Follow this simple checklist to avoid a lot of unnecessary pain.


Centos Continuous Release with caution


Centos Continuous release is very easy to install. Download and install 1 rpm and you are all set.

( sha256: 9fc78d2d79abeb1513f0851d075a2860f5039fc8db3fb0db4c660252fffda894 )

( sha256: bd55e1505caae2f78c306290d235b7f54833fcad5a9f1942b3cb54e28f7bfe73 )

But I urge caution.

In my lab, I downloaded the 64 bit version and installed it. Then I ran the following command:

yum -y update

A lot of stuff got updated and a major problem was introduced, a bad release of apache was spun out, probably by Red Hat. I don’t know if Centos approves the content before it is released, I’m guessing probably not. I will contact them and let you know.

The problem:

[Sun Nov 06 05:25:27 2011] [notice] child pid 17890 exit signal Segmentation fault (11)

My research traced this back to a problem with the apache application. I needed to back out the httpd/apache release, install the previous release

To even have roll back, you must:

Add tsflags=repackage to /etc/yum.conf.
Add %_repackage_all_erasures 1 to /etc/rpm/macros. If /etc/rpm/macros does not exist, just create it.

You can now install, erase and update packages with yum and/or rpm, and they will save roll back information.

When you want to roll back, use rpm to do so.
You do this by specifying the --rollback switch and a date/time, like the examples below:

rpm -Uhv --rollback '19:00'
rpm -Uhv --rollback '8 hours ago'
rpm -Uhv --rollback 'december 31'
rpm -Uhv --rollback 'yesterday'

I was in a hurry, I merely removed httpd, temporarily disabled the CR repo and installed httpd and the dependencies that were removed at the same time.

To update production and exclude the bad httpd release:

yum -y update –exclude httpd


Turns out this problem was caused by a php plugin eaccellerate which is part of the ISPCONFIG3 setup I use to manage sites. The above technique did allow me to avoid the problem temporarily, and is good practice for when Red Hat does roll out bad rpm updates (which sadly happens all to often).

Network Channel Bonding (teaming) RHEL 6.0


This procedure did not actually change very much from RHEL 5.0. Knowing the changes however are pretty critical to getting it right.

Gone is the Administrator’s friend and potential cestpool, /etc/modpprobe.conf

Its replaced by anything you want to load in /etc/modprobe.d

Naming guidelines? Who needs them. Name the file anything you want.

Procedure: Still pretty darned easy.

Pre-requisite: You need two network connections to the same network subnet. Unlike HP-UX APA (Auto port aggregation) you don’t normally need special switch configuration. Though it is possible for Cisco switches to mess this up.

I’m recommending use of a standard naming convention in your shop. I, in a fit of creativity have chosen the name, bonding.conf

All you do is rip the bonding configuration you used from modeprobe.conf on RHEL 5 and put it in the fil:

options bond0 miimon=200 mode=5
alias eth0 e1000
alias eth1 e1000

This system has two Intel 1 GB cards in it, plugged into the same network and subnet.

[root@viper ~]# cat /etc/sysconfig/network-scripts/ifcfg-eth0
# Intel Corporation 82541GI Gigabit Ethernet Controller

[root@viper ~]# cat /etc/sysconfig/network-scripts/ifcfg-eth1
# Intel Corporation 82540EM Gigabit Ethernet Controller

Now the all critical bond0 configuration which has not changed.

[root@viper ~]# cat /etc/sysconfig/network-scripts/ifcfg-bond0



To implement:

/sbin/service network restart

[root@viper ~]# service network restart
Shutting down interface bond0:                             [  OK  ]
Shutting down loopback interface:                          [  OK  ]
Bringing up loopback interface:                            [  OK  ]
Bringing up interface bond0:                               [  OK  ]

One little warning. Red Has put a little tool in the OS called Network manager. It thinks it owns the network configuration.

If you for example copy in the configuration from another system with the intent of changing the IP address on a new one, be quick about it. Copy in ifcfg-bond0 without ifcfg-eth0/1 and you lose network access to the system.

It is highly recommended you do this with some form of console access.