From Nick's Wiki
Jump to: navigation, search


Linux-related Projects

I am hosting my websites and mail servers on a set of 5 servers based on the Debian GNU/Linux and CentOS distributions. The servers are assigned as described below:

  • katmai - Dell PowerEdge T105, primary node in the katmai/krakatoa cluster with 1 TB of storage.
  • krakatoa - Dell PowerEdge SC440, secondary node in the katmai/krakatoa cluster, upgrade planned after all services are moved to katmai.
  • cotopaxi - HP Proliant DL360 G5.
  • izalco - HP Proliant DL360 G5, inactive, awaiting the retirement of stromboli.
  • stromboli - HP Proliant DL360 G5, to be decommissioned and used as spare parts source for izalco and cotopaxi.
  • unzen - HP Pentium III, used as main Internet gateway, firewall and load balancer.
  • fuji - HP Pentium III, inactive, to be rebuilt as Internet gateway.
  • ruapehu - Dell Pentium III, used for monitoring.
  • tambora - HP Pavillion, being rebuilt for the Delaware & Raritan Railroad.

Two other systems are used for model railroading projects - merapi (an old Pentium 100MHz box slated for usage as computer-based interlocking) and orodruin (a Sun Ultra-60 with Solaris 10 operating system used for development).

I named my machines after active volcanoes: Katmai (Alaska, site of the most powerful eruption of the 20th century in 1912); Kilauea (Hawaii); Krakatoa (Indonesia, well known for its 1883 eruption); Orodruin (situated in Tolkien's Middle Earth, also known as Mt. Doom; it is the only non-Intel system in my basement); Rainier (Washington, situated not far from Redmond); Ruapehu (New Zealand); Stromboli (Italy); Tambora (Indonesia, had the most powerful eruption in recorded history in June 1815); Unzen (Japan); Fuji (Japan).

The main servers katmai and krakatoa are set up as a 2-node cluster, based on the High Availability Linux (HA-Linux) system. Several disk areas are mirrored trough DRBD between the two systems.

Another HA-Linux 2-node cluster is formed by fuji and unzen.

2012/13 Rebuild of the SanDESNet systems

Recently I have added a new server - HP Proliant DL360 G5 - to be used primarily as Oracle VirtualBox server, software repository (Spacewalk from Red Hat) and configuration manager (puppet from Puppetlabs). The new server is called stromboli, replacing an old Pentium 100MHz system with the same name which was retired in August 2011.

Three systems have been allocated for the model railroad: tambora, merapi and orodruin.

I am setting up new virtual servers on stromboli for incoming email, outgoing email, web server and mailbox (cyrus) server. Once the virtual servers are set up I will start rebuilding the katmai/krakatoa cluster.

The katmai/krakatoa cluster will have their disk capacity increased by adding 1TB disk drives and the operating system will be upgraded to Debian 6 from Debian 5.

A Spacewalk-managed kickstart server and a Puppet master have been installed on stromboli.

The old desktop system kilauea has been relocated in the basement and is being used as console for the servers and as print server for the Delaware & Raritan model railroad project.

May 20, 2012

The incoming mail server prototype has been built as a virtual host on stromboli. It is based on the Debian Squeeze GNU/Linux distribution and has been built from a minimal installation to a fully functional mail server with the help of the Puppet Configuration Management system. The incoming mail server Puppet modules developed for the prototype will be used for the production mail servers when the katmai/krakatoa cluster is rebuilt.

July 2012 - I am working on two projects as part of the Sandesnet network rebuild process: kilauea is being upgraded to CentOS 6 and a networked storage array is being designed to supplement the katmai/krakatoa cluster.

July 14, 2012 - The incoming and outgoing email services have been directed to the new virtual servers hosted on stromboli. The old virtual servers hosted on katmai/krakatoa have been decommissioned and will be rebuilt together with their hosts.

August 2012 - New plans for katmai, krakatoa and stromboli: katmai and stromboli are to be converted to [VMware ESX] servers, while krakatoa will be allocated to the storage array project.

August 2, 2012 - The new Debian 6 based web server, built with Puppet, is being tested on stromboli. If everything goes well the two old web servers running katmai and krakatoa will be decommissioned in the coming days. I decided to use separate IP addresses for the 5 sites hosted on the new server - this will require changes in the external DNS zones and the load balancer setup. I have set the TTL on the DNS records to 300s in order to ease the transition to the new server. For some zones it is the first modification since 2009.

April-May 2013 - katmai upgraded by replacing one of its two 250 GB drives with a 1 TB drive. It was then rebuilt with Debian 7. I installed LDAP and migrated the users' home directories to katmai. The plan is to phase out the NIS-based user authentication with LDAP. LDAP will also be used for user access to the Cyrus mailboxes.

September 2013 - I have acquired two HP Proliant DL 360 servers. One of them - named cotopaxi - has been built with CentOS 6.4 and Xen 4.1. The second one - named izalco - will be fitted with memory and disk drives taken from stromboli.

October 2013 - Several websites, hosted on a virtual server on krakatoa, migrated to a new server on katmai.

November 2013 - The email system, hosted on virtual servers on krakatoa and stromboli, migrated to new servers hosted on katmai. The spam and malware blocking and filtering system has been rewritten in Perl and simplified. The incoming mail server configuration has been redesigned so that it does not rely on NFS mounts and access to MySQL databases. The Cyrus and outgoing mail servers have been merged into one server so that the LDAP-based user authentication system can be used for access to both mailboxes and outgoing email. The Mysql databases have been migrated to the katmai.

The SanDESNet Email System

The email sent to or originating from domains hosted on is handled by three types of servers:

  • The incoming email is handled by incoming mail servers based on the Postfix MTA and is delivered - after spam and malware filtering - to the mailbox server. The Postfix MTA allows for 4 different levels of spam blocking by running 4 instances with different configurations. A 5th instance handles email originating from web forms on the websites hosted on the SanDESNet servers.
  • The user mailbox server distributes the incoming mail to the user mailboxes and provides access through the IMAP and POP3 protocols, allowing both plain text and encrypted communications. The outgoing email is handled by a Postfix server using the SMTPAUTH protocol on TCP/587 with SSL or TLS encryption. The user authentication is done by LDAP. Access via UUCP is available as backup for remote systems without permanent connectivity.

Spam and Malware Filtering System

The mail servers are protected by a system that blocks and filters unwanted email and malware, in accordance with the acceptable use policy.

High Availability Linux

The network includes two pairs of systems in HA-Linux settings: unzen/fuji and katmai/krakatoa. The katmai/krakatoa pair is scheduled to be decommissioned as part of the Sandesnet network rebuild project.

Linux-Xen Virtual Hosts

The katmai and krakatoa servers are hosting several Xen domains. katmai is scheduled to be rebuilt with VMware ESX, while krakatoa will be reassigned to the storage array project.

VMWare ESX Server

A VMWare ESX server is to be installed on katmai after the all the virtual hosts running on that machine are decommissioned. After katmai is recommissioned the next step will be converting stromboli to VMWare ESX.

Spacewalk Software Repository

All CentOS systems on the Sandesnet network are built using a Spacewalk repository management system. Support for Debian and Ubuntu distributions is planned.

Puppet Configuration Manager

All the Linux and Solaris system builds and rebuilds in the Sandesnet network are managed with the help of a Puppet configuration management system.

Linux Firewall

The Sandesnet network is protected by Linux iptables-based firewalls built with the ipmasq(1) utility. The ipmasq(1) utility is distributed with the Debian and Ubuntu GNU/Linux distributions and allows the modular building of firewalls.

Storage Array

As part of the Sandesnet network rebuild project a networked storage array will be added to the katmai/krakatoa cluster. The storage array project will eventually result in storage products designed for small and medium sized businesses.