Infrastructure
Mapa Europy (nakładka)

Infrastructure

Internal IT Infrastructure Optimization and Migration

Project Timeline: April 2025 – August 2025


Project Summary and Objectives:

The goal of this project was a deep restructuring and modernization of the internal server infrastructure. The main objectives included consolidating services to reduce the number of virtual machines (VMs), migrating operating systems from CentOS to modern standards based on Gentoo and Debian, and optimizing key development and administrative processes. The project resulted in a significantly simplified architecture, increased performance, and enhanced security.


Infrastructure before migration diagram

smartSystems.fallbackTitle

smartSystems.fallbackMessage

smartSystems.fallbackAlt

smartSystems.fallbackPrompt

Infrastructure before migration:

ServerTypeOSRoleCPURAMHDDDisk Type
esx01PHESXi 8.0.3VMware Hypervisor40CPU64GB2TBSDD HW RAID6
gatewayVMCentOS 9Firewall with port forwards and http proxy2vCPU4GB16GBVMDK
dbVMCentOS 9MySQL and PostgreSQL server2vCPU4GB25GBVMDK
ocVMCentOS 9OwnCloud server2vCPU2GB16GBVMDK
webVMCentOS 9Apache http with PHP 82vCPU4GB16GBVMDK
web-legacyVMCentOS 9Apache http with PHP 72vCPU4GB16GBVMDK
gitlabVMCentOS 9GitLab2vCPU6GB16GBVMDK
puppetVMCentOS 9Puppet server2vCPU4GB16GBVMDK
ns1VMCentOS 9Bind9 main server for owned domains2vCPU2GB16GBVMDK
ns2VMCentOS 9Bind9 backup server for owned domains2vCPU2GB16GBVMDK
jenkinsVMCentOS 9Jenkins CI server2vCPU4GB16GBVMDK
repoVMCentOS 9Apache http rpms server2vCPU4GB16GBVMDK
vpnVMCentOS 9OpenVPN server2vCPU2GB16GBVMDK
mxVMCentOS 9Posfix, Dovecot MTA server2vCPU4GB16GBVMDK
nagiosVMCentOS 9Nagios monitoring server2vCPU2GB16GBVMDK
adminVMCentOS 9Administration server2vCPU2GB16GBVMDK
ldapVMCentOS 9OpenLDAP server2vCPU2GB16GBVMDK
k8sVMCentOS 9Kubernetes control-plane server2vCPU2GB16GBVMDK
k8s-nodeVMCentOS 9Kubernetes worker node2vCPU2GB16GBVMDK
nasPHDebian 10Office NAS server with backups2CPU8GB1TBHDD SW RAID1

Key Initiatives and Execution Stages:

1. Infrastructure Consolidation and Resource Rationalization:

The primary driver of the project was the reduction of a sprawling infrastructure. This was achieved through the strategic merging of services:

  • Web Services Consolidation: All web services, including file servers, RPM repositories, and applications (initially OwnCloud, later NextCloud and PartDB), were consolidated onto a single target machine (webserver).
  • Administrative Services Centralization: Key tools such as Puppet, OpenLDAP, and the secondary DNS server were integrated within a single dedicated machine.
  • Decommissioning of Redundant Machines: Following successful service migrations, numerous obsolete virtual machines were decommissioned and deleted, including jenkins, db, admin, web-legacy, vpn, ns1, ns2, and k8s-node, which freed up significant system resources.
2. Technology Stack Modernization:

A strategic migration of operating systems was carried out to standardize and modernize the environment:

  • Most machines were migrated to a performance-optimized Gentoo Linux system.
  • Key services requiring stability, such as GitLab and Puppet, were migrated to Debian 12.
  • Dedicated Gentoo build servers were created and made available on the company network, significantly speeding up compilation processes.
3. Performance and Process (DevOps) Optimization:
  • Compilation Speed-Up: Tests using distcc on the newly created Gentoo builders showed a drastic reduction in compilation time for large projects (e.g., Firefox, LibreOffice) from ~60 minutes to just ~20 minutes.
  • CI/CD Migration: All deployment processes were moved from Jenkins to a modern GitLab platform, centralizing the software development lifecycle.
4. Enhancement of Security and Reliability:
  • New Backup System: The backup strategy was completely overhauled—the old directory was archived, and a new, clean backup process was initiated and synchronized with the office location.
  • Backup Issue Resolution: An issue with backing up the new VMs was diagnosed and fixed by converting their disks to the VMFS zeroedthick format.
  • Internal Security Scanning: A dedicated machine with OWASP tools was deployed for regular vulnerability scanning of the internal infrastructure.

Technical Implementation Details:

  • Virtualization Platform: VMware ESXi 8.0.3 on a DELL PowerEdge R640 server (40 logical cores, 64 GB RAM, RAID6 SSD).
  • Gentoo VM Standard: gentoo-kernel-bin Kernel to ensure portability.
  • Additional Environments: Two machines with Red Hat 9/10 were deployed for RHCSA certification preparation.

Infrastructure after migration diagram

smartSystems.fallbackTitle

smartSystems.fallbackMessage

smartSystems.fallbackAlt

smartSystems.fallbackPrompt

Infrastructure after migration:

ServerTypeOSRoleCPURAMHDDDisk Type
pve01PHProxmox VE 9Proxmox VE40CPU64GB2TBSDD HW RAID6
gatewayVMCentOS 9Firewall with port forwards and http proxy
Distcc gateway based on HAProxy
Bind9 main server for owned domains
2vCPU4GB16GBLVM
dbsVMGentooMySQL and PostgreSQL server2vCPU4GB45GBLVM
distccgwVMGentooDistcc cluster gateway server2vCPU4GB25GBLVM
gitVMDebian 13GitLab repos, docker registry, Gitlab CI4vCPU8GB35GBLVM
puppetVMDebian 13Puppet master, OpenLDAP server
Bind9 backup server for owned domains
2vCPU4GB25GBLVM
worker1VMGentooDistcc builder machine and GitLab runner8vCPU6GB60GBLVM
worker2VMGentooDistcc builder machine and GitLab runner8vCPU6GB60GBLVM
webserverVMGentooApache http server with websites
NextCloud Server with Collabora CODE
4vCPU6GB25GBLVM
mxVMCentOS 9Posfix, Dovecot MTA server2vCPU4GB16GBLVM
nagiosVMCentOS 9Nagios monitoring server2vCPU2GB16GBLVM
k8sVMGentooKubernetes server2vCPU4GB25GBLVM
rh9VMRedHat 9Server for training before RHCSA and RHCE certification2vCPU2GB16GBLVM
rh10VMRedHat 10Server for training before RHCSA and RHCE certification2vCPU2GB16GBLVM
owaspVMGentooOWASP scanner based on GVM
Distcc builder machine and GitLab runner
6vCPU16GB95GBLVM
nasVMGentooOffice NAS server with backups
MacOS X timemachines
2CPU8GB1TBSSD SW RAID1

Outcome:

The project was successfully completed, transforming a complex and distributed infrastructure into an optimized, consolidated, and modern ecosystem. The main benefits include reduced maintenance costs, a radical increase in the performance of development processes, and a significant improvement in the overall security posture.