livialima

joined 2 years ago
MODERATOR OF
linuxupskillchallenge
sorted by: new top controversial old
20
Day 1 - Get to know your server (programming.dev)
 

INTRO

You should now have a remote server setup running the latest Ubuntu Server LTS (Long Term Support) version. You alone will be administering it. To become a fully-rounded Linux server admin you should become comfortable working with different versions of Linux, but for now Ubuntu is a good choice.

Once you have reached a level of comfort at the command-line then you'll find your skills transfer not only to all the standard Linux variants, but also to Android, Apple's OSX, OpenBSD, Solaris and IBM AIX. Throughout the course you'll be working on Linux - but in fact most of what is covered is applicable to any system derived from the UNIX Operating System - and the major differences between them are with their graphic user interfaces such as Gnome, Unity, KDE etc - none of which you’ll be using!

YOUR TASKS TODAY

  • Connect and login to your server, preferably using a SSH client
  • Run a few simple commands to check the status of your server - like this demo

USING A SSH CLIENT

Remote access used to be done by the simple telnet protocol, but now the much more secure SSH (Secure SHell) protocol is always used. If your server is a local VM or WSL, you could skip this section by simply using the server console/terminal if you want. We will explore SSH more in detail at the server side on Day 3 but knowing how to use a ssh client is a basic sysadmin skill, so you might as well do it now.

In MacOS and Linux

On an MacOS machine you'll normally access the command line via Terminal.app - it's in the Utilities sub-folder of Applications.

On Linux distributions with a menu you'll typically find the terminal under "Applications menu -> Accessories -> Terminal", "Applications menu -> System -> Terminal" or "Menu -> System -> Terminal Program (Konsole)"- or you can simply search for your terminal application. In many cases Ctrl+Alt+T will also bring up a terminal windows.

Once you open up a "terminal" session, you can use your command-line ssh client like this:

ssh user@<ip address>

For example:

ssh support@192.123.321.99

If the remote server was configured with a SSH public key (like AWS, Azure and GCP), then you'll need to point to the location of the private key as proof of identity with the -i switch, typically like this:

ssh -i ~/.ssh/id_rsa support@192.123.321.99

A very slick connection process can be setup with the .ssh/config feature - see the "SSH client configuration" link in the EXTENSION section below.

In Windows

On recent Windows 10 versions, the same command-line client is now available, but must be enabled (via "Settings", "Apps", "Apps & features", "Manage optional features", "Add a feature", "OpenSSH client").

There are various SSH clients available for Windows (PuTTY, Solar-PuTTY, MobaXterm, Termius, etc) but if you use Windows versions older than 10, the installation of PuTTY is suggested.

Alternatively, you can install the Windows Subsystem for Linux which gives you a full local command-line Linux environment, including an SSH client - ssh.

Regardless of which client you use, the first time you connect to your server, you may receive a warning that you're connecting to a new server - and be asked if you wish to cache the host key. Yes, you do. Just type/click Yes.

But don't worry too much about securing the SSH session or hardening the server right now; we will be doing this in Day 3.

For now, just login to your server and remember that Linux is case-sensitive regarding user names, as well as passwords.

You'll be spending a lot of time in your SSH client, so it pays to spend some time customizing it. At the very least try "black on white" and "green on black" - and experiment with different monospaced fonts, ("Ubuntu Mono" is free to download, and very nice).

It's also very handy to be able to cut and paste text between your remote session and your local desktop, so spend some time getting confident with how to do this in your SSH client and terminal.

Perhaps you might now try logging in from home and work - even from your smartphone! - using an ssh client app such as Termux, Termius for Android or Termius for iPhone. As a server admin you'll need to be comfortable logging in from all over. You can also potentially use JavaScript ssh clients like consolefish and ShellHub, but these options involve putting more trust in third-parties than most sysadmins would be comfortable with when accessing production systems.

To log out, simply type exit or close the terminal.

LOGIN TO YOUR SERVER

Once logged in, notice that the "command prompt" that you receive ends in $ - this is the convention for an ordinary user, whereas the "root" user with full administrative power has a # prompt (but we will dive into this difference in Day 3 as well).

Here's a short vid on using ssh in a work environment.

GENERAL INFORMATION ABOUT THE SERVER

Use lsb_release -a to see which Linux distro and version you're using. lsb_release may not be available in your server, as it's not widely adopted, but you will always have the same information available in the system file os-release. You can check its content by typing cat /etc/os-release

uname -a will also print the system information and it can show some interesting things like kernel version, hardware platform, etc.

uptime will show you how long the system has been running. It kinda makes the weird numbers you get from cat /proc/uptime a lot more readable.

whoami will print the user name you logged on with, who will show who is logged on and w will also show what they are doing.

HARDWARE INFORMATION

lshw can give some detailed information on the hardware configuration, and there's a bunch of switches we can use to filter the information we want to see, but it's not the only tool we use to check hardware with. Some of the used commands are:

MEASURE MEMORY AND CPU USAGE

Don't worry! Linux won't eat your RAM. But if you want to check the amount of memory used in the system, use free -h . vmstat will also give some memory statistics.

top is like a Task Manager for Linux, it will display the processes and the consumption of resources. htop is an interactive, prettier version.

MEASURE DISK USAGE

Use df -h to see disk space usage, but go with du -h if you want to estimate the size of your folders.

MEASURE NETWORK USAGE

You will have a general idea of your network interfaces and their IP addresses by using ifconfig or its modern substitute ip address, but it won't show you bandwidth usage.

For that we have netstat -i in a more static view and ifstat in a continuous view. To interrupt ifstat just use CTRL+C.

But if you want more info on that traffic, sudo iftop -i eth0 is a nice display. Change eth0 for the interface you wish to capture traffic information. To exit the monitor view, type q to quit.

POSTING YOUR PROGRESS

Regularly posting your progress can be a helpful motivator. Feel free to post to the subreddit/community or to the discord chat a small introduction of yourself, and your Linux background for your "classmates" - and notes on how each day has gone.

Of course, also drop in a note if you get stuck or spot errors in these notes.

EXTENSION

If this was all too easy, then spend some time reading up on:

RESOURCES

Some rights reserved. Check the license terms here

26
Day 0 - Creating Your Own Server in the Cloud (but cheaper) (programming.dev)
 

INTRO

First, you need a server. You can't really learn about administering a remote Linux server without having one of your own - so today we're going to buy one!

Through the magic of Linux and virtualization, it's now possible to get a small Internet server setup almost instantly - and at very low cost. Technically, what you'll be doing is creating and renting a VPS ("Virtual Private Server"). In a datacentre somewhere, a single physical server running Linux will be split into a dozen or more Virtual servers, using the KVM (Kernel-based Virtual Machine) feature that's been part of Linux since early 2007.

In addition to a hosting provider, we also need to choose which "flavour" of Linux to install on our server. If you're new to Linux then the range of "distributions" available can be confusing - but the latest LTS ("Long Term Support") version of Ubuntu Server is a popular choice, and what you'll need for this course.

Signing up with a VPS

Sign-up is immediate - just provide your email address and a password of your choosing and you're in! To be able to create a VM, however, you may need to provide your credit card information (or other information for billing) in the account section.

Comparison

Provider Instance Type vCPU Memory Storage Price Trial Credits
Digital Ocean Basic Plan 1 1 GB 25 GB SSD $6.00 $200 / 60 days
Linode Nanode 1GB 1 1 GB 25 GB SSD $5.00 $100 / 60 days
Vultr Cloud Compute - Regular 1 1 GB 25 GB SSD $5.00 $250 / 30 days

For more details:

Create a Virtual Machine

The process is basically the same for all these VPS, but here some step-by-steps:

VM with Digital Ocean (or Droplet)

  • Choose "Manage, Droplets" from the left-hand sidebar. (a "droplet" is Digital Ocean's cute name for a server!)
  • Click on Create > Droplet
  • Choose Region: choose the one closes to you. Be aware that the pricing can change depending on the region.
  • DataCenter: use the default (it will pick one for you)
  • Choose an image: Select the image "Ubuntu" and opt for the latest LTS version
  • Choose Size: Basic Plan (shared CPU) + Regular. Click the option with 1GB Mem / 1 CPU / 25GB SSD Disk
  • Choose Authentication Method: choose "Password" and type a strong password for the root account.
  • Note that since the server is on the Internet it will be under immediate attack from bots attempting to "brute force" the root password. Make it strong!
  • Or, if you want to be safer, choose "SSH Key" and add a new public key that you created locally
  • Choose a hostname because the default ones are pretty ugly.
  • Create Droplet

VM with Linode (or Node)

  • Click on Create Linode (a "linode" is Linode's cute name for a server)
  • Choose an Distribution: Select the image "Ubuntu" and opt for the latest LTS version
  • Choose Region: choose the one closest to you. Be aware that the pricing can change depending on the region.
  • Linode Plan: Shared CPU + Nanode 1GB. This option has 1GB Mem / 1 CPU / 25GB SSD Disk
  • Linode Label: Choose a hostname because the default ones are pretty ugly.
  • Choose Authentication Method: on the "Root Password" and type a strong password for the root account.
  • Note that since the server is on the Internet it will be under immediate attack from bots attempting to "brute force" the root password. Make it strong!
  • And, if you want to be safer, click "Add An SSH Key" and add a new public key that you created locally
  • Create Linode

VM with Vultr

  • Choose "Products, Instances" from the left-hand sidebar. (no cute names)
  • Click on Deploy Server
  • Choose Server: Cloud Compute (Shared vCPU) + Intel Regular Performance
  • Server Location: choose the one closest to you. Be aware that the pricing can change depending on the region.
  • Server image: Select the image "Ubuntu" and opt for the latest LTS version
  • Server Size: Click the option with 1GB Mem / 1 CPU / 25GB SSD Disk
  • SSH Keys: click "Add New" and add a new public key that you created locally
  • Note that since that there's no option to just authenticate with root password, you will need to create a SSH key.
  • Server Hostname & Label: Choose a hostname for your server.
  • Disable "Auto Backups". They will not be required for the challenge and are only adding to the bill.
  • Deploy Now

Logging in for the first time with console

We are going to access our server using SSH but, if for some reason you get stuck in that part, there is a way to access it using a console:

Remote access via SSH

You should see a "Public IPv4 address" (or similar) entry for your server in account's control panel, this is its unique Internet IP address, and it is how you'll connect to it via SSH (the Secure Shell protocol) - something we'll be covering in the first lesson.

  • Digital Ocean: Click on Networking tab > Public Network > Public IPv4 Address
  • Linode: Click on Network tab > IP Addresses > IPv4 - Public
  • Vultr: Click on Settings tab > Public Network > Address

If you are using Windows 10 or 11, follow the instructions to connect using the native SSH client. In older versions of Windows, you may need to install a 3rd party SSH client, like PuTTY and generate a ssh key-pair.

If you are on Linux or MacOS, open a terminal and run the command:

ssh username@ip_address

Or, using the SSH private key, ssh -i private_key username@ip_address

Enter your password (or a passphrase, if your SSH key is protected with one)

Voila! You have just accessed your server remotely.

If in doubt, consult the complementary video that covers a lot of possible setups (local server with VirtualBox, AWS, Digital Ocean, Azure, Linode, Google Cloud, Vultr and Oracle Cloud).

Creating a working admin account

We want to follow the Best Practice of not logging as "root" remotely, so we'll create an ordinary user account, but one with the power to "become root" as necessary, like this:

adduser snori74

usermod -a -G admin snori74

usermod -a -G sudo snori74

(Of course, replace 'snori74' with your name!)

This will be the account that you use to login and work with your server. It has been added to the 'adm' and 'sudo' groups, which on an Ubuntu system gives it access to read various logs and to "become root" as required via the sudo command.

To login using your new user, copy the SSH key from root.

You are now a sysadmin

Confirm that you can do administrative tasks by typing:

sudo apt update

Then:

sudo apt upgrade -y

Don't worry too much about the output and messages from these commands, but it should be clear whether they succeeded or not. (Reply to any prompts by taking the default option). These commands are how you force the installation of updates on an Ubuntu Linux system, and only an administrator can do them.

REBOOT

When a kernel update is identified in this first check for updates, this is one of the few occasions you will need to reboot your server, so go for it after the update is done:

sudo reboot now

Your server is now all set up and ready for the course!

Note that:

  • This server is now running, and completely exposed to the whole of the Internet
  • You alone are responsible for managing it
  • You have just installed the latest updates, so it should be secure for now

To logout, type logout or exit.

When you are done

You should be safe running the VM during the month for the challenge, but you can Stop the instance at any point. It will continue to count to the bill, though.

When you no longer need the VM, Terminate/Destroy instance.

Now you are ready to start the challenge. Day 1, here we go!

11
Day 0 - Creating Your Own Server in the Cloud (programming.dev)
 

INTRO

First, you need a server. You can't learn about administering a remote Linux server without having one of your own - so today we're going to get one - completely free!

Through the magic of Linux and virtualization, it's now possible to get a small Internet server setup almost instantly - and at a very low cost. Technically, what you'll be doing is creating and renting a VPS ("Virtual Private Server"). In a data center somewhere, a single physical server running Linux will be split into a dozen or more Virtual servers, using the KVM (Kernel-based Virtual Machine) feature that's been part of Linux since early 2007.

In addition to a hosting provider, we also need to choose which "flavor" of Linux to install on our server. If you're new to Linux then the range of "distributions" available can be confusing - but the latest LTS ("Long Term Support") version of Ubuntu Server is a popular choice, and what you'll need for this course.

Signing up with a VPS

Sign-up is fairly simple - just provide your email address and a password of your choosing - along with a phone number for a 2FA or another second method of authentication. You will need to also provide your credit card information.

Comparison

Provider Instance Type vCPU Memory Storage Price* Trial Credits
AWS t2.micro 1 1 GB 8 GB SSD $18.27 Free Tier for 1 year
Azure B1 1 1 GB 30 GB SSD $12.26 $200 / 30 days + Free Tier for 1 year
GCP e2-micro 1 1 GB 10 GB SSD $ 7.11 $300 / 90 days
Oracle VM.Standard.E2.1.Micro 1 1 GB 45 GB SSD $19.92 $300 / 30 days + Always Free services
  • Estimate prices

On a side note, avoid IBM Cloud as much as you can. They do not offer good deals and, according to some reports from previous students, their Linux VM is tampered with enough to the point some commands do not work as expected.

Educational Packs

Create a Virtual Machine

The process is basically the same for all these VPS, but here are some step-by-steps:

VM with Oracle Cloud

  • Choose "Compute, Instances" from the left-hand sidebar menu.
  • Click on Create Instance
  • Choose a hostname because the default ones are pretty ugly.
  • Placement: it will automatically choose the one closest to you.
  • Change Image: Select the image "Ubuntu" and opt for the latest LTS version
  • Change Shape: Click on "Specialty and previous generation". Click VM.Standard.E2.1.Micro - the option with 1GB Mem / 1 CPU / Always Free-eligible
  • Add SSH Keys: select "Generate a key pair for me" and download the private key to connect with SSH. You can also add a new public key that you created locally
  • Create

Logging in for the first time

Select your instance and click "SSH", it will open a new window console. To access the root, type "sudo -i passwd" in the command line then set your own password. Log in by typing "su" and "password". Note that the password won't show as you type or paste it.

Remote access via SSH

You should see a "Public IPv4 address" (or similar) entry for your server in the account's control panel, this is its unique Internet IP address, and it is how you'll connect to it via SSH (the Secure Shell protocol) - something we'll be covering in the first lesson.

If you are using Windows 10 or 11, follow the instructions to connect using the native SSH client. In older versions of Windows, you may need to install a 3rd party SSH client, like PuTTY, and generate an ssh key pair.

If you are on Linux or MacOS, open a terminal and run the command:

ssh username@ip_address

Or, using the SSH private key, ssh -i private_key username@ip_address

Enter your password (or a passphrase, if your SSH key is protected with one)

Voila! You have just accessed your server remotely.

If in doubt, consult the complementary video that covers a lot of possible setups (local server with VirtualBox, AWS, Digital Ocean, Azure, Linode, Google Cloud, Vultr, and Oracle Cloud).

What about the root user?

Working on a different approach from smaller VPS, the big guys don't let use root to connect. Don't worry, root still exists in the system, but since the provider already created an admin user from the beginning, you don't have to deal with it.

You are now a sysadmin

Confirm that you can do administrative tasks by typing:

sudo apt update

(Normally you'd expect this would prompt you to confirm your password, but because you're using public key authentication the system hasn't prompted you to set up a password - and AWS has configured sudo* to not request one for "ubuntu").

Then:

sudo apt upgrade -y

Don't worry too much about the output and messages from these commands, but it should be clear whether they succeeded or not. (Reply to any prompts by taking the default option). These commands are how you force the installation of updates on an Ubuntu Linux system, and only an administrator can do them.

REBOOT

When a kernel update is identified in this first check for updates, this is one of the few occasions you will need to reboot your server, so go for it:

sudo reboot now

Your server is now all setup and ready for the course!

Note that:

  • This server is now running and completely exposed to the whole Internet
  • You alone are responsible for managing it
  • You have just installed the latest updates, so it should be secure for now

To logout, type logout or exit.

When you are done

You should be safe running the VM during the month for the challenge, but you can Stop the instance at any point. It will continue to count toward the bill, though.

When you no longer need the VM, Terminate/Destroy instance.

Now you are ready to start the challenge. Day 1, here we go!

13
Day 0 - Creating Your Own Local Server (programming.dev)
 

It's difficult to create a server in cloud without a credit card

We normally recommend using Amazon's AWS "Free Tier" or Digital Ocean - but both require that you have a credit card. The same is true of the Microsoft Azure, Google's GCP and the vast majority of providers listed at Low End Box (https://lowendbox.com/).

Some will accept PayPal, or Bitcoin - but typically those who don't have a credit card don't have these either.

WARNING: If you go searching too deeply for options in this area, you're very likely to come across a range of scammy, fake, or fraudulent sites. While we've tried to eliminate these from the links below, please do be careful! It should go without saying that none of these are "affiliate" links, and we get no kick-backs from any of them :-)

Cards that work as, or like, credit cards

But what if I don’t want to use a cloud provider? You can just work with a local virtual machine

You can run the challenge on a home server and all the commands will work as they would on a cloud server. However, not being exposed to the wild certainly loses the feel of what real sysadmins have to face.

If you set your own VM at a private server, go for the minimum requirements like 1GHz CPU core, 1GB RAM, and a couple of gigs of disk space. You can always adapt this to your heart's desire (or how much hardware you have available).

Our recommendation is: use a cloud server if you can, to get the full experience, but don't get limited by it. This is your server.

Download the Linux ISO

Go to the Official Ubuntu page and download the latest LTS (Long Term Support) available version.

NOTE: download the server version, NOT the desktop version.

Create a Virtual Machine with VirtualBox

Install VirtualBox, when ready:

  • Click on Machine > New
  • Give a name to your VM and select the Type Linux. Click Next.
  • Adjust hardware: 1024MB memory and 1 CPU (this is the minimum, but you can reserve more if your host machine can provide it). Click Next
  • Virtual hard disk: 2,5GB is minimum, 5GB is a good number. Click Next.
  • Finish but we're not done yet.
  • The new VM should show up in a list of VMs, select it.
  • Click on Machine > Settings
  • Click on Storage. Right-click on Controllet IDE, click on Optical Drive.
  • Select the Linux ISO you downloaded from the list if available, if not click Add and find it in your directories. Click Choose.
  • Click on Network and change the network adapter to Bridged Adapter.
  • Click OK
  • Click Start or Machine > Start > Normal Start.

Installing Linux

After a few seconds the welcome screen will load. At the end of each page there's DONE and BACK buttons. Use arrow keys and the enter key to select options. When you're ok with your selection, use the arrow key to go down to DONE and enter to go to the next page.

  • Welcome Screen: Select your language
  • Keyboard Configuration: Select Keyboard type
  • Choose type of install: Select Ubuntu Server (minimized). It comes with most of the packages you need without being bloated. It will install faster too.
  • Network Connections: If you have setup the VM to use a bridged adapter like instructed, you don’t really have to worry a lot. The installer will automatically detect the DHCP settings from your local network router and you just have to select DONE.
  • Configure Proxy: If your system requires any http proxy to connect to the internet enter the proxy address, otherwise just select DONE.
  • Configure Ubuntu archive mirror: Leave it as default. DONE.
  • Guided Storage Configurations: We are going to utilize the entire storage space reserved for this VM and that's why we select Use the Entire Disk option.
  • Storage Configuration: Leave it the standard storage configuration and select DONE. When prompted to confirm, don't worry. This will only use the VM disk, not your computer disk.
  • Profile Setup: Enter your name, your server’s name, your username and password. This user will be your administrator user in the system (or sudo), so don't forget this password.
  • Update to Ubuntu Pro: No. Skip for now.
  • SSH setup: Select on Install OpenSSH server because that’s how you will connect to your server later.
  • Featured Server Snaps: None of these packages are important now, just select DONE.
  • Installing System: Now you have to wait for a few minutes for your system to install. You can "cheat" and speed up the install by skipping the downloading of updates, select Cancel update and Reboot when it appears at the bottom of the page, a few moments later. You can complete the updates after the first boot. After the installation is complete the system will reboot automatically.

Logging in for the first time

After the first reboot, it will show a black screen asking for the login. That's when you use that username and password you created during the install.

Note: the password will not show up, not even ***, just trust that is taking it in.

If you need to find out the IP address for the VM, just type in the console:

ip address

That will give you the inet, i.e., the ip address. You will need that to connect with SSH.

Remote access via SSH

If you are using Windows 10 or 11, follow the instructions to connect using the native SSH client. In older versions of Windows, you may need to install a 3rd party SSH client, like PuTTY and generate a ssh key-pair.

If you are on Linux or MacOS, open a terminal and run the command:

ssh username@ip_address

Or, using the SSH private key, ssh -i private_key username@ip_address

Enter your password (or a passphrase, if your SSH key is protected with one)

Voila! You have just accessed your server remotely.

If in doubt, consult the complementary video that covers a lot of possible setups (local server with VirtualBox, AWS, Digital Ocean, Azure, Linode, Google Cloud, Vultr and Oracle Cloud).

You are now a sysadmin

Confirm that you can do administrative tasks by typing:

sudo apt update

Then:

sudo apt upgrade -y

Don't worry too much about the output and messages from these commands, but it should be clear whether they succeeded or not. (Reply to any prompts by taking the default option). These commands are how you force the installation of updates on an Ubuntu Linux system, and only an administrator can do them.

REBOOT

When a kernel update is identified in this first check for updates, this is one of the few occasions you will need to reboot your server, so go for it after the update is done:

sudo reboot now

Your server is now all set up and ready for the course!

Note that:

  • This server is now running but is not exposed to the Internet, i.e. other people will not be able to attempt to connect. We recommend you keep it that way. It is one thing to expose a server in the cloud, exposing your home network is another story. For your own security, don't do it.

To logout, type logout or exit.

When you are done

Just type:

sudo shutdown now

Or click on Force Shutdown

Some Other Options

Now you are ready to start the challenge. Day 1, here we go!

9
PLEASE READ THIS FIRST! HOW THIS WORKS & FAQ (programming.dev)
 

RESOURCES

HOW THIS WORKS

In a nutshell

  • Completely free and open source
  • Focused on practical skills
  • Heavily hands-on
  • Starts at the 1st Monday of each month
  • Runs for 20 weekdays (Mon-Fri)
  • Often points to curated external links, expanding on the topic of the day.
  • Much less ‘formal’ than RHEL or Linux Foundation training

Requirements

  • A cloud-based Ubuntu Linux server - full instructions on how to set this up are in the ‘Day 0’ lessons
  • Basic computer literacy - no prior knowledge of Linux is required but you should be fairly confortable operating your own Windows/Mac machine
  • Requires a daily commitment of 1-2 hours each day for a month but can be self-paced

FREQUENTLY ASKED QUESTIONS - FAQ

Is this course for me?

This course is primarily aimed at two groups:

  1. Linux users who aspire to get Linux-related jobs in industry, such as junior Linux sysadmin, devops-related work and similar, and
  2. Windows server admins who want to expand their knowledge to be able to work with Linux servers.

However, many others have happily used the course simply to improve their Linux command line skills or to learn Linux for the first time – and that’s just fine too.

Will I pass LPIC/RHCA/LFCS/Linux+ certification if I take this course?

NO! This is NOT a preparation course for any Linux certification exam. It can help you, sure, but please refer to a more specific cert training if that's what you are aiming for.

When does it start?

The course always starts on the first Monday of the month. One of the key elements of the course is that the material is delivered in 20 bite-sized lessons, one each workday.

How long does it take? How many hours should I dedicate to it?

Depending on your experience and dedication, you can expect to spend 1-2 hours going through each lesson. The first few days are pretty basic and it might take you just minutes, but there's generally some "Extension" items to spice things up a bit.

I just learned about the challenge and it's already on Day X. Should I wait for next month to start?

Only if you want to. The material is available year-round so you can totally self-pace this if you prefer.

Do I really need a cloud-based server?

Yes, if you’re in the target audience (see above) you definitely should. The fact that such a server is very remote, and open to attack from the whole Internet, “makes it real”. Learning how to setup such a VPS is also a handy skill for any sysadmin.

Instructions for setting up a suitable server with a couple of providers are in the "Day 0" lessons. By all means use a different provider, but ensure you use Ubuntu LTS (preferably the latest version) and either use public key authentication or a Long, Strong, Unique password (we also have instructions on how to do that).

Of course, you’re perfectly entitled to use a local VM, a Raspberry Pi or even just WSL instead – and all of these will work fine for the course material. Just keep in mind what you are missing.

But what if I don't have a credit card (or don't want to use one) to setup an AWS/Azure/GCP server?

Please read Day 0 - Creating Your Own Local Server. There are other options of cloud providers and different payment options. But if none of them works for you, try creating your own local VM.

But what if I don’t want to use a cloud provider? I have a server/VM at home.

Then use your server. Check the post Day 0 - Creating Your Own Local Server

Why Ubuntu, can I use another distro?

The notes assume Ubuntu Server LTS (latest version) and it would be messy to include instructions/variations for other distros (at least right now). If you use Debian or other Debian-based distros (Mint, Pop!OS, Kali) it will make little to no difference because they all have the same structure.

But if you choose RedHat-based distros (Fedora, CentOS, AlmaLinux) or distros like Arch, Gentoo, OpenSUSE, you yourself will need to understand and cope with any differences (e.g. apt vs yum vs pacman).

If none of those names make any sense to you, you shouldn't be picking distros. Go read Linux Journey first lesson instead.

Should I be stopping or terminating my server when not in use?

Using a free-tier VPS, the load of the course does not exceed any thresholds. You can leave it running during the challenge but it's good to keep an eye on it (i.e. don't forget about it later or your provider will start charging you).

I noticed there was a kernel update, but no one said to reboot.

Reboot it. This is one of the few occasions you will need to reboot your server, go for it. The command for that is sudo reboot now

I still have questions/doubts! What do I do?!

Feel free to post questions or comments in Lemmy, Reddit or chat using the Discord server.

If you are inclined to contribute to the material and had the means to do it (i.e. a github account) you can submit an issue to the source directly.

CREDITS

The magnificent Steve Brorens is the mastermind behind the Linux Upskill Challenge. Unfortunately, he passed away but not before ensuring the course would continue to run in his absence. We miss you, snori.

Livia Lima is the one currently maintaining the material. Give her a shout out on Mastodon or LinkedIn.

8
Day 21 - What next? (programming.dev)
 

What is this madness – surely the course was for just 20 days?

Yes, but hopefully you’ll go on learning, so here’s a few suggestions for directions that you might take.

Play with your server

You’re familiar with the server you used during the course, so keep working with it. Maybe uninstall Apache2 and install NGINX, a competing webserver. Keep a running stat on ssh “attackers”. Whatever. A free AWS will last a year, and a $5/mo server should be something you can easily justify.

Add services that you’ll use

You should now be capable of following tutorials on installing and running your own instance of Minecraft, Wordpress, WireGuard VPN, or Mediawiki. Expect to have some problems – it's all good experience!

Take a look at Server World for some inspiration.

Extend your learning

Stop browsing articles on Gnome, KDE or i3 – and start checking out any articles like “20 Linux commands every sysadmin should know”. Try these out, delve into the options. Like learning a foreign vocabulary, you will only be able to use these “words” if you know them!

Check out Linux Journey if you haven't already, specially if you are still pretty new to Linux and would like to see a different learning approach. Linux 101 Hacks is also a good resource.

Practice what you've learned with some challenges at SadServers.com. There you'll find a collection of scenarios where you have to do, fix or hack something in a Linux server. It's great to exercise your troubleshooting skills without messing with your own server.

To get crazy fast in the command line, try Command Line Challenge, practicelinux.com, learnshell.org and commandlinefu.com.

If your next level goal is to get into DevOps, take a look at the DevOps Roadmap.

Certifications

If you’re looking to do Linux professionally, and you don’t have an impressive CV or resume already, then you should be aiming at getting a Linux certification. There are really just three certs/tracks that count:

  • CompTIA Linux+ - one and done exam, distro independent but doesn't hold much value in the market. Do this if you don't want to get too deep into Linux, or you have other CompTIA tracks going on and an employer is paying for them.
  • LPI LPIC-1: Linux Administrator – Very extensive description of the coverage of their various certs/courses. You can go very deep with this exams, they cover everything you can think of pure Linux. Not so popular with employers but the knowledge certainly holds it value.
  • Red Hat – You could spend a lot of time and money here, but it might well pay off! Geared to RedHat Enterprise Linux distribution and its particularities, it is a practical exam (the others are multiple question) and it's well known in Enterprise circles, it really pops up in any resume.

Even if you don’t want/need certs, the outline of the topics in these references can give you a good idea of areas to focus on in your self-learning.

Affordable professional training

Show your appreciation!

Steve Brorens (@snori74) was a collector of postcards and enjoyed greatly all the "Snail Mail" he received from the students.

But since his passing there's nowhere to send postcards anymore. You can show your appreciation for the course by letting everyone else know how awesome it was! Recommend the course to other people, invite your friends to do the challenge together, have fun! Show the world you finished the challenge by posting about it on social media.

Contribute

Livia Lima is the one currently maintaining the material. But she's only one person and appreciates any help to keep this challenge running consistently every month, and available to everyone.

If you'd like to contribute, here a few things you can do:

  • Answer other students questions in our channels. Help a friend through the challenge.
  • Correct typos, dead links, etc by submitting a correction request to the source material.
  • Suggest improvements by submitting a feature request to the source material.
  • Help moderate Lemmy, Reddit or Discord. Are you a whiz in one (or more) of those platforms? Help admin them.
  • Support the infrastructure by donating or sponsoring. The challenge is free but the website servers and the domains costs money, so we appreciate if you can spare a buck.

Thanks for all and happy linuxing!

14
Day 20 - Scripting (programming.dev)
 

INTRO

Today is the final session for the course. Pat yourself on the back if you worked your way through all lessons!

You’ve seen that a continual emphasis for a sysadmin is to automate as much as possible, and also how in Linux the system is very “transparent” - once you know where to look!

Today, on this final session for the course, we’ll cover how to write small programs or “shell scripts” to help manage your system.

When typing at the Linux command-line you're directly communicating with "the command interpreter", also known as "the shell". Normally this shell is bash, so when you string commands together to make a script the result can be called either a '"shell script", or a "bash script".

Why make a script rather than just typing commands in manually?

  • It saves typing. Remember when we searched through the logs with a long string of grep, cut and sort commands? If you need to do something like that more than a few times then turning it into a script saves typing - and typos!
  • Parameters. One script can be used to do several things depending on what parameters you provide
  • Automation. Pop your script in /etc/cron.daily and it will run each day, or install a symlink to it in the appropriate /etc/rc.d folder and you can have it run each time the system is shut down or booted up.

YOUR TASKS TODAY

  • Write a short script that list the top 3 IP addresses that tried to login into your server

START WITH A SHEBANG!

Scripts are just simple text files, but if you set the "execute" permissions on them then the system will look for a special line starting with the two characters “#” and “!” - referred to as the "shebang" (or "crunchbang") at the top of the file.

This line typically looks like this:

 #!/bin/bash

Normally anything starting with a "#" character would be treated as a comment, but in the first line and followed by a "!", it's interpreted as: "please feed the rest of this to the /bin/bash program, which will interpret it as a script". All of our scripts will be written in the bash language - the same as you’ve been typing at the command line throughout this course - but scripts can also be written in many other "scripting languages", so a script in the Perl language might start with #!/usr/bin/perl and one in Python #!/usr/bin/env python3

YOUR FIRST SCRIPT

You'll write a small script to list out who's been most recently unsuccessfully trying to login to your server, using the entries in /var/log/auth.log.

Use vim to create a file, attacker, in your home directory with this content:

 #!/bin/bash
 #
 #   attacker - prints out the last failed login attempt
 #
 echo "The last failed login attempt came from IP address:"
 grep -i "disconnected from" /var/log/auth.log|tail -1| cut -d: -f4| cut -f7 -d" "

Putting comments at the top of the script like this isn't strictly necessary (the computer ignores them), but it's a good professional habit to get into.

To make it executable type:

chmod +x attacker

Now to run this script, you just need to refer to it by name - but the current directory is (deliberately) not in your $PATH, so you need to do this either of two ways:

 /home/support/attacker
 ./attacker

Once you're happy with a script, and want to have it easily available, you'll probably want to move it somewhere on your $PATH - and /usr/local/bin is a normally the appropriate place, so try this:

sudo mv attacker /usr/local/bin/attacker

...and now it will Just Work whenever you type attacker

EXTENDING THE SCRIPT

You can expand this script so that it requires a parameter and prints out some syntax help when you don't give one. There are a few new tricks in this, so it's worth studying:

 #
 ##   topattack - list the most persistent attackers
 #
 if [ -z "$1" ]; then
 echo -e "\nUsage: `basename $0` <num> - Lists the top <num> attackers by IP"
 exit 0
 fi
 echo " "
 echo "Persistant recent attackers"
 echo " "
 echo "Attempts      IP "
 echo "-----------------------"
 grep "Disconnected from authenticating user root" /var/log/auth.log|cut -d: -f 4 | cut -d" " -f7|sort |uniq -c |sort -nr |head -$1

Again, use vim to create "topattack", chmod to make it executable and mv to move it into /usr/local/bin once you have it working correctly.

(BTW, you can use whois to find details on any of these IPs - just be aware that the system that is "attacking" you may be an innocent party that's been hacked into).

A collection of simple scripts like this is something that you can easily create to make your sysadmin tasks simpler, quicker and less error prone.

If automating and scripting many of your daily tasks sounds like something you really like doing, you might also want to script the setup of your machines and services. Even though you can do this using bash scripting like shown in this lesson, there are some benefits in choosing an orchestration framework like ansible, cloudinit or terraform. Those frameworks are outside of the scope of this course, but might be worth reading about.

And yes, this is the last lesson - so please, feel free to write a review on how the course went for you and what you plan to do with your new knowledge and skills!

RESOURCES

Some rights reserved. Check the license terms here

12
Day 19 - Inodes, symlinks and other shortcuts (programming.dev)
 

INTRO

Today's topic gives a peek “under the covers” at the technical detail of how files are stored.

Linux supports a large number of different “filesystems” - although on a server you’ll typically be dealing with just ext3 or ext4 and perhaps btrfs - but today we’ll not be dealing with any of these; instead with the layer of Linux that sits above all of these - the Linux Virtual Filesystem.

The VFS is a key part of Linux, and an overview of it and some of the surrounding concepts is very useful in confidently administering a system.

THE NEXT LAYER DOWN

Linux has an extra layer between the filename and the file's actual data on the disk - this is the inode. This has a numerical value which you can see most easily in two ways:

The -i switch on the ls command:

 ls -li /etc/hosts
 35356766 -rw------- 1 root root 260 Nov 25 04:59 /etc/hosts

The stat command:

 stat /etc/hosts
 File: `/etc/hosts'
 Size: 260           Blocks: 8           IO Block: 4096   regular file
 Device: 2ch/44d     Inode: 35356766     Links: 1
 Access: (0600/-rw-------)  Uid: (  0/   root)   Gid: (	0/	root)
 Access: 2012-11-28 13:09:10.000000000 +0400
 Modify: 2012-11-25 04:59:55.000000000 +0400
 Change: 2012-11-25 04:59:55.000000000 +0400

Every file name "points" to an inode, which in turn points to the actual data on the disk. This means that several filenames could point to the same inode - and hence have exactly the same contents. In fact this is a standard technique - called a "hard link". The other important thing to note is that when we view the permissions, ownership and dates of filenames, these attributes are actually kept at the inode level, not the filename. Much of the time this distinction is just theoretical, but it can be very important.

TWO SORTS OF LINKS

Work through the steps below to get familiar with hard and soft linking:

First move to your home directory with:

cd

Then use the ln ("link") command to create a “hard link”, like this:

ln /etc/passwd link1

and now a "symbolic link" (or “symlink”), like this:

ln -s /etc/passwd link2

Now use ls -li to view the resulting files, and less or cat to view them.

Note that the permissions on a symlink generally show as allowing everthing - but what matters is the permission of the file it points to.

Both hard and symlinks are widely used in Linux, but symlinks are especially common - for example:

ls -ltr /etc/rc2.d/*

This directory holds all the scripts that start when your machine changes to “runlevel 2” (its normal running state) - but you'll see that in fact most of them are symlinks to the real scripts in /etc/init.d

It's also very common to have something like :

 prog
 prog-v3
 prog-v4

where the program "prog", is a symlink - originally to v3, but now points to v4 (and could be pointed back if required)

Read up in the resources provided, and test on your server to gain a better understanding. In particular, see how permissions and file sizes work with symbolic links versus hard links or simple files

The Differences

Hard links:

  • Only link to a file, not a directory
  • Can't reference a file on a different disk/volume
  • Links will reference a file even if it is moved
  • Links reference inode/physical locations on the disk

Symbolic (soft) links:

  • Can link to directories
  • Can reference a file/folder on a different hard disk/volume
  • Links remain if the original file is deleted
  • Links will NOT reference the file anymore if it is moved
  • Links reference abstract filenames/directories and NOT physical locations.
  • They have their own inode

EXTENSION

RESOURCES

Some rights reserved. Check the license terms here

9
Day 18 - Log rotation (programming.dev)
 

INTRO

When you’re administering a remote server, logs are your best friend, but disk space problems can be your worst enemy - so while Linux applications are generally very good at generating logs, they need to be controlled.

The logrotate application keeps your logs in check. Using this, you can define how many days of logs you wish to keep; split them into manageable files; compress them to save space, or even keep them on a totally separate server.

Good sysadmins love automation - having the computer automatically do the boring repetitive stuff Just Makes Sense.

YOUR TASKS TODAY

  • Check the logs for apache2 that are Severity 3
  • Edit logrotate configuration for apache2 to rotate daily

ARE YOUR LOGS ROTATING?

Look into your logs directories - /var/log, and subdirectories like /var/log/apache2. Can you see that your logs are already being rotated? You should see a /var/log/syslog file, but also a series of older compressed versions with names like /var/log/syslog.1.gz

WHEN DO THEY ROTATE?

You will recall that cron is generally setup to run scripts in /etc/cron.daily - so look in there and you should see a script called logrotate - or possibly 00logrotate to force it to be the first task to run.

CONFIGURING LOGROTATE

The overall configuration is set in /etc/logrotate.conf - have a look at that, but then also look at the files under the directory /etc/logrotate.d, as the contents of these are merged in to create the full configuration. You will probably see one called apache2, with contents like this:

 /var/log/apache2/*.log {
 weekly
 missingok
 rotate 52
 compress
 delaycompress
 notifempty
 create 640 root adm
 }

Much of this is fairly clear: any apache2 .log file will be rotated each week, with 52 compressed copies being kept.

Typically when you install an application a suitable logrotate “recipe” is installed for you, so you’ll not normally be creating these from scratch. However, the default settings won’t always match your requirements, so it’s perfectly reasonable for you as the sysadmin to edit these - for example, the default apache2 recipe above creates 52 weekly logs, but you might find it more useful to have logs rotated daily, a copy automatically emailed to an auditor, and just 30 days worth kept on the server.

RESOURCES

TROUBLESHOOT AND MAKE A SAD SERVER HAPPY!

Practice what you've learned with some challenges at SadServers.com:

Some rights reserved. Check the license terms here

9
Day 17 - Build from the source (programming.dev)
 

INTRO

A few days ago we saw how to authorise extra repositories for apt-cache to search when we need unusual applications, or perhaps more recent versions than those in the standard repositories.

Today we're going one step further - literally going to "go to the source". This is not something to be done lightly - the whole reason for package managers is to make your life easy - but occasionally it is justified, and it is something you need to be aware of and comfortable with.

The applications we've been installing up to this point have come from repositories. The files there are "binaries" - pre-compiled, and often customised by your distro. What might not be clear is that your distro gets these applications from a diverse range of un-coordinated development projects (the "upstream"), and these developers are continuously working on new versions. We’ll go to one of these, download the source, compile and install it.

(Another big part of what package managers like apt do, is to identify and install any required "dependencies". In the Linux world many open source apps take advantage of existing infrastructure in this way, but it can be a very tricky thing to resolve manually. However, the app we're installing today from source is relatively unusual in being completly standalone).

FIRST WE NEED THE ESSENTIALS

Projects normally provide their applications as "source files", written in the C, C++ or other computer languages. We're going to pull down such a source file, but it won't be any use to us until we compile it into an "executable" - a program that our server can execute. So, we'll need to first install a standard bundle of common compilers and similar tools. On Ubuntu, the package of such tools is called “build-essential". Install it like this:

sudo apt install build-essential

GETTING THE SOURCE

First, test that you already have nmap installed, and type nmap -V to see what version you have. This is the version installed from your standard repositories. Next, type: which nmap - to see where the executable is stored.

Now let’s go to the "Project Page" for the developers http://nmap.org/ and grab the very latest cutting-edge version. Look for the download page, then the section “Source Code Distribution” and the link for the "Latest development nmap release tarball" and note the URL for it - something like:

 https://nmap.org/dist/nmap-7.70.tar.bz2

This is version 7.70, the latest development release when these notes were written, but it may be different now. So now we'll pull this down to your server. The first question is where to put it - we'll put it in your home directory, so change to your home directory with:

cd

then simply using wget ("web get"), to download the file like this:

wget -v https://nmap.org/dist/nmap-7.70.tar.bz2

The -v (for verbose), gives some feedback so that you can see what's happening. Once it's finished, check by listing your directory contents:

ls -ltr

As we’ve learnt, the end of the filename is typically a clue to the file’s format - in this case ".bz2" signals that it's a tarball compressed with the bz2 algorithm. While we could uncompress this then un-combine the files in two steps, it can be done with one command - like this:

tar -j -x -v -f nmap-7.70.tar.bz2

....where the -j means "uncompress a bz2 file first", -x is extract, -v is verbose - and -f says "the filename comes next". Normally we'd actually do this more concisely as:

tar -jxvf nmap-7.70.tar.bz2

So, lets see the results,

ls -ltr

Remembering that directories have a leading "d" in the listing, you'll see that a directory has been created :

 -rw-r--r--  1 steve  steve  21633731    2011-10-01 06:46 nmap-7.70.tar.bz2
 drwxr-xr-x 20 steve  steve  4096        2011-10-01 06:06 nmap-7.70

Now explore the contents of this with mc or simply cd nmap-7.70 - you should be able to use ls and less find and read the actual source code. Even if you know no programming, the comments can be entertaining reading.

By convention, source files will typically include in their root directory a series of text files in uppercase such as: README and INSTALLATION. Look for these, and read them using more or less. It's important to realise that the programmers of the "upstream" project are not writing for Ubuntu, CentOS - or even Linux. They have written a correct working program in C or C++ etc and made it available, but it's up to us to figure out how to compile it for our operating system, chip type etc. (This hopefully gives a little insight into the value that distributions such as CentOS, Ubuntu and utilities such as apt, yum etc add, and how tough it would be to create your own Linux From Scratch)

So, in this case we see an INSTALL file that says something terse like:

 Ideally, you should be able to just type:

 ./configure
 make
 make install

 For far more in-depth compilation, installation, and removal notes
 read the Nmap Install Guide at http://nmap.org/install/ .

In fact, this is fairly standard for many packages. Here's what each of the steps does:

  • ./configure - is a script which checks your server (ie to see whether it's ARM or Intel based, 32 or 64-bit, which compiler you have etc). It can also be given parameters to tailor the compilation of the software, such as to not include any extra support for running in a GUI environment - something that would make sense on a "headless" (remote text-only server), or to optimize for minimum memory use at the expense of speed - as might make sense if your server has very little RAM. If asked any questions, just take the defaults - and don't panic if you get some WARNING messages, chances are that all will be well.
  • make - compiles the software, typically calling the GNU compiler gcc. This may generate lots of scary looking text, and take a minute or two - or as much as an hour or two for very large packages like LibreOffice.
  • make install - this step takes the compiled files, and installs that plus documentation to your system and in some cases will setup services and scheduled tasks etc. Until now you've just been working in your home directory, but this step installs to the system for all users, so requires root privileges. Because of this, you'll need to actually run: sudo make install. If asked any questions, just take the defaults.

Now, potentially this last step will have overwritten the nmap you already had, but more likely this new one has been installed into a different place.

In general /bin is for key parts of the operating system, /usr/bin for less critical utilities and /usr/local/bin for software you've chosed to manually install yourself. When you type a command it will search through each of the directories given in your PATH environment variable, and start the first match. So, if /bin/nmap exists, it will run instead of /usr/local/bin - but if you give the "full path" to the version you want - such as /usr/local/bin/nmap - it will run that version instead.

The “locate” command allows very fast searching for files, but because these files have only just been added, we'll need to manually update the index of files:

sudo updatedb

Then to search the index:

locate bin/nmap

This should find both your old and copies of nmap

Now try running each, for example:

/usr/bin/nmap -V

/usr/local/bin/nmap -V

The nmap utility relies on no other package or library, so is very easy to install from source. Most other packages have many "dependencies", so installing them from source by hand can be pretty challenging even when well explained (look at: http://oss.oetiker.ch/smokeping/doc/smokeping_install.en.html for a good example).

NOTE: Because you've done all this outside of the apt system, this binary won't get updates when you run apt update. Not a big issue with a utility like nmap probably, but for anything that runs as an exposed service it's important that you understand that you now have to track security alerts for the application (and all of its dependencies), and install the later fixed versions when they're available. This is a significant pain/risk for a production server.

POSTING YOUR PROGRESS

Pat yourself on the back if you succeeded today - and let us know in the forum.

EXTENSION

Research some distributions where “from source” is normal:

None of these is typically used in production servers, but investigating any of them will certainly increase your knowledge of how Linux works "under the covers" - asking you to make many choices that the production-ready distros such as RHEL and Ubuntu do on your behalf by choosing what they see as sensible defaults.

RESOURCES

Some rights reserved. Check the license terms here

8
Day 16 - Archiving and compressing (programming.dev)
 

INTRO

As a system administrator, you need to be able to confidently work with compressed “archives” of files. In particular two of your key responsibilities; installing new software, and managing backups, often require this.

CREATING ARCHIVES

On other operating systems, applications like WinZip, and pkzip before it, have long been used to gather a series of files and folders into one compressed file - with a .zip extension. Linux takes a slightly different approach, with the "gathering" of files and folders done in one step, and the compression in another.

So, you could create a "snapshot" of the current files in your /etc/init.d folder like this:

tar -cvf myinits.tar /etc/init.d/

This creates myinits.tar in your current directory.

Note 1: The -v switch (verbose) is included to give some feedback - traditionally many utilities provide no feedback unless they fail. Note 2: The -f switch specifies that “the output should go to the filename which follows” - so in this case the order of the switches is important.

(The cryptic “tar” name? - originally short for "tape archive")

You could then compress this file with GnuZip like this:

gzip myinits.tar

...which will create myinits.tar.gz. A compressed tar archive like this is known as a "tarball". You will also sometimes see tarballs with a .tgz extension - at the Linux commandline this doesn't have any meaning to the system, but is simply helpful to humans.

In practice you can do the two steps in one with the "-z" switch, like this:

tar -cvzf myinits.tgz /etc/init.d/

This uses the -c switch to say that we're creating an archive; -v to make the command "verbose"; -z to compress the result - and -f to specify the output file.

TASKS FOR TODAY

  • Check the links under "Resources" to better understand this - and to find out how to extract files from an archive!
  • Use tar to create an archive copy of some files and check the resulting size
  • Run the same command, but this time use -z to compress - and check the file size
  • Copy your archives to /tmp (with: cp) and extract each there to test that it works

POSTING YOUR PROGRESS

Nothing to post today - but make sure you understand this stuff, because we'll be using it for real in the next day's session!

EXTENSION

  • What is a .bz2 file - and how would you extract the files from it?
  • Research how absolute and relative paths are handled in tar - and why you need to be careful extracting from archives when logged in as root
  • You might notice that some tutorials write "tar cvf" rather than "tar -cvf" with the switch character - do you know why?

RESOURCES

Some rights reserved. Check the license terms here

6
Day 15 - Deeper into repositories... (programming.dev)
 

INTRO

Early on you installed some software packages to your server using apt install. That was fairly painless, and we explained how the Linux model of software installation is very similar to how "app stores" work on Android, iPhone, and increasingly in MacOS and Windows.

Today however, you'll be looking "under the covers" to see how this works; better understand the advantages (and disadvantages!) - and to see how you can safely extend the system beyond the main official sources.

REPOSITORIES AND VERSIONS

Any particular Linux installation has a number of important characteristics:

  • Version - e.g. Ubuntu 20.04, CentOS 5, RHEL 6
  • "Bit size" - 32-bit or 64-bit
  • Chip - Intel, AMD, PowerPC, ARM

The version number is particularly important because it controls the versions of application that you can install. When Ubuntu 18.04 was released (in April 2018 - hence the version number!), it came out with Apache 2.4.29. So, if your server runs 18.04, then even if you installed Apache with apt five years later that is still the version you would receive. This provides stability, but at an obvious cost for web designers who hanker after some feature which later versions provide. (Security patches are made to the repositories, but by "backporting" security fixes from later versions into the old stable version that was first shipped).

WHERE IS ALL THIS SETUP?

We'll be discussing the "package manager" used by the Debian and Ubuntu distributions, and dozens of derivatives. This uses the apt command, but for most purposes the competing yum and dnf commands used by Fedora, RHEL, CentOS and Scientific Linux work in a very similar way - as do the equivalent utilities in other versions.

The configuration is done with files under the /etc/apt directory, and to see where the packages you install are coming from, use less to view /etc/apt/sources.list where you'll see lines that are clearly specifying URLs to a “repository” for your specific version:

 deb http://archive.ubuntu.com/ubuntu precise-security main restricted universe

There's no need to be concerned with the exact syntax of this for now, but what’s fairly common is to want to add extra repositories - and this is what we'll deal with next.

EXTRA REPOSITORIES

While there's an amazing amount of software available in the "standard" repositories (more than 3,000 for CentOS and ten times that number for Ubuntu), there are often packages not available - typically for one of two reasons:

  • Stability - CentOS is based on RHEL (Red Hat Enterprise Linux), which is firmly focussed on stability in large commercial server installations, so games and many minor packages are not included
  • Ideology - Ubuntu and Debian have a strong "software freedom" ethic (this refers to freedom, not price), which means that certain packages you may need are unavailable by default

So, next you’ll adding an extra repository to your system, and install software from it.

ENABLING EXTRA REPOSITORIES

First do a quick check to see how many packages you could already install. You can get the full list and details by running:

apt-cache dump

...but you'll want to press Ctrl-c a few times to stop that, as it's far too long-winded.

Instead, filter out just the packages names using grep, and count them using: wc -l (wc is "word count", and the "-l" makes it count lines rather than words) - like this:

apt-cache dump | grep "Package:" | wc -l

These are all the packages you could now install. Sometimes there are extra packages available if you enable extra repositories. Most Linux distros have a similar concept, but in Ubuntu, often the "Universe" and "Multiverse" repositories are disabled by default. These are hosted at Ubuntu, but with less support, and Multiverse: "contains software which has been classified as non-free ...may not include security updates". Examples of useful tools in Multiverse might include the compression utilities rar and lha, and the network performance tool netperf.

To enable the "Multiverse" repository, follow the guide at:

After adding this, update your local cache of available applications:

sudo apt update

Once done, you should be able to install netperf like this:

sudo apt install netperf

...and the output will show that it's coming from Multiverse.

EXTENSION - Ubuntu PPAs

Ubuntu also allows users to register an account and setup software in a Personal Package Archive (PPA) - typically these are setup by enthusiastic developers, and allow you to install the latest "cutting edge" software.

As an example, install and run the neofetch utility. When run, this prints out a summary of your configuration and hardware. This is in the standard repositories, and neofetch --version will show the version. If for some reason you wanted to be have a later version you could install a developer's Neofetch PPA to your software sources by:

sudo add-apt-repository ppa:ubuntusway-dev/dev

As always, after adding a repository, update your local cache of available applications:

sudo apt update

Then install the package with:

sudo apt install neofetch

Check with neofetch --version to see what version you have now.

Check with apt-cache show neofetch to see the details of the package.

When you next run "sudo apt upgrade" you'll likely be prompted to install a new version of neofetch - because the developers are sometimes literally making changes every day. (And if it's not obvious, when the developers have a bad day your software will stop working until they make a fix - that's the real "cutting edge"!)

SUMMARY

Installing only from the default repositories is clearly the safest, but there are often good reasons for going beyond them. As a sysadmin you need to judge the risks, but in the example we came up with a realistic scenario where connecting to an unstable working developer’s version made sense.

As general rule however you:

  • Will seldom have good reasons for hooking into more than one or two extra repositories
  • Need to read up about a repository first, to understand any potential disadvantages.

RESOURCES

Some rights reserved. Check the license terms here

view more: ‹ prev next ›