abstract: Setting up and managing a server, whether actual hardware, a virtual machine pretending to be a cloud server, or a real cloud VPS. Covers SSH setup, security, backing up, and Debian-specific cloud advice.
Setting up an actual server is similar to setting up the virtual machine modelling it, except you have to worry about the server getting overloaded and locking up.
On an actual server, it is advisable to enable passwordless sudo for one user.
issue the command visudo and edit the sudoers file to contain the line:
%sudo ALL=(ALL) NOPASSWD:ALLWhich means sudoers do not have to enter a password.
Check that none of the files /etc/sudoers.d/* helpfully undo what you just did. If they do, delete them.
If visudo or the files /etc/sudoers.d/* have a wheel group, then give the wheel group
the privilege of passwordless sudo, add your primary user to the wheel group, and continue
to require entering the password over and over and over again from the sudo group.
Add your primary user to the sudoers group: usermod -a -G sudo «username»
That user can now sudo any root command, with no password login nor
ssh in for root. And can also get into the root shell with sudo su -l root
On an actual server, you may want to totally disable passwords to accounts that have sensitive information. Unfortunately any method for totally disabling passwords is likely to totally disable ssh login, because the people writing the software have “helpfully” decided that that is what you probably intended, even though it is seldom what people want, intend, or expect. So the nearest thing you can do is set a long, random, non memorable password, and forget it.
If a server is configured with an ample swap file an overloaded server will lock up and have to be ungracefully powered down, which can corrupt the data on the server. If the swap file is inadequate, the OOM killer will shut down processes, which is also very bad, but does not risk losing data. So by default, servers tend to be out of the box configured with a grossly inadequate swap file, so that they will fail gracefully under overload, rather than locking up, needing to be powered down, and then needing to be recreated from scratch because of data corruption.
This looks to me like a kernel defect. The kernel should detect when it is thrashing the swap file, and respond by sleeping entire processes for lengthy and growing periods, and logging these abnormally long sleeps on wake. Swapping should never escalate to lockup, and if it does, bad memory management design, though this misfeature seems common to most operating systems.
When the OS detects the cpu idling while waiting for pages to be loaded into memory, should disable one process so its pages do not get loaded for a while, and derank all pages in memory that belong to that process, and derank all pages that belong to processes waiting on that process. When the cpu has idle time, and nothing to do for enabled processes, because everything they need has been done, and is only awaiting for disabled processes to get their pages loaded, then the OS can re-enable a disabled process, whereupon its virtualed paged get loaded back into physical memory, possibly resulting in some other process starting to thrash and getting disabled. So instead paging out the least recently used page, pages out an entire process, and stalls it until the cpu is adequately responsive to the remaining processes, and has been adequately responsive for a little while. This is inefficient, but it is a lot more efficient than a computer thrashing on paging. If the computer is stalling waiting on page load, then it is just running more processes than it can run, and the least recently used page algorithm is not going to accomplish anything useful. Some entire processes just have to be paged out, and stay paged out, until the remaining processes have completed and are idling. A thrashing computer is not running anything at all. Better that is run some things, and from time to time changes those things.
When the cpu has nothing to do because all the processes are waiting for pages to be loaded, something has to be done.
I prefer an ample swap file, larger than total memory, plus thrash protect,
which will result in comparatively graceful degradation, plus the existence of
the file /tmp/thrash-protect-frozen-pid-list will tell you that your
overloaded server is degrading (if it is not degrading, the file exists only briefly).
To have it look like a cloud server, but one you can easily snapshot and restore, set it up in bridged mode. Note the Mac address. After having it is running as a normal system, and you can browse the web with it, after guest additions and all that, then shut it down, go to your router, and give it a new static IP and a new entry in hosts.
Then configure ssh access to root. so that you can go ssh <server>as
if on your real cloud system. See setting up a server in the cloud
On a system that only I have physical access to and which runs no services that can be accessed from outside my local network my username is always the same and the password always a short easily guessed single word. Obviously if your system is accessible to the outside world, you need a strong password. An easy password could be potentially really bad if we have openssh-server installed, and ssh can be accessed from outside. If building a headless machine with openssh-server (the typical cloud or remote system) then need to set up public key sign in only, if the machine should contain anything valuable. Passwords are just not good enough – you want your private ssh key on a machine that only you have physical access to, and runs no services that anyone on the internet has access to, and which you don’t use for anything that might get it infected with malware, and you use that private key to access more exposed machines by ssh public key corresponding to that private key.
apt-get -qy update && apt-get -qy upgrade
# Fetches the list of available updates and
# strictly upgrades the current packagesTo automatically start virtual boxes on bootup, which we will need to do if publishing them, Open VirtualBox and right click on the VM you want to autostart, click the option to create a shortcut on the desktop, cut the shortcut. Open the windows 10“Run” box (Win+R) and enter shell:startup Paste the shortcut. But all this is far too much work if we are not publishing them.
If a virtual machine is always running, make sure that the close default is to save state, for otherwise shutdown might take too long, and windows might kill it when updating.
If we have a gui, don’t do openssh. Terminal comes up with Ctrl Alt T
When your hosing service gives you a server, you will probably initially
have to control it by password. And not only is this unsafe and lots of
utilities fail to work with passwords, but your local ssh client may well fail
to do a password login, endelessly offering public keys, when no
~/.ssh/authorized_keys file yet exists on the freshly created server.
To force your local client to employ passwords:
ssh -o PreferredAuthentications=password -o PubkeyAuthentication=no -o StrictHostKeyChecking=no root@«server»And then the first thing you do on the freshly initialized server is
apt update -qy
apt upgrade -qy
shutdown -r now && exitAnd the next thing you do is login again and set up login by ssh key, because if you make changes and then update, things are likely to break (because your hosting service likely installed a very old version of linux).
Login by password is second class, and there are a bunch of esoteric special cases where it does not quite 100% work in all situations, because stuff wants to auto log you in without asking for input.
Putty is the windows ssh client, but you can use the Linux ssh client in
windows in the git bash shell, which is way better than putty, and the
Linux remote file copy utility scp is way better than the putty utility
PSFTP, and the Linux remote file copy utility rsync way better than
either of them, though unfortunately rsync does not work in the windows bash shell.
The filezilla client works natively on both windows and linux, and it is very good gui file copy utility that, like scp and rsync, works by ssh (once you set up the necessary public and private keys.) Unfortunately on windows, it insists on putty format private keys, while the git bash shell for windows wants linux format keys.
Usually a command line interface is a pain and error prone, with a multitude of mysterious and inexplicable options and parameters, and one typo or out of order command causing your system to unrecoverably die,but even though Putty has a windowed interface, the command line interface of bash is easier to use.
(The gui interface of filezilla is the easiest to us, but I tend not to bother setting up the putty keys for it, and wind up using rsync linux to linux, which, like all comand line interfaces is more powerful, but more difficult and dangerous)
It is easier in practice to use the bash (or, on Windows, git-bash) to manage keys than PuTTYgen. You generate a key pair with
ssh-keygen -t ed25519 -f ssh_host_ed25519_key(I don’t trust the other key algorithms, because I suspect the NSA has been up to cleverness with the details of the implementation.)
On windows, your secret key should be in %HOMEPATH%/.ssh, on linux
in /home/«username»/.ssh, as is your config file for your ssh client, listing
the keys for hosts. The public keys of your authorized keys are in
/home/«username»/.ssh/authorized_keys, enabling you to login from afar as
that user over the internet. The linux system for remote login is a cleaner
and simpler system that the multitude of mysterious, complicated, and
failure prone facilities for remote windows login, which is a major reason
why everyone is using linux hosts in the cloud.
In Debian, I create the directory ~/.ssh for the user, and, using the
editor nano, the file authorized_keys
mkdir ~/.ssh
nano ~/.ssh/authorized_keys
chmod 700 .ssh
chmod 600 .ssh/*I set the ssh session host IP under /Session, the auto login username under /Connection/data, the autologin private key under /Connection/ssh/Auth.
If I need KeepAlive I set that under /Connection
I make sure auto login works, which enables me to make ssh do all sorts of
things, then I disable ssh password login, restrict the root login to only be
permitted via ssh keys.
In order to do this, open up the sshd_config file (which is ssh daemon
config, not ssh_config. If you edit this into the ssh_config file
everything goes to hell in a handbasket. ssh_config is the global
.ssh/config file):
nano /etc/ssh/sshd_configYour config file should have in it
HostKey /etc/ssh/ssh_host_ed25519_key
PermitRootLogin prohibit-password
PubkeyAuthentication yes
PasswordAuthentication no
UsePAM no
ChallengeResponseAuthentication no
AllowAgentForwarding yes
AllowTcpForwarding yes
GatewayPorts yes
X11Forwarding yes
TCPKeepAlive yes
PermitTunnel yes
ciphers chacha20-poly1305@openssh.com
macs hmac-sha2-256-etm@openssh.com
kexalgorithms curve25519-sha256
pubkeyacceptedkeytypes ssh-ed25519
hostkeyalgorithms ssh-ed25519
hostbasedacceptedkeytypes ssh-ed25519
casignaturealgorithms ssh-ed25519
# no default banner path
Banner none
PrintMotd no
# Allow client to pass locale environment variables
AcceptEnv LANG LC_*
# override default of no subsystems
Subsystem sftp /usr/lib/openssh/sftp-serverPermitRootLogin defaults to prohibit-password, but best to set it
explicitly Within that file, find the line that includes
PermitRootLogin and if enabled modify it to ensure that users can only
connect with their ssh key.
ssh out of the box by default allows every cryptographic algorithm under the sun, but we know the NSA has been industriously backdooring cryptographic code, sometimes at the level of the algorithm itself, as with their infamous elliptic curves, but more commonly at the level of implementation and api, ensuring that secure algorithms are used in a way that is insecure against someone who has the backdoor, insecurely implementing secure algorithms. On the basis of circumstantial evidence
and social connections, I believe that much of the cryptographic code used
by ssh has been backdoored by the nsa, and that this is a widely shared
secret.
They structure the api so as to make it overwhelmingly likely that the code will be used insecurely, and subtly tricky to use securely, and then make sure that it is used insecurely. It is usually not that the core algorithms are backdoored, as that the backdoor is on a more human level, gently steering the people using core algorithms into a hidden trap.
The backdoors are generally in the interfaces between layers, the apis, which are subtly mismatched, and if you point at the backdoor they say “that is not a backdoor, the code is fine, that issue is out of scope. File a bug report against someone else’s code. Out of scope, out of scope.”
And if you were to file a bug report against someone else’s code, they would tell you they are using this very secure NSA approved algorithm with the approved and very secure api, the details of the cryptography are someone else’s problem, “out of scope, out of scope”, and they have absolutely no idea what you are talking about, because what you are talking about is indeed very obscure, subtle, complicated, and difficult to understand. The backdoors are usually where one api maintained by one group is using a subtly flawed api maintained by another group.
The more algorithms permitted, the more places for backdoors. The certificate algorithms are particularly egregious. Why should we ever allow more than one algorithm, the one we most trust?
Therefore, I restrict the allowed algorithms to those that I actually use, and only use the ones I have reason to believe are good and securely implemented. Hence the lines:
ciphers chacha20-poly1305@openssh.com
macs hmac-sha2-256-etm@openssh.com
kexalgorithms curve25519-sha256
pubkeyacceptedkeytypes ssh-ed25519
hostkeyalgorithms ssh-ed25519
hostbasedacceptedkeytypes ssh-ed25519
casignaturealgorithms ssh-ed25519Not all ssh servers recognize all these configuration options, and if you
give an unrecognized configuration option, the server dies, and then you
cannot ssh in to fix it. But they all recognize the first three, HostKey, ciphers, macs which are the three that matter the most.
To put these changes into effect:
shutdown -r nowNow that putty can do a non interactive login, you can use plink to have a
script in a client window execute a program on the server, and echo the
output to the client, and psftp to transfer files, though scp in the Git Bash
window is better, and rsync (Unix to Unix only, requires rsync running on
both computers) is the best. scp and rsync, like git, get their keys from
~/.ssh/config
On windows, FileZilla uses putty private keys to do scp. This is a much more user friendly and safer interface than using scp – it is harder to issue a catastrophic command, but rsync is more broadly capable.
Life is simpler if you run FileZilla under linux, whereupon it uses the same keys and config as everyone else.
All in all, on windows, it is handier to interact with Linux machines
using the Git Bash command window, than using putty, once you have set
up ~/.ssh/config on windows.
Of course windows machines are insecure, and it is safer to have your
keys and your ~/.ssh/config on Linux.
Putty on Windows is not bad when you figure out how to use it, but ssh
in Git Bash shell is better:
You paste stuff into the terminal window with right click, drag stuff
out of the terminal window with the mouse, you use nano to edit stuff in
the ssh terminal window.
Once your you can ssh into your cloud server without a password, you now need to update it and secure it with ufw. You also need rsync, to move files around
apt-get -qy update && apt-get -qy install build-essential module-assistant dialog rsync ufw
cat /etc/default/ufw | sed 's/^\#*[[:blank:]]*MANAGE_BUILTINS[[:blank:]]*=.*$/MANAGE_BUILTINS=yes/g' >tempufw
mv tempufw /etc/default/ufw
chmod 600 /etc/default/ufw
ufw status verbose
ufw disable
ufw default deny incoming && ufw default allow outgoing
ufw allow ssh && ufw limit ssh/tcp
echo "Y
" |ufw enable && ufw status verboseThis is done by xrdp and a windowing system. I use Mate
The server should not boot up with the windowing system running because it mightily slows down boot, sucks up lots of memory, and because you cannot get at the desktop created at boot through xrdp – it runs a different instance of the windowing system.
The server should not be created as a windowing system, because the default install creates no end of mysterious defaults differently on a multi user command line system to what it does in desktop system, which is configured to provide various things convenient and desirable in a system like a laptop, but undesirable and inconvenient in a server. You should create it as a server, and install the desktop system later through the command line, over ssh, not through the install system’s gui, because the gui install is going to do mystery stuff behind your back.
Set up the desktop after you have remote access over ssh working At this point, you should no longer be using the keyboard and screen you used to install linux, but a remote keyboard and screen.
apt update && apt upgrade -y
apt install mate-desktop-environment
# on ubuntu apt install ubuntu-mate-desktop
systemctl get-default
systemctl set-default multi-user.target
# on a system that was created as a server,
# set-default graphical-target
# may not work anyway
apt install xrdp -y
systemctl start xrdp
systemctl status xrdp
systemctl stop xrdp
usermod -a -G ssl-cert xrdp
systemctl start xrdp
systemctl status xrdp
systemctl enable xrdp
ufw allow 3389
ufw reloadThis does not result in, or even allow, booting into mate desktop, because it does not supply the lightdm, X-windows and all that. It enables xrdp to run the mate desktop remotely
xrdp has its graphical login manager in place of lightdm, and does not have anything to display x-windows locally.
If you want the option of locally booting int mate desktop you also want lightDM and local X11, which is provided by:
apt update && apt upgrade -y
apt install task-mate-desktop$ systemctl status xrdp
● xrdp.service - xrdp daemon
Loaded: loaded (/lib/systemd/system/xrdp.service; enabled; vendor preset: enabled)
Active: active (running) since Sat 2024-01-06 20:38:07 UTC; 1min 19s ago
Docs: man:xrdp(8)
man:xrdp.ini(5)
Process: 724 ExecStartPre=/bin/sh /usr/share/xrdp/socksetup (code=exited, status=0/S>
Process: 733 ExecStart=/usr/sbin/xrdp $XRDP_OPTIONS (code=exited, status=0/SUCCESS)
Main PID: 735 (xrdp)
Tasks: 1 (limit: 2174)
Memory: 1.4M
CPU: 19ms
CGroup: /system.slice/xrdp.service
└─735 /usr/sbin/xrdp
systemd[1]: Starting xrdp daemon...
xrdp[733]: [INFO ] address [0.0.0.0] port [3389] mode 1
xrdp[733]: [INFO ] listening to port 3389 on 0.0.0.0
xrdp[733]: [INFO ] xrdp_listen_pp done
systemd[1]: xrdp.service: Can't open PID file /run/xrdp/xrdp.pid >
systemd[1]: Started xrdp daemon.
xrdp[735]: [INFO ] starting xrdp with pid 735
xrdp[735]: [INFO ] address [0.0.0.0] port [3389] mode 1
xrdp[735]: [INFO ] listening to port 3389 on 0.0.0.0
xrdp[735]: [INFO ] xrdp_listen_pp donersync is the openssh utility to synchronize directories locally and
remotely.
Assume rsync is installed on both machines, and you have root logon
access by openssh to the remote_host
Shutdown any daemons that might cause a disk write during backup, which would be disastrous. Login as root at both ends or else files cannot be accessed at one end, nor permissions preserved at the other.
rsync -aAXvzP --delete remote_host:/ --exclude={"/dev/*","/proc/*","/sys/*","/tmp/*","/run/*","/media/*","/lost+found"} local_backupOf course, being root at both ends enables you to easily cause catastrophe at both ends with a single typo in rsync.
To simply logon with ssh
ssh remote_hostTo synchronize just one directory.
rsync -aAXvzP --delete remote_host:~/name .To make sure the files are truly identical:
rsync -aAXvzc --delete «example.com»:~/name .rsync, ssh, git and so forth know how to logon from the
~/.ssh/config(not to be confused with /etc/ssh/sshd_config or
/etc/ssh/ssh_config
Host remote_host
HostName remote_host
Port 22
IdentityFile ~/.ssh/id_ed25519
User root
ServerAliveInterval 60
TCPKeepAlive yesGit on windows users %HOMEPATH/.ssh/config and that is how it knows
what key to use
To locally do a backup of the entire machine, excluding of course your
/local_backup directory which would cause an infinite loop:
rsync -raAvX --delete /
--exclude={"/dev/*","/proc/*","/sys/*","/tmp/*","/run/*","/local_backup/*",/
"/media/*","/lost+found"} /local_backupThe a and X options means copy the exact file structure with permission and all that recursively, The z option is for compression of data in motion. The data is uncompressed at the destination, so when backing up local data locally, we don’t use it.
To locally just copy stuff from the Linux file system to the windows file system
rsync -acv --del source dest/Which will result in the directory structure dest/source
To merge two directories which might both have updates:
rsync -acv source dest/A common error and source of confusion is that:
rsync -a dir1/ dir2means make dir2 contain the same contents as dir1, while
rsync -a dir1 dir2is going to put a copy of dir1 inside dir2
Since a copy can potentially take a very long time, you need the -v flag.
The -P flag (which probably should be used with the -c flag) does incremental backups, just updating stuff that has been changed. The -z flag does compression, which is good if your destination is far away.
Your cloud server is going to keep timing you out and shutting you down,
so if using OpenSSH need to set up ~/.ssh/config to read
ForwardX11 yes
Protocol 2
TCPKeepAlive yes
ServerAliveInterval 10Putty has this stuff in the connection configuration, not in the config file. Which makes it easier to get wrong, rather than harder.
Your cloud server is probably virtual private server, a vps running on KVM, XEN, or OpenVZ.
KVM is a real virtual private server, XEN is sort of almost a virtual server, and OpenVZ is a jumped up guest account on someone else’s server.
KVM vps is more expensive, because when they say you get 2048 meg, you actually do get 2048 meg. OpenVZ will allocate up to 2048 gig if it has some to spare – which it probably does not. So if you are running OpenVZ you can, and these guys regularly do, put far too many virtual private servers on one physical machine. Someone can have a 32 Gigabyte bare metal server with eight cores, and then allocate one hundred virtual servers each supposedly with two gigabytes and one core on it, while if he is running KVM, he can only allocate as much ram as he actually has.
Debian is significantly more lightweight than Ubuntu, harder to configure and use, will crash and burn if you connect up to a software repository configured for Ubuntu in order to get the latest and greatest software. You generally cannot get the latest and greatest software, and if you try to do so, likely your system will die on its ass, or plunge you into expert mode, where no one sufficiently expert can be found.
Furthermore, in the course of setting up Debian, highly likely to break it irretrievably and have to restart from scratch. After each change, reboot, and after each successful reboot, take a snapshot, so that you do not have to reboot all the way from scratch.
But, running stuff that is supposed to run, which not always the latest and greatest, is more stable and reliable than Ubuntu. Provided it boots up successfully after you have done configuring, will likely go on booting up reliably and not break for strange and unforeseeable reasons. Will only break because you try to install or reconfigure software and somehow screw up. Which you will do with great regularity.
On a small virtual server, Debian provides substantial advantages.
Go Debian with ssh and no GUI for servers, and Debian with lightdm Mate for your laptop, so that your local environment is similar to your server environment.
On any debian you need to run through the apt-get cycle till it stops updating:
apt-get -qy update && apt-get -qy upgrade
apt-get -qy install dialog dialog nano build-essential
apt-get -qy rsync linux-headers-genericOn windows, edit the command line of the startup icon for a virtual box that you have iconized to add the command line option --type headless, for example
"C:\Program Files\Oracle\VirtualBox\VirtualBoxVM.exe" --comment "vmname" --startvm --type headless "{873e0c62-acd2-4850-9faa-1aa5f0ac9c98}"To uninstall a package
apt-get -qy --purge remove <package>To uninstall a package without removing the settings
apt-get -qy remove <package>On your home computer, Ubuntu has significant ease of use advantages. On the cloud, where computing power costs and you are apt to have a quite large number of quite small servers, Debian has significant cost advantages, so perhaps should have Debian locally, despite its gross pain the ass problems, in order to have the same system in the cloud and locally.
reaction.la gpg key 154588427F2709CD9D7146B01C99BB982002C39F
This work is licensed under the Creative Commons Attribution 4.0 International License.