Being a sysadmin is not sexy (part 1)

A sysadmin is someone whose job is the care and feeding of one or more servers (sysadmins frequently look after several servers). This is a job which is mostly boring (day-to-day stuff) but which is occasionally terrifying (when something unexpectedly stops working). There’s often not a lot in between those two things. If you are writing a story with a character who does this kind of work, don’t portray them as someone whose job is constant excitement, because no one will believe that.

A sysadmin’s celebrity among the non-technology people in the organization is often inversely proportional to how well she does her job. If this sysadmin is effective at her work, people tend not to know who she is. If that sysadmin is not good at her job, many people tend to know her name. This is a job where anonymity can be desirable.

Servers tend to run server applications (also known as services), which can be all kinds of things:

  1. A server might run an application commonly called a web server (Apache and Microsoft’s IIS are popular web servers). You typically interact with this service through your web browser.
  2. A server might be a file server. If you’ve ever used the “map a network drive” feature, then you’ve downloaded files from (or uploaded files to) a file server.
  3. If your organization runs a centralized accounting system, then that accounting system likely runs as a service on one or more servers. You might interact with this service through a desktop application or even a sophisticated web-based interface.
  4. email is another good example of a high-profile server application. Although users interact with email through a single desktop application (like Outlook), email usually happens by way of a pair of services: one for sending and receiving email (to and from other email servers), and one for retrieving email messages from the server to the desktop application.

A single server might run several different services.

Ideally a server and its server applications run smoothly and don’t need a lot of help beyond ordinary maintenance. This is the relatively tedious part of the job:

  • The server requires backups, which are frequently automated processes.
  • If the sysadmin manages a complicated application running on a server, that application may have many users. As people come and go, there’s a lot of giving this new person access to that, taking away that person’s access because they left, etc.
  • The server and its applications often have upgrades become available. This is a never-ending chore. It’s not uncommon for the organization to schedule a maintenance period during which the services will not be available to users. This is called “downtime,” and it is unpopular with users. The sysadmins use this time to apply the updates. This is typically routine, but can be stressful, because it can be difficult to recover from a failed upgrade (this may require restoring data and/or the operating system from a backup). These maintenance periods are commonly at night, over a weekend, or even during a holiday, which is not a fun time for the sysadmin to have to work.

In the next post we’ll look at what happens when things don’t go smoothly.

The worst explanation of networking, ever

(This post is going to introduce a lot of jargon, and I’ll probably refer to it from future posts.)

Making a network connection to a computer involves an IP address (the computer’s address on the internet) and a port number. This is a flimsy analogy, but you could think of it like finding your way into a house: the IP address (the host) is like the street address, and the port number is like which entrance to use (the front door, or the window on east side).

For example, most web sites are on port 80 or 443: 80 is the standard port for web servers, and 443 is the standard port for a secure web site (HTTPS). When you want your browser to display the CNN homepage, your browser figures out the IP address for (which at the moment appears to be, connects to port 80 on that host, and begins an HTTP transaction to download the home page.

A firewall is network software that allows or rejects network traffic according to a set of rules. An organization which hosts its own web site might have a firewall which allows the Internet to connect to the web server on allowed ports like 80 and 443, but the firewall would reject other inbound traffic to that host.

A port scanner is a program used to interrogate a host’s ports, looking for services listening on those ports. If you point a port scanner at a web server, it’s likely to tell you that a web service is listening on port 80 (and maybe also on port 443). Port scanners are powerful programs which can be used in many ways. A company might hire a security analyst who would use a port scanner to identify weak points on the company’s network (a good analyst would also give the company some suggestions about how to address those shortcomings). A malicious person might use a port scanner to the same effect but for a different reason: the port scanner can tell the attacker the ports on which services are listening on the company’s hosts, identifying targets he or she might try to compromise.

Sophisticated port scanners can identify specific software products listening on a port, and even the version of that software. If a cybercriminal used a port scanner to determine that a host was running version 2.2.15 of the Apache web server, he could use a search engine to look for vulnerabilities in that version of that product. He might find that that version has a race condition error resulting in a remote execution vulnerability and that someone has published an exploit which takes advantage of that software defect. He could download the exploit and run it against the web server for any number of malicious purposes (like stealing otherwise inaccessible information, sending spam and phishing attacks, etc.).

nmap has been a popular port scanner for a long time (it was first released in 1997). In The Matrix Reloaded, Trinity uses nmap to port scan a host she wants to compromise. nmap identifies a vulnerable version of a network service called ssh, and she then uses an exploit to elevate her privileges on that host. This is a good (and rare) example of credible hacking in popular media: the writer(s) included enough realistic detail to make the scene plausible.

nmap has starred in lots of films.

Statement of purpose

For years I’ve had a ridiculous fantasy of being a fiction writer. It seems that the best-selling novel I want to have written isn’t going to write itself. I’m having trouble getting motivated, so maybe what I need is another distraction: a blog.

I thought that technology in writing might be an interesting theme. Nothing ruins a story for me faster than a character hacking the FBI network after tapping on a keyboard for ten seconds. It probably works for many readers/viewers, but some of us see it as lazy writing.

In my day job I write lots of web applications for a public university. Many of my assignments are to convert paper processes into online forms. My job also involves a fair bit of Linux server administration. Most of this goes on the open Internet and is subject to daily cyber-attacks from all over the world (my server logs once revealed malicious traffic from Antarctica).

So the purpose of this blog has a couple of goals. One is to get me in the habit of writing. But I thought it might be useful to share some of what I’ve learned in a format that may be helpful to other prospective writers. I may also write about how technology can affect a writer. Here are some topics I have in mind:

  • credible hacking
    • port scanning
    • realistic exploitable security vulnerabilities
    • case studies of actual security breaches (like Target)
  • a writer’s technology
    • safe(r) Internet use (account security, security-related Firefox extensions, password managers)
    • affordable and effective backups
    • writing tools like scrivener and wordpress (I know a fair bit about the latter and would like to learn more about the former)
  • the day-to-day life of a web programmer
    • server administration is not sexy
    • the importance (and challenge) of making web sites accessible
    • the horrors of working with vendors and ticketing systems

This blog may at times earn a PG-13 rating. I’ll mostly keep it clean, but there may be the occasional bit of salty language.

I’ll try to post every seven to fourteen days (historically I’ve really struggled with self-imposed routines like that), and I’ll try to keep individual posts fairly short (preferring to break up longer topics into multiple posts).