In 2002, I was a database administrator managing Oracle databases running on Solaris. One day I asked our system admin if he knew a way to find processes that consume the most I/O. He said he didn’t know. A Google or Yahoo search returned nothing. This surprised me because I thought many people would need this utility just as people need `top’ to check for CPU hogging processes. At the time I was slowly trudging through Jim Mauro’s Solaris Internals at night and on weekend (as my boss may frown upon me reading this book during the day as a DBA). I was always fascinated by OS internals but this book is a hard read probably or partially because I don’t have a CS background. Anyway, feeling the need to have a program to sort processes by I/O and reading about the dry concepts in Jim’s book with no hands-on practice combined triggered the thought of writing the topio program myself.

After some research, I found that the pseudo file /proc/pid/usage provides process I/O counters. The rest is simply writing code to read them, for all running processes, sample them at intervals, and sort the processes by the delta of the counters between two samplings. The low-level layer was written in C, using the skeleton program in the Appendix of Jim’s book as a starting point. The upper layer, written in the language I feel most comfortable with, Perl, does the rest of the work, i.e. sampling and sorting on delta. Since the single-screen refresh of `top’ seems unnecessary to me, my `topio’ just shows the result continually on screen, like `vmstat’, or `top’ run in batch mode. I put the source code and documentation on my personal website, which has very low visibility, but that doesn’t bother me.

Some time later, another idea came to my mind: Why not port topio to other OSes? In 2004, I was working at another company, after the previous company laid off all contractors. The job was not challenging so I had plenty of free time. I checked process statistics on Windows and found them in Win32_PerfRawData_PerfProc_Process class of the WMI Service. I rewrote the lower layer of my topio program to read from this service and porting to Windows was done. In 2006, HP still had the open access to HPUX free trial program. So I applied for a free shell account on HPUX and found where I can get these process counters. Porting to HPUX was done that year. Lastly, in 2010, porting to Linux was finished. But by this time, a French (?) software developer had already written the iotop program, which gradually became incorporated into major Linux distributions and is now known to most Linux users. Last time I checked his source code, it read the same pseudo file as my program does.

I know some people used my topio program or discussed it on various comp.os.* newsgroups, but the impact was minimal. Now, twenty years later, when I look back to this history, I think the experience of writing this program mostly gives me a sense of fulfillment and enjoyment or fun, instead of benefiting the IT community. As a side effect, it probably had helped me find jobs. In those years, I did DBA work at various companies as a contractor and had to find new jobs once in a while. My skills in OS, in addition to the primary skills in Oracle, were a big plus, even though there was no definitive evidence to prove they helped. (My tiny, crude, assembly program written in 1996–97 to allow a user to use a mouse to copy and paste text in DOS definitely helped me, because the senior programmer that hired me in 1999 told me so.) In 2023 and this year, IT professionals have a very hard time to find jobs. If I were a hiring manager, I would surely read a resume for a little longer if the candidate did some extra work, at night or on weekend if his/her daytime job does not permit it. When I started my IT career, I was disappointed at the fact that companies check candidates’ *years* of experience while I would love them to check people’s accumulated *hours* of experience, although that fantasy can never be realized. No matter. Interest in technology, enthusiasm, and energy have been and still are very much required of a good IT professional.

Creating an interesting project helps learn a new technology. As I said earlier, Solaris Internals was a tough read and writing topio helped me understand certain concepts as well as overcome boredom in reading the book. Twenty-eight years ago I wrote my DOSCSR (for “DOS cursor”) program when I was learning PC assembly. Ten years ago I decided to learn PHP and wrote small programs to do some word analysis in PHP instead of Perl, intentionally stepping out of my comfort zone (Perl) to challenge myself with what I decided to learn. A few years ago I decided to learn Python and started to write some scripts in Python at work, and told my manager and the DBA team that some scripts a little too complicated for bash to handle are better not written in Perl because future DBAs that will join and replace us, all in the middle age now, may be good at Python but not Perl. In short, to learn a new technology with an actually useful project is better than learning it with only learning materials. That’s why I don’t agree with LeetCode that tests people’s skills in programming for programming’s sake. Well, that’s another story to tell, or no story, as our generation are aging into irrelevance to these matters that we ought to leave to the youth of the nation and the world.

April 2024
also published on Medium