Post #3104 J

• Corrupt, Arrogant, Self-Entitled, Unaccountable Federal Civil Servants. I've probably told this story a few times during the history of the blog. It's not clear given my limited readership that anyone ever picked up on the story, but it still outrages me 12.5 years later. Why does this story come to mind? There are stories of federal civil service employees conspiring to work against the Trump Administration. Now for those unacquainted with the statistics, certain pro-employee "protections" essentially make civil servants all but unfireable, especially after 1 year or so on the job. The termination rate is trivial--something like 0.2%, far below the comparable rate in the private sector. And the bulk of those terminations are in the first (unprotected) year. (Of course, the corrupt public sector unions have rationalizations; for example, the reason why so many public servants have much higher compensation (especially benefits) than comparable private sector positions, not to mention below-industry termination rates: the public sector is far "more selective" in hiring.  If you have ever been a contractor, you know the truth. We have often been the ones doing the real work. In my first federal gig at NASA-Clear Lake City (i.e., Houston), I was doing some FORTRAN programming work. Some of the civil servants I met were inspirational, this one guy was a logistics master, sort of like the Radar O'Reilly character in M*A*S*H. Others that I needed to meet, not so much: they spent most of their time shooting the breeze at the water fountain or in the hallways, taking long lunch and/or workout breaks, etc. Plus, the agency had a habit of raiding perm hires from contractors (no, that never happened to me, but contractors often go up against the end of short-term contracts, and my employers, on at least 3 or 4 cases that come to mind, lost their re-competes and/or had to reduce headcounts with the winning bid.)

In this one particular case I was living in the Chicago area after the 2000 recession, 9/11, and Arthur Andersen (Enron) disasters. A lot of consultancies went out of business, and my professional Oracle opportunities were at best limited and highly competitive. A lot of non-IT companies were worried about hiring consultants full-time and seeing them leave as soon as the economy bounced back. Even though I had cut my compensation requirements, the ones that were hiring had limited budgets and thought they could make do with junior staffers. Others were very highly selective.  In one case, this DBA had been promoted to a senior role and decided his successor had to have been employed by a Fortune 500 employer over the past 5 years.

So in 2003/2004 I had worked with this Indian immigrant-founded consultancy located in the DC metro Virginia suburbs on a bid for Oracle software installation project work in the New Orleans area. Apparently the bid wasn't submitted in time, not the first time that something like this fell through. I thought I would never hear from them again, when all of a sudden they called me up to ask if I would be interested in a full-time gig on their project for the National Archives. That's how I ended up living in Maryland for 10 years (no, not all of them working for the consultancy). I really didn't want to move to blue state Maryland, but the National Archives headquarters are located at the north end of the Beltway, very close to the University of Maryland in College Park. Employees in VA often faced 90-minute commutes to work.

I was basically the ERP database administrator for an emilrecs Siebel application at National Archives. For those who are unaware, there's an application where military veterans can apply for copies of their military records. This was an unusual contract where we also principally administered the server racks for the application, which didn't make the civil service staffers happy.

A couple of incidents stand out (in addition to the one I originally intended to discuss first). One day I came into work (probably 7-7:30 AM), and my colleagues tell me there's an auditor in our racks who is causing application servers to reboot. Now just to explain: we had a group of something like 450 hourly-paid contractors in the St. Louis area. An hour's outage cost something like a minimum of $10,000 to the government in terms of lost output, etc. No one had briefed me on an auditor's visit. Me, I had to do maintenance outside our primary support hours (say, install Oracle critical security patch updates), which I seem to recall were 7 AM - 7 PM daily, and I didn't get paid overtime. An unescorted auditor pulling computer equipment out of racks was sheer madness. The first thing I did was confront the operations staff, like how can you guys let this happen? They took pleasure at my dismay, saying that auditors were gods and if I got in their way, I would be terminated in a heartbeat. I then get to my racks and catch the auditor red-handed, pulling my production DLT drive out of my primary database server rack. He has this sheepish grin on his face and asks me what the hell he's holding. I'm thinking (but not saying), "Dude, if you don't know what it is, why have you taken it out of a production server rack? Are you crazy?" I didn't have an issue with the guy doing an audit--outside of our business operations window. I escalated the issue with my civil servant boss in St. Louis. I think for the most part the auditor was done, and luckily he didn't do major damage or cause a serious outage. But the operations staff loved the disruption and cheered on the auditor. They also played a minor role in a separate incident.

 I soon discovered in my multi-faceted role (I also did some Unix administration and some light networking stuff on our Cisco switch connections to servers) that one of the disk drives had died on our RAID-5 device (for those non-techies, a RAID-5 device basically allows up to 1 disk drive in an array to fail; it's like carrying a spare, and if a drive fails, the spare takes its place). You need to replace the failed drive, just like your first priority after a blowout and mounting your spare is to get a new tire; a second flat leaves your vehicle inoperable. Of course, for a DBA, you make damn sure you've got good backups. So a telltale sign is an amber (vs. green) light. So I see the amber light and I ask the operations staffers, "Are you blind? Can't you see the amber light? Don't you think you might have given us a heads up?" "Hey, buddy: those are YOUR servers: not our problem. Give us the contract, and we'll look after them." So much for government employees being all in on government computers. (I had no idea how long the drive was out, probably neglected by my predecessors; fortunately, I opened a support ticket in time with the  relevant vendor in question.)

But I saved the "best" for last. I soon discovered there was a disk mirroring issue for my test database server. (It was a weird problem to diagnose. I was encountering problems with file copies.) So while I'm having this problem ticket serviced to replace the drive in question, the guy says, "Dude, did you notice one of your (3) power supplies on the production database server is not up?" This was not the case of a defective power supply; it simply reflected the relevant cord was plugged into the wrong socket. It boggles my mind who signed off on the server installation without noticing the power supply wasn't working.

I know where to plug in the power supply but I immediately encountered bureaucratic resistance, that somehow the server might panic, risking application operations (including my boss' agreement). Not only that, but I, a mere mortal, cannot be trusted with that highly complex operation of inserting a plug into a socket; I needed to have an electrical engineer do this mighty feat. I finally say, "Fine, Have your engineer meet me at 7 PM during our maintenance period." The bureaucrat says, "Oh, hell no! Our engineers go home before 5 PM, and I'm not about to call one back here at 7 PM just to accommodate you." "When can you get it done?" "Well, our engineers get in here at 3:30 AM." I think we settled at 4 AM, but I was in at 3:30 AM. I look for the engineers and find them--only none of them know about a work order, and none of them are doing squat without a work order. I go up to the operations room and ask them if they've seen the engineer; they're openly laughing at me, shaking their heads no. I'm rather ticked off and go to Security; I asked them to call the bureaucrat in  question to locate his promised engineer. Security refuses. I basically wait until 4:30 AM and then dash off emails to him and my boss, go home (no traffic). The civil servant responds to my email that I must have just missed him and left too soon, that he had gotten stuck in traffic, was there at 4:10 AM and had been told that I had already left for home.  Now I had caught him in a deliberate lie; I had emails with later time stamps. I escalated it with my boss. (I think the same guy had a retirement party scheduled 2 or 3 weeks later; let's just say that I had better things to do.)

Oh, but I didn't get to the issue involving politicization at National Archives. I was working in an open cubicle cluster which included contractors and civil servants. It was hard not to overhear the civil servants watching JibJab cartoons and laughing at the parodies of Bush. And in particular, no lie, I heard them discussing the deliberate work slowdown "until President Kerry is in office". This was a shocking, deliberate corruption of the principles of our democratic republic. The Executive Branch is elected by the people, not by, of and for the self-serving benefit of 2M unaccountable civil servants. What they do in the privacy of their homes or voting booth is their business--but conspiring against doing the work funded by taxpayers and overseen by the duly-elected President is a criminal abomination. If you feel that you cannot in good conscience work for Trump, quit. But don't think you have the moral standing to subvert his lawful authority. You may disagree with policies, but you don't have the right to substitute your own.

I do not support Trump's immigration/refugee policies, but Sally Yates had a professional responsibility to represent the President's policies unless or until SCOTUS decides otherwise. I don't think Trump had an alternative but to terminate Yates for cause, and I fully support that. If Yates wants to impose her policies, she should run for President in 2020.

• Encryption and Federal Employee Emails.  Having recently become Security+ certified, I have a great deal of interest in email security. Hence, a recent Politico piece about EPA, Labor or other Administration federal employees using encryption technologies as away of masking their anti-Trump caught my eye.

At the risk of oversimplification, the way encrypted emails normally work is through the use of certificates and private/public keys. I can basically use someone's public key to encrypt an email message to him or her. (I can also sign my emails with my private key, which can assure a recipient than an email came from me, by using my public key.) The recipient can use his private key to decrypt my message. The use of a private key requires knowing its passcode/password. (I have a certificate/private key, but most of the people I normally correspond to haven't set up their own certificates.)

The use of encryption is generally a good thing, especially when the government is referencing certain private identifiable information (called PII), such as social security numbers, never mind sensitive/classified data, like state secrets. However, if you use sensitive government data for unauthorized uses, like Manning and Snowden, you can and will be prosecuted.

Federal employees and contractors regularly undergo security awareness training and know that there's no reasonable expectation of privacy in the use of government-owned infrastructure, including computing/networking resources. I have installed banner warnings on government servers. People who abuse their privileged access to government resources risk prosecution and imprisonment.

 Personally, I don't have an issue with what federal employees think about the President and what they say or do on their own time. I do have issues with people abusing their privileged access as government employees violating the public trust, by conspiring against the lawful mandates of the President. (The 'lawful' doesn't mean that you agree with Administration policy; public employees are not directly accountable to the people, but they are accountable via the three branches of government.)

Public employee communications are subject to the Federal Records Act. Transparency is critical to establish accountability. If you are insubordinate or subversive, you risk termination. If you are unwilling to serve under this President, resign. But whether or not I personally agree with you on policy, and I've probably disagreed with every POTUS in my lifetime for differing reasons, I've been a federal contractor for varying periods of time under 5 different Presidents, and I fulfill the terms of my contracts.

• Portraits of Our Fearless Leaders, Trump and Pence. It turns out some federal employees have been waiting 8 long years to be rid of Obama and Biden's photos in federal buildings, and the government, given its typical efficiency, was slow to the draw; apparently stock portraits of the new kids in town were in short supply. But fear not: I noticed earlier this week that the portrait of Trump was now scowling down on me as I came to work.