BSides London is taking place and due to the pandemic and things, I’m not going and it’s put me in a contemplative mood about the early days of my career.
When I started there were no such things as conferences such as BSides. We only had Infosec Europe and the most we got out of there was some free USB sticks… If we were lucky they would be 500 megs.
A lotta things have changed since those days. Back then No one knew what secops was all about… There was none of this cyber malarkey we see nowadays. Not even IT in general new what we did. All they knew was that if they wanted their password reset and they didn’t want to go to the helpdesk – if they come to us with a box of chocolates and compliment and whether we’ve been working out and how we looked ripped we do it for them.
It’s so easy to manipulate anyone that works in infosec. Forget complex social engineering techniques – just go up to a guy, and ask him if he’s been working out? He will instantaneously become putty in your hands. Telling you about how his strong genetics and clean diet allow him to maintain a herculean physique even when the pressures of work keep him bound to a desk all day long.
If you want a rant – just ask what their views are on crossfit… which is guaranteed to give you a rant almost as long and passionate as if you ask them whether it’s better to have a mac or ‘proper’ linux operating system.
Yeah, infosec is full of characters. It always has been – it’s like the scene in Star Wars where Luke and Obi Wan walk into the bar, the camera pans across and there’s an assortment of freaky aliens.
But this is the industry I know and love and have been part of – so it’s a bit like family. You have the crazy uncle, the crazy grandparents, the crazy sibblings, the crazy parents – well there’s a whole ton of crazy.
I started back in 1999 on a one year work placement in Security Operations at a large bank. And from that time, there have been some things I’ve grown to love over the course of time and other things I absolutely hate.
Passwords
I mean take for example passwords – they were a funny thing even back then. We had a policy that dictated how passwords should be. The length, the strength, the expiry and rotation.
Users would have to reset the password every 90 days – some organisations still enforce that rule today. The problem is they’re like the analogy of 5 monkeys in the cage within which there is one banana. Any time a monkey goes for the banana all of the monkeys get hosed down with cold water. All the monkeys soon realised that it was a bad idea to go for the banana’s. When one monkey was removed and a new ‘dry’ monkey was introduced – if it went for the banana, all the other monkeys would beat it up because they didn’t want to get hosed down. One by one the moneys were replaced until you had all monkeys in the cage that had never been subjected to being hosed down. Yet when a new monkey went for a banana they’d all beat him up. Because that’s the way things have always been.
Looking at how security is done today, they resemble those monkeys in the cage. You ask them why they follow something and they say it’s because of policy – or because thats how it’s always been done. No-one ever says it’s because they’re blind monkeys.
Many companies still force their employees to reset their passwords every 90 days – you ask them why is that and they say it’s the policy – but they’re not too sure as to why. Some of them will try to quote something like a standard. We used to quote the orange book – yeah that’s right the DoD’s Trusted Computer System Evaluation Criteria (TCSEC)
It was the precursor to what you today will simply know as…
Compliance
Compliance, as far as I’m concerned has been helping cover asses since 1999. It’s one of those things I used to use as a crutch, but now I’ve seen the monster compliance has become – and hate it with a passion. Well, hate is probably a strong word but I am in a venting mood.
But it’s become far … far more powerful than I could have ever imagined.
What was used as a simple get out of jail card has become an industry in its own right. Compliance drives purchases, secures budgets and pulls in more people to the dark side.
But much like Anakins journey – this was a path to hell that was paved with good intentions.
There was the orange book, then came the ISO standards, then Enron happened and people lamented the loss of thousands of jobs – but where one opportunity closes, fate opens another door and it created millions of jobs for auditors wanting to become security people.
Just when you thought it couldn’t get any worse. Anakin killed the younglins and PCI DSS was born…
So if you want to know why compliance is such a mess these days… I apologise. It’s because 15/16 years ago security pros were too lazy to bother trying to understand or explain risk to companies – so they relied on the orange book – which I guess makes it kind of like a gateway drug – all the way through to the class A hard stuff that is PCI DSS… looking back, I was part of the generation that messed up the environment and the next generation are the ones that have to live with holes in the atmosphere, melting polar ice caps and dying horrible deaths.
As if compliance itself wasn’t bad enough – it fed and gave strength to security’s arch nemesis – the security people who aren’t – aka the anti-security people .. it created a whole industry of …
Auditors
Now you’re probably thinking I’m bitter – or maybe an auditor stole my lunch when I was young. And you know, you wouldn’t be too far wrong. The first time I encountered an auditor it was in 1999. Our secops team’s processes were being audited and I was to sit and talk the auditor through our Windows NT4.0 processes.
He was a friendly chap and engaged in lively banter… joking around about how its so difficult to find what you need and why there ends up being some random undeleted accounts. Stopping at one point to ask me how I preferred editing the registry and how we validate the hardened build.
Being the mug I was I went along with it all hook line and sinker…
When the audit report came in – it was like a hundred knives plunged into my back at once. I was betrayed. Do you know what it’s like to be betrayed at that level?
Once an auditors draft report was shared with us and one of the key findings was that we had documentation that was ‘grossly’ out of date… we were quite alarmed to hear that because we took a lot of care in making sure I documentation was up to date prior to the audit. So we asked to see a copy of the exact finding and I kid you not it said, the documentation was found to be deficient as it was grossly out of date. This along with other out of date documentation was found in the following directory path, “Security/documentation…../archived documentation”
Can you believe it? Documentation in an “archive” folder is out of date… that’s the whole point of an archive folder – it’s where you put stuff that is out of date. It’s like me coming to your house, looking through the bins and saying, “Mate, your leftover chicken is properly spoilt. I’m gonna report you to health and safety or something like that cos this is unacceptable levels of hygiene “
But then I met Andy – and he showed me the errors of my way. He told me that auditors were like mushrooms, you need to keep them in the dark and feed them shit.
The basic premise of dealing with an auditor is as follows:
- Be extremely polite to them – offer them tea and coffee as soon as they walk in. The more liquid they consume, the more toilet breaks they need.
- Take them out to lunch – a long lunch – that happens to be on the other side of town where traffic is horrendous. Feed them well.
- Finally the most important part is – give them the magical audit box. Which is basically a box that contains all the things you need to give to an auditor to ensure perfect results. In case that works too well – and it resulted in one of our managers once suggesting throwing something out there to let the auditors find that we can then remediate – win-win.
Separation of Duties
Separation of duties was and still is an important principle in security. It’s about splitting out an important task so that one person can’t do something totally irresponsible in isolation. Like you see in them movies where two people have to turn their keys at the same time before the door opens or they can launch the nukes.
But back in the day we had this separation of duties where a new HSM key (key change ceremony) needed to be loaded. I’d be in the team that would have half the password and another team would hold the other half.
Once a project was underway and it meant I’d have to fly up to the data centre with my half of the password and go and change the key with the help of a colleague.
The only problem is that – have you ever worked on a project? It’s never on time – always delayed.
And datacenters are COLD!
So here I was sat in a datacenter with this other guy who was about 50 with really really bad acne and BO – sitting under a blanket he bought, reading his book and munching on some snacks.
Here I was just getting bored listening to his munch munch munch and mouth-breathing.
Even worse, I had no idea what I was doing or how to do it… so I ended up having to ask mouth-breather to help me – which inevitably meant I gave him my half of the password and asked him to enter it… yeah, separation of duties kind of fell apart right there.
But it’s something we still see people getting wrong. Every time we see a rogue trader bringing a bank down – there wasn’t proper separation and accountability.
Privilege Account Management
Managing privileged ID’s is a big thing these days. Lots of companies are trying to sell you data vaults and secure triple encrypted systems where it encrypts the password separately from the identity and then puts all of that in a secure container within an encrypted container surrounded with monitoring controls that record every key stroke… not even Tom Cruise is mission impossibling his way in and out of there.
Then I think back to what we had as our privileged password piece of high tech technology.
We’d write the passwords down on a post-it sized piece of paper. Fold it up twice and pop it into a small envelope. Then we’d initial each corner of the envelope and apply cello tape around all edges to make it tamper evident.
Then on the envelope we wouldn’t write what userID that password belonged to… no no, we were far too clever for that. We’d simply write a number… that number corresponded to the row number on an excel spreadsheet which only the ops guys had access to which would say what the user ID was.
Hey the system worked. Every night one of us would have to walk the binder upstairs to the overnight ops team – then in the morning someone would walk it back down and we’d check to make sure the seal hadn’t been broken on any envelopes – and if it had been – we’d search for an incident that correlated with it.
Absolute genius.
I guess what I liked about that workaround isn’t that it was genius (it wasn’t and there were many flaws) but that we had freedom to be creative with our solutions and nobody expected it to be perfect as long as we saved the company half a million quid.
Monitoring
We had monitoring processes in place back in the days before SIEM’s – and it was just as ineffective and pointless as it is today in most organisations. Sure, your cloud provider has logs, your on prem servers have logs, your endpoints have logs – but unless you’re investing in a SOC or a good quality SIEM, and you can afford to pay a small army of analysts to go through the alerts it feels pretty much like Sisyphus continually pushing the boulder up the hill.
Incident management
I didn’t actually appreciate how much incidents taught me at the time. It was mainly a chance to earn some extra money by being on call despite having to carry a huge boulder of a laptop home and ask everyone at home to not use the phone as I dialled into the systems.
Reflections
I’ve rambled on about the bad old days a bit more than I intended to. But upon reflecting, while I may not have enjoyed some of my earlier jobs, they really did lay the foundation for my future career.
And so, I guess, my advice to anyone new in the industry (if you are seeking advice) is to be patient, put in the hours to learn your craft. Yes, you may have some great qualifications and full understanding of security concepts. But you can’t be an expert swimmer without getting into the pool. So dive in, spend your time not just applying and building your security knowledge, but getting to know people and the business you work in.
It’s not always about finding the next big problem to fix. But understanding what problems we do have, why they exist, and what we can do to just make them slightly less of an issue every day. I really do hope nobody is writing a blog in 20 years time complaining about why we still haven’t fixed passwords, privileged ID’s, monitoring, and whatever else there may be. Although that would make for good job security!
Javvad Malik 