Category Archives: Operations

On April 11th, 2023 Microsoft added support for LAPS (Local Administrator Password Solution) natively to Windows Server. This is a major milestone because LAPS was historically an additional add-on item to a Windows system and the additional security it provides is critical for any Windows network.

What is it?

LAPS is a tool used to cycle local administrator account credentials on Windows systems. Historically all Windows systems come with a built-in Administrator account local to the machine for configuration and changes. It used to be common practice to set these accounts with some common password that the IT department could use in case of emergencies to work on the machines, but this very quickly became a security concern for a number of reasons:

1. Common passwords are a major vulnerability. If one user or malicious party gains access to that password, they’d essentially have an all access pass to every single system in your network, potentially including critical servers with sensitive data.
2. If an attacker knows that an account exists on every system (e.g. the default Administrator account that comes with all Windows installations) they can leverage that knowledge to save time and focus attacks on potentially compromising or using that account for access.
3. Traditionally local accounts are difficult to manage. Unlike Domain based accounts there is no central management, so without some kind of remote command execution on company systems (another potential security hole) it would be difficult to change or update these passwords across the board without logging into every machine and modifying the values.

Why do we need local admin accounts?

It is not unusual in an Active Directory Domain Services or Azure AD network to need a local administrator account to fix issues. Most problems that arise can be resolved with domain accounts, but sometimes local machines drop their trust with a domain unexpectedly or potentially a policy could be implemented with unexpected consequences that lock domain accounts out of a system. In these scenarios it is necessary to have some sort of local administrator account to fix the issues.

An additional note: It is also not unusual for organizations to need some level of administrative access to a machine to allow a user to perform actions like software installations, updates, or changes. While most of these scenarios can be managed effectively using proper policy architecture, small to medium sized businesses may not have the resources to perform those proper configurations. In these scenarios having a temporary admin account to use for updates or installs may be desirable, but there are multiple factors to consider when evaluating that.

How does LAPS help?

LAPS does a few things which dramatically improve security for these necessary accounts:

1. LAPS policies can be used to deploy a custom defined Local Administrator account (think localadmin rather than Administrator). This allows the organization to disable the built-in Administrator accounts without losing access to a local admin. NO ONE SHOULD EVER USE THE BUILT-IN ADMINISTRATOR ACCOUNTS!

1. LAPS policies can be used to cycle the passwords for this custom defined account on a regular basis. The password cycles automatically and then updates the value back to AD DS or Azure AD depending on your primary identity provider. This is great because even if someone uses the account and accidentally stores the password, your risk is still minimized because the password changes on it’s own at a regular interval.

1. LAPS policies allow each system to have a unique password for each local admin account. This means that even if an attacker were able to get a password for the local admin account on one machine, your risk is still minimized because each machines local account has a unique password value.

Why wouldn’t we be using this already?

LAPS was previously an add-on package for Windows Server and Desktops. Microsoft has made LAPS available for several years, but until this year an administrator had to go out and deploy the package for all systems. This added overhead and raised the “cost of entry” to a point that many organizations have overlooked it.

Additionally, cybersecurity was a historically overlooked topic in small to medium sized business networks. This means that many administrators traditionally overlooked these accounts preferring security through obfuscation versus something more effective like LAPS.

Ultimately, there is no longer an excuse to not be using LAPS. This system is an extremely exciting addition to what has historically been a major gap in enterprise security and it’s important that all network administrators start implementing these services across the board.

A question we get asked all the time working in IT Operations is “Why should I pay for support from <insert vendor name here>? That’s what I pay you guys for!” This seems like a logical argument: We as a support team are your one stop shop for all things technology. When you have an issue and need help you want to have the team available that you can call and know it will all get resolved quickly and accurately without worrying about where the problem originated or how it’s fixed. You want to pay a fixed amount in salaries for resources that will get the issues ironed out and if those issues aren’t getting fixed then you just need to find different resources.

There’s good and bad news for this logic. The good news: It’s totally possible to have the team you’re looking for! The bad news: It’s going to cost you just as much (if not more) to avoid vendor support than just paying for it. At the end of the day your technology is only getting more complex. Systems change, features are added, hardware components fail, and critical systems only become more critical as the business adopts and engages them in operations. This means that your team needs to adapt and learn everything about the application or equipment to anticipate potential failures and react quickly and completely when unexpected failures occur. How does your team get to that point? Training.

The ultimate question is whether you want to invest your support dollars in training your resources or having vendors available to respond in an adequate time frame to fix issues. Often training resources becomes less cost effective as continued learning needs to be constantly pursued to adapt to changing technology, so you end up investing more to keep your resources abreast versus paying the vendor to keep their resources available. You are also impacted by availability with your resources as in order to have an effective emergency plan you likely need two or three people trained to the proper extent in order to respond at times when one resource may be unavailable.

This is ultimately where the problems arise: companies avoid vendor support options and rely on a team of people that may not always be available, may not be completely trained on every potential failure mode of a system and how to respond, or may just ultimately not want to engage in keeping up to speed on the latest changes. This leads to outages, major impacts, and more often than not tense situations where you end up paying for the vendor support in a pinch anyways.

My recommendation will usually be to avoid those potential scenarios with critical systems and have the vendors at the ready to a degree that is reasonable for operations and the support team. Most vendors will offer options in the extent of support they provide (24×7, 8×5, 4 hour shipping on parts, next business day shipping, etc.) and this is where actual savings can be achieved. Assessing these options requires risk analysis that can start with some of these questions:

• How critical is this system?
• How much time can we wait in the event of a hardware related outage?
• When a feature does not work does that halt operations?
• What about the entire solution?
• Can we invest in redundancy to avoid major impacts from failure modes?

Every system will have different answers to these questions, so every system may require a different level of support.

There may also be systems where maintaining in house support makes more sense. A very critical system may be preferred to be handled by your IT team as they understand the operations, the impact, the appropriate parties to notify, and potentially can achieve a faster turn around than the vendor. Just know that in those scenarios you’re likely going to be spending more in training for your in house resources than you would for vendor support.