Clouded by doubt: Removing anxiety through monitoring the cloud

Mat Clothier at Cloudhouse describes how issues around misconfiguration, configuration drift, compliance and process auditing can be amplified by the quick provisioning and flexible environments that cloud infrastructure provides

 

The digital transformation accelerated by the pandemic has left many firms comfortably accommodated in a new cloud environment. And they are not alone. Spending on public cloud services is set to hit $591.8 billion in 2023, according to Gartner research.

 

The cloud provides many benefits, including inherent elasticity, flexibility and scalability, which means that lead time for new infrastructure is no longer a blocker to quickly spinning up new instances, services and applications, enabling companies to charge full speed ahead with their digital transformation.

 

But with the added agility the cloud brings, it presents some new complications in the form of visibility and integrity validation which threatens security and can cause the monthly cloud budget to rise unexpectedly. Initially, it can be a battle to simply understand what is hidden away in the cloud estate and what purpose it serves.

 

Despite rigorous housekeeping measures, changes caused by troubleshooting, ad hoc requests, fault fixes, one-off projects and multiple teams working within the same environment inevitably build up over time and resulting in confusion.

 

As misconfigurations and configuration drift accumulate, it’s time for security and network managers to put in place measures to tackle issues before the network not only becomes a security risk, but costs spiral out of control.

 

The key to developing oversight of a cloud environment, and therefore taking the first steps to tame it is to put proper monitoring software in place to shine a light across the entire environment and track how each element is performing.

 

Gaining oversight

The most important step is to gain visibility across the estate. This is crucial for many reasons, including being able to provide independently verifiable billing points, identify what’s out of date and non-compliant, and automatically achieve compliance by aligning with best practices. 

 

The second core element is cloud integrity validation, the process of checking whether the data and structures uploaded and stored in the cloud remains unchanged. This typically involves using security controls such as access controls, firewalls, and encryption to protect data and applications from unauthorised access, tampering, or deletion.

 

Managing configurations

Once end-to-end visibility has been established across the estate, nodes can be scanned to provide an indicator of the health of the environment. It also acts as a launch pad to identify further issues. Nodes should be scanned daily to maintain visibility into the systems being run.

 

Misconfigurations typically occur during troubleshooting and can involve cloud storage being set to public – spilling data to whoever is looking for it, or leaving default passwords unchanged. Monitoring tools alert to these issues and recommend the best course of action to take.

 

Putting a stop to configuration drift

Configuration drift is another significant problem that occurs when a system deviates from its original configuration, based on how the system is managed and used, after being imaged from a golden master. Drift can have adverse effects on the behaviour of services and the security of a system.

 

This issue is slow and almost invisible. Network states need to be recorded by monitoring tools, which will also help make comparisons to the original state and highlight where the deviations have occurred.

 

Addressing configuration drift creates a dependable infrastructure that enables the smooth movement of code through the DevSecOps pipeline with the assurance that all environments will function predictably. High availability pairs running on different software versions or clusters which no longer form a consensus will ring alarms in a central location and generate reports that are simple to action. Monitoring tools are helpful in making the most of expensive architecture and devices by ensuring that they are performing as designed.

 

Managing configuration drift creates a dependable infrastructure by ensuring that expensive architecture and devices are used to their fullest and they are performing as designed. In a DevSecOps pipeline, this ensures that all environments will function predictably and consistently.

 

In high availability scenarios, consensus can be tracked across the clusters and ring alarm bells when a device falls out of alignment. In any scenario, these tools provide a solution that validates environments are running as expected and where issues arise they are handled efficiently and effectively.

 

Managing processes across multiple environments

Before the cloud, errant processes would lead to clogged networks, backlogs, and probable downtime of some kind or another. But in the cloud, where applications and instances can scale in line with demand, rogue applications can lead to unexpected usage and the consequent increase in billing.

 

But tracking doesn’t have to be confined to the production environment. Staging environments are key to understanding how applications will perform, but the reality is there will always be some differences between the staging environment and the production environment.

 

Monitoring tools will provide a surface view of what is the same and what is different across the two environments, allowing developers to check that everything is configured as required, ensuring that the applications perform in the wild as they do in the test setting and maintaining a relevant test setting for the next development sprint.

 

Effective cloud management begins with monitoring

Sometimes, monitoring tools are put aside or neglected, but the cost of doing so is high, particularly in the cloud where a misconfigured node can rack up multiples in the monthly bill, or invite unwanted guests within the perimeter.

 

Additional benefits are also secured through accurately tracking instances and utilisation, which provide an independently verifiable source of truth should doubts appear over the accuracy of the cloud bill.

 

Providing instant visibility along with automated configuration management and monitoring tools are a fundamental aspect of taming even the most complex cloud environments.

 

---

 

Mat Clothier is CEO and Founder at Cloudhouse

 

Main image courtesy of iStockPhoto.com