This blog is the first post in a new series titled Tales from the Field, where we will explore a variety of scenarios that our engineers, technicians, and researchers have faced while supporting customers in the field.
The Scene
A few years ago, a couple of us went to visit a large manufacturing site in the Midwest, where we were doing a pilot in a facility that manufactured a number of different materials on multiple production lines. This, coupled with a large variety of equipment types, meant the site had a complex architecture, and our goal was the work with the local IT and OT engineers to dissect and understand the different network areas.
As an ICS Security Consultant, my job was using Claroty Continuous Threat Detection (CTD) to identify connections between the various network areas so that I could provide the local site personnel a better view into them. While reviewing the traffic between assets, we repeatedly came across unusual communications between one network area and the internet. This concerned the local engineers, because the only connection to the outside world that should have been allowed is the one they used for updating equipment.
On first look it appeared that all of these communications were DNS queries, the type of thing you get when you try to visit a website. This isn't uncommon, as sometimes, browsers like Chrome or Explorer will automatically try to query the outside world, even if they can't get there, to check for updates. After some further investigation, to our surprise, all of these DNS queries were directed towards wedding websites like The Knot, Brides, and others, and there was definitive evidence that the asset was being used to browse these websites.
After some initial confusion and laughs, we learned two things. We were able to pinpoint the exact network path that led to the outside world, and that the operator in charge of that area was getting married! He had broken outside of the software on one of the assets using standard Windows commands (ex. alt+tab to cycle between open windows) that were not disabled so that he would be able to browse the websites between production batches.
While this was a particularly wholesome reason to browse the web between batches, it highlighted a very real security vulnerability within their industrial network. For starters, this path into their industrial network could have been exploited via a remote connection, granting an intruder direct access into the operational level of their network.
The Resolution
The large and segmented nature of this customer's site made it difficult to find these types of vulnerabilities in the past. The fact that the customer did not have an automated system to look for open network communications meant that they were relying on administrative procedures to manually check devices in a large manufacturing environment in order to prevent people from bypassing security policies.
To fix the issue we helped the customer enact changes to ensure this type of vulnerability was not created again in the future:
We reconfigured the firewall with additional rules to prevent these types of external communications from occurring in that part of the network.
The customer physically and logically hardened this and similar workstations to remove keyboard access and disable Windows commands that allowed users to break out of the device application.
We reviewed specific alerts and Virtual Zones within CTD to make sure there were no other instances of similar external communications and to ensure that an attacker or malware had not already managed to exploit the discovered vulnerability.
All in all what turned out to be less than a day's work in identifying, investigating, and resolving a network anomaly meant closing and further hardening against a potentially impactful network vulnerability. It really showed me that something as harmless as browsing the web for wedding ideas can be just what a malicious actor needs to gain a foothold in a critical part of a network.
To learn more about how Claroty reveals network vulnerabilities and works with customers to remediate them, request a demo.
