This blog is the latest in a series titled Tales from the Field, where we explore a variety of scenarios that our engineers, technicians, and researchers have faced while supporting customers in the field.
It's not often that a vending machine is the cause of potentially tens of millions of dollars of loss for your company. We imagine the worst case scenario with this type of asset is that your employees lose a few coins every now and then, but when we connect anything to everything without proper visibility or segmentation, the attack surface of our networks grows exponentially.
A few years ago, I was part of a highly competitive proof-of-value (PoV) process at the flagship production site of a major consumer goods brand ahead of one of the largest sporting tournaments on the planet. The company which runs the site has a total of 300 facilities across the globe, and because of the sheer scale of onboarding, a new solution had invited multiple industrial cybersecurity vendors like Claroty to take part in the evaluation, KPI creation, and to provide implementation proposals. In a way, our very own league of industrial cybersecurity vendors was also having its biggest tournament of the year.
At the time, I was a pre-sales engineer at Claroty and led the tech side of the engagement and subsequent investigation. During a test deployment of Claroty Continuous Threat Detection (CTD) at the site, we began receiving alerts relating to a security breach. Since Claroty was the only one receiving these alerts, we began working with the potential customer to uncover their origins. The breach was the WannaCry ransomware. It had been picked up by one of CTD's threat detection engines, and as we began to investigate, the magnitude of the situation became clear–it had already spread to multiple sites around the globe and was busy encrypting production systems.
A Vending Machine?
The situation was clear, there was absolutely no scenario where this site could afford downtime given that they were on a full production schedule ahead of the major sports tournament. Recognizing this, Claroty immediately dispatched a SME on the WannaCry ransomware to assist me and the team in the investigation.
Our biggest challenge when approaching how to resolve the breach was a total lack of visibility across assets and network communications. Adding to this, a lack of proper segmentation meant the malware had spread to other sites with little to no effort and since we were not currently deployed across all 300 sites we had to think out of the box in order to identify, prioritize, and isolate affected parts of the network.
The first step was to act fast in identifying the entry point into the network. With little visibility into the overall network we identified—you guessed it—a breakroom vending machine to be the culprit. Whether this connection to the broader network was a mistake or done purposefully in order to process payment information and monitor inventory, this device was given unfettered access to an operational network worth billions of dollars due to a lack of proper visibility and segmentation.
From here, we followed a trail of network communications, relying on CTD to provide visibility into asset communications, quickly, at more sites along the way to reveal and map the propagation path. This allowed us to work with the potential customer to quickly isolate areas of the network which had been affected by the ransomware. Within hours of the discovery, we went from a single alert spawned by a vending machine to multi-site visibility that enabled the customer to halt the malware in its tracks. Through this action we were also able to extend their existing IT security controls (such as a combined IT/OT threat signature database and indicators of compromise) to the industrial environment.
Through the collaborative efforts of Claroty engineers and site personnel we were able to achieve a zero downtime outcome. With downtime costs at a company this size in their industry ranging above $5M per hour, this was the only acceptable solution.
To learn more about how Claroty works with you to provide visibility into even the most peripheral assets on your network, request a demo.
