We had high hopes for the ICS Detection Challenge at S4x19. We had a better than ever set of packet captures in terms of size and diversity of systems, and a good inventory from the asset owner donor. Ron Brash and the team did a great job anonymizing the packet captures and inserting the attack events. And with 20+ vendors in the ICS detection space we thought the biggest problem would be when more than eight companies, our limit, wanted to compete.
Unfortunately we did not have that problem, we never had more than 3 active vendor competitors. Claroty and Dragos signed up early, and kudos to them for being up to facing the competitors in the Challenge. CyberBit joined in, but decided not to compete in November at about the same time that Kaspersky Lab joined the Challenge. With only three competitors, the Challenge became higher risk and lower reward for the competitors, resulted in another late but understandable drop out, and ultimately led to the Challenge being a late scratch for S4x19.
While very disappointed, the team tried to determine how to salvage all of the hard work and quite frankly amazing set of tests they had developed. The best part of the Challenge was going to be a multi-event incident that the competitors would need to identify, analyze and then show their analysis via their product interface to the judges. We have decided to proceed with that part of Challenge except it will not be scored, and we will not be announcing top tier winners (of little practical value when only two are involved).
During the Main Stage Detection Session on Thursday at 11:00 we will go over the multi-event incident, what the two detection solutions identified, and how that looked in the GUI’s. This should provide some insight into what ICS detection solutions can and cannot do and the analyst experience. We will also go over useful information from the Challenge team on tips to create your own ICS detection pilot or test with your system.