Problem Statement

During user interviews and contextual inquiries, we observed the time it took for analysts to search vast logs within USM (our SIEM platform) was a pain point we could eliminate for them. We looked for alternatives to the current “scroll and click” behavior of the search and filter rail in the platform.

Analysis

If the core problem while conducting searches and queries was the required motor-function and visual noise of unrelated filters within the search and filter rail, offering a fluid option for analysts to structure results could reduce time on task and improve accessibility.

Solutions

After careful analysis, I determined a natural language search could reduce friction across the platform by allowing direct query prompting instead of the current "scroll and click" behavior. To achieve this an AI assistant was conceptualized to present users. Next, the location and behavior for the assistant needed to be designed. We considered three possible implementations of an assistant:

One, a chat field floating over the existing UI.
Two, a standalone section where analysts could search through events and alarms using natural language.
Three, a copilot interface to leverage the learned search and filter location so the analyst could see changes in the table and data visualizations, intuitively.

We created HTML prototypes with Claude to test the usability and gather feedback quickly for our initial round of testing. Due to the substantial investment to implement, I determined we would test with two rounds, within subject for a total of 36 participants. Afterwards we had conclusive evidence that the third option, a copilot interface, was the strongest solution. We socialized the substantial user feedback supporting the AI assistant tool with product, engineering and sales. The teams unanimously supported the feature and we landed on a name for our AI assistant — SideKick.

A prototype, which pulled realtime data, was built to verify if the solution could simplify analysts' real-world workflows. Initial time on task had a baseline of 5-10 minutes using the traditional search interface, depending on the complexity of the parameters. With SideKick analysts were configuring search results in 2-5 minutes. After testing with 20 participants, we found that analysts spent 50% less time searching with SideKick. During testing we observed some users toggled from SideKick to the traditional interface to fine tune the results manually, which was a convenient outcome. Now analysts had the ability to search intuitively and then take immediate action with the results on the same screen

50% less time searching with SideKick

Outcomes

After implementing SideKick into the search and filter rail across the platform, we found that the assistant was being used in over 80% of searches. The search and filter page template is the most utilized of the three major page templates — which made the SideKick Search Assistant a powerful addition to the platform. Sales found that the functionality was a substantial selling point for both existing and perspective customers. The project was deemed a great success across the entire company and brought deserved acclaim to all teams involved.

For Later

With more funding and time I'm looking to add additional functionality to help analysts during their interactions with SideKick. Inline tools to search through existing alarms, events and investigations during prompt entries would help with cognitive load so users won’t have to memorize or copy/paste numeric IDs during searching. Another is dictation so users can use voice to conduct searches. Another valuable feature would be allowing SideKick to provide contextual suggestions pertaining to the users searches to help analysts take actions on mitigating threats and conducting investigations.