American View: How I Learned to Stop Worrying and (Just Barely) Accept ML Technology

I sat down to write this week’s column Sunday afternoon, less than a day after returning from this year’s Security Human Risk Summit in Las Vegas. Despite bringing home a notebook full of cool program ideas, staring at yet another blank new Microsoft Word document left me exasperated. After twelve years of crafting columns for the American View by-line, I still find it daunting trying to come up with a timely, entertaining, useful, and new topic every week… especially when I’ve got a backlog of laundry to do, groceries to put up, and appointments to schedule for the new workweek sitting just outside my office door. Ugggggh ... 

Fortunately, I remembered that I’d emailed myself a decent column idea from the summit. After an hour of clearing out backlogged new messages, I found it. As I’d suspected, I found a great premise from two lectures I’d sat on the potential use of (and limitations of) machine learning platforms in our mission space. Despite taking opposite tacks at the outset, both speakers eventually came to the roughly the same conclusion. 

First, Kerry Tomlinson from Ampere News delivered a great cautionary segment titled How ChatGPT can both help and hurt cybersecurity and awareness programs. She deftly explored the inherent weaknesses of a technology that’s completely devoid of self-awareness and prone to dangerous hallucinations. She walked us through specific prompts she’d entered and showed us how the tool’s outputs had consistently contradicted its own premises … when it wasn’t outright lying. 

A few lectures and coffee breaks later, Horatiu Petrescu of Aura Information Security gave an upbeat presentation about his own research titled Combining ChatGPT and Fogg Behaviour Model to Design Your Program. I was quite sceptical of Horatiu’s conclusions until we got the Q&A session; insightful queries from the audience allowed Horatiu to clarify that his advice to let ML tools “do the heavy lifting” for one’s program content goals was meant in terms of generating ideas, not finished products. Supplement and stimulant, not replacement.

This position synched up well Kerry’s: ML tools, she warned, are great predictors of text based on content models that they’ve studied but the tools themselves have no awareness of what they’re “saying.” They don’t understand anything; they only mimic the styles and phrasing of what others have written based on whatever libraries the tools were trained on. Use ML tools with extreme caution, she said. Triple-check everything they give you … because an ML tool’s output could well be complete drek.