Technology’s dangerous-by-design era to come to an end

“We’re thinking about these things in a really systemic way, that you shouldn’t be allowed to put things on a shelf for an ordinary Australian to come by and pick them up and buy them if you know those products are inherently unsafe,” O’Neil said.

Mind shift change

Declaring there needed to be “mind shift change”, O’Neil said “we need to use the power of government and the power of big institutions to help protect people better from this problem, and shift responsibility to those who can actually literally change it”.

At a practical level, she signalled the current internet-of-things regime her agency administers will move from the hopelessly inadequate voluntary code to a credentialing system that will mandate some sort of cyber safety rating system for consumers.

But it was her clarion call for the makers of digital services, the software vendors, to be accountable for the safety and security of their code that will be most profound.

Across multiple fronts, most notably in the scam, competition and privacy worlds, cabinet is facing proposals that similarly seek to shift responsibility for ensuring safety away from beleaguered consumers and onto the source of the problem, the lack of core safety-by-design protections.

With scams, consumers are literally seeing their savings disappear from their bank accounts, with banks unable or unwilling to stop customers hard earned savings being stolen by fraudsters.

At core, the banks as a sector have underinvested in their systems and controls. Which is why the UK has moved to make the sending and receiving banks jointly liable for these unauthorised withdrawal’s.

In Australia, Treasury is prepping what promises to be a convoluted code of mixed responsibilities between the banks, telcos and social platforms. Who is actually going to pay the consumer their lost money is not yet clear.

Similarly, in the privacy space, responsibility for privacy is effectively sheeted back to the very consumers who are meant to be protected, by forcing them to agree to long-winded legalistic privacy statements barely anyone comprehends.

The proposal before Attorney General Mark Dreyfus is to effectively end this fiction by giving consumers clear rights to make firms that are reckless with people’s personal information accountable.

The strategic intent is clear, that those who seek to use people’s data should be accountable if it is misused. This will also include small businesses, which until now are not even covered by privacy regulation. Small and medium firms make up 97 per cent of the firms in the economy, making the current regime laughable.

Cyber accountability

But it is in the cybersecurity space where this paradigm change to producer responsibility is likely to have its most profound impact.

This would end the current default model where end users are forced to shoulder most of the burden for cybersecurity rather than the industry that develops and makes the products.

Driven by a culture of “move fast and break things”, software vendors have released code with major flaws that hackers have exploited with gay abandon.

Software will always need updating, but on the first Tuesday of each month there are literally hundreds of (mostly Microsoft) software patches, pointing to the core problem of major platforms being riddled with poor code and vulnerabilities.

Consumers (of mostly Windows-based machines) have been forced to buy antivirus software, pay for VPNs to firewall their devices and almost ridiculously treat every hyperlink with suspicion.

Global push for safety

O’Neil’s US equivalent, Jen Easterly, the director of the federal Cybersecurity and Infrastructure Security Agency, earlier this year declared that society had “unwittingly come to accept as normal that technology is dangerous-by-design”.

“We often blame a company today that has a security breach because they didn’t patch a known vulnerability. What about the manufacturer that produced the technology that required too many patches in the first place?” Easterly said

“We’ve normalised the fact that technology products are released to market with dozens, hundreds, or thousands of defects, when such poor construction would be unacceptable in any other critical field.”

The Biden administration has signalled these laissez-faire days are over and is looking to codify best practices in secure code as a baseline which developers must follow.

Europe is similarly moving to a security-by-design default, with proposed legislation requiring vendors to commit to a full life cycle approach to code development.

This includes five years of security updates, mandatory third-party assurance for critical software such as firewalls and operating systems, and clear consumer labelling.

The bill has seen push back, especially from the commercial open-source community who claim it will scare off module developers and ruin the open-source innovation model.

But in the same way as the new UK banking scam rules are changing the business incentives towards investing in security, the global moves to finally make software vendors liable is already seeing the tech giants change their ways.

Apple long ago foresaw the inevitability of government action and is making privacy and security part of its brand promise, building in control features and application rules that protect its customers, rather than leaving them vulnerable.

Google is also running advertisements in The Australian Financial Review pitched at government decision makers, showcasing its security efforts.

There are more than enough international standards to build best practice codes around. Simple options such as requiring federal agencies to only buy compliant code would also go along way to changing the culture and practice of the big proprietary and commercial open-source vendors.

Here comes Gen AI

The global push for provider safety liability comes as the explosion in generative AI apps is fuelling major safety challenges.

In a highly influential article in the Foreign Affairs journal Open AI founder Mustafa Suleyman and governance expert Ian Bremmer argue traditional risk based regulatory approaches will not work for AI. They called for a techno-prudential approach, similar to financial prudential regulators.

“AI is different – different from other technologies and different in its effect on power,” the two thought leaders say.

“It does not just pose policy challenges; its hyper-evolutionary nature also makes solving those challenges progressively harder. That is the AI power paradox.

“The technology’s complexity and the speed of its advancement will make it almost impossible for governments to make relevant rules at a reasonable pace.”

They argue for a dedicated regime that is specific to the technology—very different from the generic risk-based approach, built on current consumer and corporate rules, Industry Minister Ed Husic is considering.