A robot beat Australia’s top fundies (but it still can’t count)

The test

Purcell asked the platforms to build an equally weighted, 23-share portfolio of businesses with attributes including a sustainable competitive advantage, strong management track record and financials, and which were trading below their intrinsic values.

ChatGPT’s stock suggestions were based on data up to September 2021, but Bard accesses up-to-date data. Purcell wanted 23 holdings, as that was the median number held by the 10 funds, excluding outliers.

“Bard tended to focus more on large cap technology stocks, but it also excluded cash ... and I think the outperformance of the Bard portfolio is a reflection of the market at the moment,” says Purcell.

“ChatGPT was more accurate when it came to reflecting the fund managers’ strategies and a little less risky and a bit more diversified across markets.”

The results

Between May 11 and August 30, 2023:

• Bard’s portfolio of 19 holdings rose 8.2 per cent.
• ChatGPT’s portfolio of 23 holdings rose 4.21 per cent.
• The active funds rose 6.3 per cent on average.
• The S&P 500 rose 9.3 per cent.
• The S&P/ASX 200 fell 0.63 per cent.

The upshot

We’re a long way from using AI to build investment portfolios, for at least one reason: Bard can’t count. Asked multiple times, Bard couldn’t create a portfolio of 23 stocks, says Purcell. Instead, the closest it could get was 19.

“We had the same problem just getting it to give us a certain level of exposure, even just broadly speaking, to European stocks,” says Purcell.

But it goes beyond this, says Michael Kollo, former fund manager and chief executive of AI advisory firm Evolved Reasoning.

Simply speaking, large language models such as ChatGPT and Bard harness available information, but they can’t predict the future.

“The only way that you can actually build a portfolio that beats the market is by trying to work out what people don’t yet know, perhaps by thinking about what the earnings announcements will look like,” he says.

To do that, you need to create a forecast either through empirical data or qualitative storytelling.

Analysing sentiment

“What large-language models do is exactly none of that,” he says. But, he adds, active managers can’t predict the future either.

A more useful way to use AI is to analyse sentiment around a company, and how likely investors are to be buoyed or disappointed by it in future, he says. That’s because the way sentiment is measured and recorded, such as in news articles, broker notes and more, requires a hefty amount of time and research.

“If you wanted to use these large-language models to track the sentiment of a security, and topic sentiment [such as views on whether a company was a polluter or not] ... this is where these kinds of models can be quite powerful,” he says.

Financial adviser Tim Mackay says investors should still exercise caution when using an AI app such as ChatGPT, as – in his experiences – it frequently offers information that is incorrect or out of date.

“It’s looking at investments [based on information] available two years ago. You’re driving, looking in the rearview mirror.”

Noting AI’s shortcomings, Australian Shareholders’ Association chief executive Rachel Waterhouse believes that while AI won’t spell the end of stock picking, it could change the game.

That’s because if everyone is using the same information, they’ll end up making similar decisions. Then the difference will come down to the quality of the prompts, the AI used and investors’ ability to parse the information.

She also believes that while AI’s ability to predict an investment’s collapse is limited, there’s a possibility that AI itself could trigger a black swan event.

“If everyone becomes so reliant on AI technology to make their decisions, and it’s telling us all that something should be sold, and then everyone sold it – that’s the part I’ve been thinking about.”