Australia has joined other countries in announcing a ban on the use of TikTok on government devices, with some states and territories following suit. The rationale was based on security fears and, in particular, the risk that the platform will be used for foreign interference operations by China.

TikTok is a video-sharing platform operated by ByteDance, a company headquartered in Beijing but incorporated in the Cayman Islands. Data is allegedly stored in the US and Singapore.

Like those of similar sites, TikTok’s privacy policy indicates an expansive approach to the collection and use of personal information. The app can collect information from users and third parties (such as advertisers), and it can draw inferences about its users’ interests. All of this information can then be shared with TikTok’s partners and service providers to, among other things, personalise content and advertising.

The policy also says information will be shared when there is a legal requirement to do so. China’s national intelligence law obliges citizens and organisations to support, assist and cooperate with national intelligence efforts, which could include ByteDance sharing people’s TikTok data.

While TikTok denies it would hand over data in such circumstances, there are reports that data from American users has been accessed by China-based employees. TikTok has also censored content that is politically sensitive in China.

While the Australian government’s response can be explained through this logic, questions remain.

Given the ban only affects government devices, couldn’t the same people be susceptible to foreign interference through their use of TikTok on personal devices? And what about other apps, such as Facebook, that collect significant amounts of user data. Are they more secure than TikTok?

Even if other digital platforms don’t have connections with China, couldn’t they share or sell data to other entities, such as advertisers, data brokers or business partners? And mightn’t those third parties have connections with China? Or other countries with similar laws?

But the problem of digital security and foreign interference is bigger than just one app or the use of government devices. Russia has run information campaigns designed to influence US elections using platforms such as YouTube, Tumblr, Google, Instagram, PayPal, Facebook and Twitter.

Indeed, the Department of Home Affairs notes that foreign interference activities are not only directed towards governments, but also academia, industries, the media and other communities (which is actually everyone).

Banning TikTok on government devices may eliminate one risk, but the broader pool of risks remains, both in government and beyond.

The government is currently developing a new cybersecurity strategy to replace the one put in place by the previous government just three years ago. A discussion paper on the new strategy was released earlier this year. This process will hopefully result in a more holistic strategy on how to manage the cybersecurity and foreign interference concerns that led to the TikTok ban.

Rather than the whack-a-mole tactical response of banning one app at a time, the strategy could provide clarity on how the government will manage the issue of weak security on mobile apps (particularly when used by people in sensitive sectors), as well as the potential for this to be an entry point for foreign interference.

This could include such things as:

• educating people on digital security and foreign interference
• streamlining the reporting channels for data breaches, foreign interference attempts, cybercrime, bugs and vulnerabilities
• developing or recommending the use of appropriate standards on cybersecurity, which could include references to international standards in areas such as information security and data governance
• strengthening cooperation between government and platforms and civil society
• introducing targeted prohibitions, which may include bans on apps that could share data with countries that might then use it for foreign interference.

This kind of strategic approach, particularly on the education side, would give Australians better tools to arm themselves against foreign interference online, which, as Home Affairs emphasises, is the ‘best defence’ available.

Another relevant policy development is the government’s review of the Privacy Act, which is the primary Australian law on data protection.

Changing the rules about how data is collected and used by platforms could provide less fodder for those running foreign interference operations. This could include banning unfair uses, such as targeted messaging based on a psychological profile. If the platforms don’t facilitate these uses, it becomes more difficult for foreign governments to use these tools for manipulation.

Enhancing funding for the primary data regulator, the Office of the Australian Information Commissioner, could also strengthen enforcement across the board.

These two reform initiatives exist within a maze of others, including inquiries or proposals relating to online privacy, digital platform services, the influence of international digital platforms, electronic surveillance and digital economy regulation.

Beyond Australia, at the United Nations level, some questions about whether international law can be applied to cyberspace have been resolved, while others remain open. Australia’s position on these issues could also be clarified.

Ultimately, what’s needed is a strategy, rather than tactics, and better coordination of relevant policies across government. The TikTok example also highlights a truism that we shouldn’t think in terms of privacy or security, but rather privacy and security.

While there’s an occasional need to choose between these two values (for example, when government agencies surveil those suspected of a crime, terrorism or espionage), in the vast majority of situations security is enhanced when the privacy of personal information is protected.

For example, the more personal information a foreign agent can access about citizens working in sensitive areas, the better it can target espionage and influence operations. If social media companies are restricted in how they collect, use and share Australians’ data, we can take significant steps towards protecting everyone from foreign interference and other harms.

We need all the policies and associated agencies (cyber, privacy, education, platform regulation, international relations, national security and more) working together if we are to meet the challenges. It may make sense to ban TikTok on government devices, but we need to address this problem more than one app at a time.

There’s one thing we’re all getting wrong about TikTok: it’s not really a social media app. As TikTok Australia’s general manager told the Senate Select Committee on Foreign Interference through Social Media in September last year, the app is ‘less about social connection and more about broadcasting creativity and expression’.

Put another way, think of TikTok more as the modern incarnation of a media publisher—like a newspaper or a TV network—than as a social forum like Facebook or Twitter. That’s because TikTok is much more assertively curatorial than its competitors. It’s not a forum, it’s an editor. Its algorithm decides what each user sees, and it’s the opacity of that algorithm that presents the most worrying national security risk.

It may sound like an insignificant distinction, but TikTok’s emphasis on an ‘interest graph’ instead of a ‘social graph’ took the app’s competitors completely by surprise, and has largely gone over the heads of most lawmakers. The app, owned by Chinese technology company ByteDance, hit 2.3 billion all-time downloads in August 2020, so it’s high time policymakers understood exactly what makes TikTok tick.

An essay by Eugene Wei should be at the top of their reading list. A San Francisco–based start-up investor and former Amazon and Facebook employee, Wei dissects TikTok’s strategy and shows how its recommendation engine keeps users glued to their screens. It does it not by connecting them with friends or family, but by closely analysing their behaviour on the app and serving them more of what they’re interested in.

Wei’s opus, which approaches 20,000 words and is only the first in a three-part series, explains how TikTok is not the same as the major social media platforms we’re more familiar with. Put simply, on Facebook and Twitter, the content that users see is largely decided by who they follow. On TikTok, however, the user doesn’t have to follow anyone. Instead, the algorithm very quickly learns from how users interact with the content they’re served in the app’s ‘For You’ feed to decide what it should deliver to them next.

The approach is similar to that of Spotify and Netflix, whose recommendation algorithms take note of which songs and movies you listen to or watch in full and which you skip to decide what new content to suggest. As Wei puts it, ‘TikTok’s algorithm is so effective that it doesn’t feel like work for viewers. Just by watching stuff and reacting, the app learns your tastes quickly. It feels like passive personalization.’

It’s a strategy, Wei argues, that allowed a team of Chinese engineers—who didn’t necessarily have a good understanding of the cultures in the places where the app is available—to take the world by storm.

TikTok didn’t just break out in America. It became unbelievably popular in India and in the Middle East, more countries whose cultures and language were foreign to the Chinese Bytedance product teams. Imagine an algorithm so clever it enables its builders to treat another market and culture as a complete black box. What do people in that country like? No, even better, what does each individual person in each of those foreign countries like? You don’t have to figure it out. The algorithm will handle that. The algorithm knows.

But that’s not the only thing the algorithm knows. In a recent Protocol China exposé, a former censor at ByteDance said the company’s ‘powerful algorithms not only can make precise predictions and recommend content to users—one of the things it’s best known for in the rest of the world—but can also assist content moderators with swift censorship’.

The former employee, who described working at ByteDance as like being ‘a tiny cog in a vast, evil machine’, said that even live-streamed shows on the company’s apps are ‘automatically transcribed into text, allowing algorithms to compare the notes with a long and constantly-updated list of sensitive words, dates and names, as well as Natural Language Processing models. Algorithms would then analyze whether the content was risky enough to require individual monitoring.’

There’s no doubt that TikTok and its parent company have these abilities to monitor and censor. The question is, will they continue to use it? Certainly, the blunt censorship that typified TikTok’s earlier approach to content moderation and is par for the course on ByteDance’s domestic apps is unlikely to continue, especially after the public scrutiny over TikTok’s censoring of content related to the Tiananmen Square massacre, Black Lives Matter protests and Beijing’s persecution of Uyghurs and other ethnic minorities.

But there’s ample room for ByteDance to covertly tweak users’ feeds, subtly nudging them towards content favoured by governments and ruling parties—including the Chinese Communist Party. After all, it’s an approach that would be in line with the strategy that China’s Ministry of Foreign Affairs and state media are already deploying.

Beijing is exploiting pre-existing grievance narratives and amplifying pro-CCP Western influencers in the knowledge that Western voices are more likely to penetrate target online networks than official CCP spokespeople. The strategy, referred to as ‘Borrowing mouths to speak’ (借嘴说话), is reminiscent of the Kremlin’s approach and is perfectly suited to being covertly deployed on Chinese-owned and -operated social media apps.

Just as experiments have shown that TikTok’s algorithm can hurtle users from a politically neutral feed into a far-right firehose of content, so too can it easily be used to send users down any extreme rabbit hole. By design, the app groups people into ‘clusters’ (otherwise known as filter bubbles) based on their preferences. TikTok’s executives stress that they have measures in place to ensure people don’t become trapped in those filter bubbles. TikTok’s recommendation system ‘works to intersperse diverse types of content along with those you already know you love’, the company claims. The goal, they say, is to ensure that users are exposed to ‘new perspectives and ideas’, but who decides which new perspectives and ideas?

What’s to stop Beijing from pressuring TikTok to encourage communities of Xinjiang denialists to flourish on the platform, for instance? As our report revealed, there’s already evidence that this is happening. Our analysis of the hashtag #Xinjiang showed a depiction of the region that glosses over the human-rights tragedy unfolding there and instead provides a more politically convenient version for the CCP, replete with smiling and dancing Uyghurs.

The power of social media apps has been underestimated before. When Facebook started as a ‘hot or not’ website in a Harvard dorm room at the turn of the millennium, who would have expected it would go on to play a role in inciting violence 13,000 kilometres away?

So how do policymakers deal with a Chinese-owned social media app that isn’t really a social media app but a modern-day interactive TV station, whose editorial decisions are made by an opaque algorithm developed and maintained in Beijing?

It’s past time governments realised the unique problem TikTok presents and they must now tailor solutions to deal with it properly.

That the Chinese ‘superapp’ WeChat is subject to political control by the Chinese Communist Party is no secret; many studies have tracked its powerful censorship regime over time and documented the content that is routinely taken down from the app.

In recent months, debate over the potential risk that WeChat poses to national security has again flared up in Western countries. In the US, that culminated in an unprecedented move by the Department of Commerce, which last week announced a ban on the app. However, over the weekend, just hours before it was to take effect, a US district court temporarily blocked the order as contrary to citizens’ right to free speech under the First Amendment.

The future of the app in the US and the rest of the world is still unclear, and several issues surrounding it remain unsolved, including China’s strict censorship regime.

In response to the US ban, the Chinese version of WeChat, called Weixin, censored the department’s statement for its users in the People’s Republic of China. The link still works for WeChat users residing overseas.

There’s strong evidence that users of the app outside of the PRC are subject to less strict rules than their mainland counterparts, and that therefore two different censorship systems apply to WeChat and Weixin. But it’s also becoming more and more apparent that not only censorship, but also surveillance and political interference are rampant on the international version of the app.

Earlier this month, WeChat and Chinese social media platform Weibo censored a post by the US embassy in Beijing that criticised Chinese state media and the Chinese state’s propaganda system.

On 9 September, the official CCP newspaper, the People’s Daily, refused to publish an opinion piece by US Ambassador to China Terry Branstad about the deteriorating relationship between the two countries.

In the article, the ambassador lamented an imbalance in US–China relations, noting that the standards set by the CCP impede equal exchanges between Chinese and American businesses, as well as undermine diplomatic relations and information flows.

The paper’s refusal to carry the piece caused tit-for-tat attacks between Chinese and US officials over press freedom and freedom of speech.

On the same day, the US embassy published an article on its website titled ‘The hypocritical propaganda system of the People’s Republic of China’. The post was later shared on the embassy’s official Weibo and WeChat accounts. Both posts were quickly taken down by the two platforms. The WeChat post was read over 100,000 times before being removed.

This, however, is only the latest case of the censorship of foreign diplomatic statements on WeChat.

In our recent report, TikTok and WeChat: Curating and controlling global information flows, I, together with other researchers at ASPI’s International Cyber Policy Centre, outlined several other, more subtle cases of information suppression on the app.

As we explain in our report, ‘In the same way that Chinese government departments, spokespeople, embassies and diplomats use Twitter and Facebook to promote messaging overseas, diplomatic missions in China use platforms like WeChat to promote messaging and publish official government statements.’ However, the latter are not free to promote debates or publish any form of criticism that the CCP deems unacceptable.

We found 11 posts by the US embassy in Beijing that were censored between April and August, as diplomatic clashes between China and several Western nations intensified. The articles all touched on topics sensitive to the CCP, such as China’s repression of ethnic minorities, the Hong Kong national security law, and China’s mishandling of the coronavirus pandemic. In contrast, we found no instances of censorship from June 2019 to April 2020.

The posts were subjected to different layers of censorship. Most of the summaries of the articles remained available on WeChat, but the links redirecting to the original articles were broken. In some cases the sharing function was disabled.

The US embassy is by far the most active among foreign diplomatic missions on WeChat and also the most regularly censored. However, the British and Indian embassies in Beijing also received the same treatment earlier this year.

In June, two posts related to the UK’s involvement in the Hong Kong issue were censored on WeChat. One was taken down and the other had its sharing function disabled. The two articles aimed to address accusations by the Chinese government about the UK’s stance on Hong Kong’s independence.

Similarly, as reported by Indian media, a post containing Prime Minister Narendra Modi’s speech on the India–China border standoff in the Ladakh region that the Indian embassy published on its official WeChat and Weibo accounts was removed from both platforms.

While in our report we look at TikTok and WeChat as two arms of the same system, it’s important to note that the power an app like WeChat has in shaping and controlling information flows both within and outside the PRC, especially in Chinese diaspora communities, is unparalleled and will continue to increase as the technology behind it continues to develop.

In the light of this, we recommend that governments better regulate the content moderation environment in their jurisdictions and require that all social media platforms diligently disclose all the content they censor, penalising those that undertake content moderation covertly.