TikTok’s Trials and Tribulations Continue With UK Data Protection Fine
TikTok has been fined £12.7 million (about $15.7 million) for breaches of UK data protection laws.
The UK’s data protection regulator (the ICO) has announced a fine of £12.7 million for “failing to use children’s personal data lawfully”. More specifically, the announcement states more than one million UK children under the age of 13 used the service in 2020; personal data of children was used without parental consent; and “TikTok ‘did not do enough’ to check who was using their platform and take sufficient action to remove the underage children that were.”
John Edwards, the UK Information Commissioner, commented, “There are laws in place to make sure our children are as safe in the digital world as they are in the physical world. TikTok did not abide by those laws… TikTok should have known better. TikTok should have done better. Our £12.7m fine reflects the serious impact their failures may have had.”
Nevertheless, this fine is a reduction from the ICO’s original notice of intent in September 2022 to fine TikTok £27 million. “Taking into consideration the representations from TikTok, the regulator decided not to pursue the provisional finding related to the unlawful use of special category data. That means this potential infringement was not included in the final amount of the fine set at £12.7 million,” says the announcement.
While this finding will please many privacy and children’s advocates, many others do not believe it is enough. “1.4 million UK kids opened a TikTok account and offered their personal data to be processed,” comments Tara Taubman-Bassirian, a privacy advocate and EU GDPR specialist. “Do we know how and with whom these were shared? What we know is TikTok might pay £12.7 million for it. How much TikTok made and will continue to make… we don’t know.”
This last comment is pertinent. “Why such a drastic reduction of the fine by half, with no injunction to delete the data unlawfully collected?” Does this mean, she asks, “infringement is OK if you pay the price?”
ChatGPT, the AI Revolution, and the Security, Privacy and Ethical Implications
This is the Age of artificial intelligence (AI). We think it is new, but it isn’t. The AI Revolution has been in progress for many years. What is new is the public appearance of the large scale generative pre-trained transformer (GPT) known as ChatGPT (an application of Large Language Models – LLMs).
ChatGPT has breached our absolute sensory threshold for AI. Before this point, the evolution of AI was progressing, but largely unnoticed. Now we are suddenly very aware, as if AI happened overnight. But it’s an ongoing evolution – and is one that we cannot stop. The genius is out of the bottle, and we have little understanding of where it will take us.
At a very basic level, these implications can be divided into areas such as social, business, political, economic and more. There are no clear boundaries between them. For example, social and business combine in areas such as the future of employment.
OpenAI, the developer of ChatGPT published its own research in this area: An Early Look at the Labor Market Impact Potential of Large Language Models. (PDF). It concludes, among other things, “around 19% of workers may see at least 50% of their tasks impacted.”
But we must be clear – these wider effects of AI on society and economics are not our concern here. We are limiting ourselves to discussing the cybersecurity, privacy and ethical implications emerging from the GPT and LLM elements of AI.
ChatGPT-3 (more accurately GPT 3.5) was made available for public use in November 2022. It could generate a meaningful response to most questions, based on the vast amounts of data pre-ingested. People immediately tested it – at first for fun, but then to see if it could be used for non-intended purposes (mostly malicious). WithSecure produced a study on the use of ChatGPT to improve phishing and social engineering, using prompt engineering on the system’s input.
Check Point found evidence that malicious actors were using it to check and improve the code in their malware. But it is important to note that ChatGPT is just one of many new AI systems becoming available, and they are almost always originally intended for legitimate and beneficial purposes. On March 23, Oxford, UK-based Diffblue announced the granting of three US patents for its own generative AI product, Diffblue Cover. Diffblue Cover uses AI to generate software unit tests to improve code.
Misuse of ChatGPT – and potentially other GPTs – is possible because researchers rapidly learned that it was relatively easy to subvert the safety guardrails put in place to prevent misuse – a process unsurprisingly known as jailbreaking.
GPT-4 was announced on March 14, 2023, on Twitter, by. OpenAI CEO Sam Altman: “here is GPT-4, our most capable and aligned model yet. it is available today in our API (with a waitlist) and in ChatGPT+… it is still flawed, still limited, and it still seems more impressive on first use than it does after you spend more time with it.”
The official announcement says, “GPT-4 is a large multimodal model (accepting image and text inputs, emitting text outputs) that, while less capable than humans in many real-world scenarios, exhibits human-level performance on various professional and academic benchmarks.”
It’s time to take stock, evaluate the expansion of AI technologies, and project where the AI ride is taking us.
“There are three major differences between GPT3 and GPT4,” says Alex Polyakov, co-founder and CEO at Adversa.ai: “longer memory, support for images, and potentially better safety and security. However, the last one is not as amazing as it sounds, as we still see that prompt injection attacks are possible, jailbreaks still exist, and can even be done in multiple new ways – and it can’t keep secrets.”
The multi-modal image intake is interesting, although still in the early stages of development. “Although it cannot output pictures (as do generative AI models such as DALL-E and Stable Diffusion), it can process and respond to the visual inputs it receives,” wrote Scientific American on March 16. It can identify basic context in images. The magazine points out that future mobile apps could interpret surroundings for visually impaired users. It didn’t mention it could similarly be used in invasive surveillance looking for ‘suspicious behavior’. This is the big conundrum for AI: things designed for good can also be used for bad.
Interaction with GPT is measured by and limited to a specified number of ‘tokens’ (cost per token is also the charging method when charges are applied). The number of permissible tokens per transaction has increased in the publicly available version to 8k tokens – and to 32k tokens in an availability-limited version. A token is approximately four characters in English. The maximum token length available applies to both the input prompt and the output response combined: the longer the question, the shorter the possible answer, and vice versa.
So, 8k tokens would be roughly equivalent to 6,000 words, while 32k tokens would be four-times that amount. This vastly improves the ability to accept more complex inputs and produce longer and more detailed outputs. This, in turn, increases the potential for both use and abuse of the system.
Chester Wisniewski, field CTO of applied research at Sophos, says similar to Polyakov. “Quite a lot has changed and improved, but not all is known. At a high level it seems to be smarter, more accurate and more capable of what we might think of as ‘thinking’ than previous versions, all of which should make its responses even more realistic and capable than before. It’s important to remember that while it will lie to you less frequently, it is still a pretty good liar.”
In an interview with ABC News on March 16, 2023, Altman was asked, “What is the worst possible outcome?” He replied, “There’s a set of very bad outcomes. One thing I’m particularly worried about is that these models could be used for large scale disinformation. I am worried that these systems, now that they’re getting better at writing computer code, could be used for offensive cyberattacks.”
Disinformation comes from the ability to generate compelling but false narratives. On ChatGPT-3, WithSecure demonstrated the ability to generate convincing phishing emails to help spread such disinformation. Accurate code generation is inevitable – bad guys are already using it to debug and improve malware and change the code sufficient to defeat signature detection. In the future, it will be able to generate new malware.
These possibilities – indeed, probabilities – have already been discovered and discussed by researchers. Jailbreaking is the key – tricking the system into ignoring or bypassing the safety guardrails implemented by the developer. It turns out that this is almost as easy with GPT4 as it was with GPT3. Hallucination is a further danger; that is, the output of data as if it is correct when it is false.
For AI, security is a two-way street: It can be used by malicious actors to abuse victims, while its own security can be abused by those same malicious actors. ChatGPT has already suffered at least one breach that is known. During the week beginning March 20, 2023, OpenAI took the system offline to fix a flaw that led to the exposure of user information. It was later found that OpenAI’s original fix was itself flawed, and that ChatGPT APIs were vulnerable to a bypass.
Chester Wisniewski, Field CTO Applied Research at Sophos.
The use of AI to abuse others is theoretically prevented by internal guardrails designed to prevent misuse. These guardrails have so far been found inadequate. The question then becomes whether they will ever be adequate.
Wisniewski thinks not. “Any system to prevent abuse, but seemingly designed to let the system operate autonomously, will most likely always be able to be bypassed.” He hopes it will become harder as OpenAI and other AI researchers learn more about how jailbreaking is performed, “but it will still be possible.”
John Bambenek, principal threat hunter at Netenrich, also has doubts. “The fundamental cybersecurity problem is how to perform automation on untrusted inputs – and we are nowhere on solving that.”
“I doubt it is possible to create a GPT model that can’t be abused,” adds Mike Parkin, senior technical engineer at Vulcan Cyber. “The challenge long term will be keeping threat actors from abusing the commercially available AI engines. Ultimately though, it will be impossible to keep them from creating their own and using them for whatever purposes they decide.”
Stephanie Aceves, Sr. Director, Product Management at Tanium
But there are some hopeful thoughts. Stephanie Aceves, senior director of product management at Tanium, accepts the task is like achieving cybersecurity by turning the computer off and locking it in a vault – but the task should not preempt the attempt. “Risk should not be a showstopper,” she said. “Rather it should be an input to the policies, programs, and guardrails we develop.”
Polyakov doesn’t believe that much can be done to prevent the misuse of AI without a legal framework. But he believes that AI developers can do more to protect their own security. “It will be hard work and a continuous cat and mouse game, but it’s certainly possible to make AI much more secure.”
He further believes that making the systems more secure will have the byproduct of making them more difficult to misuse. “What is more important and amazing is that in making AI models more secure, you may make them more robust and accurate as a byproduct.”
He warns that this should be done sooner rather than later. “The earlier companies start initiatives, the better they will protect their systems and have a competitive advantage. Sometimes the goal is not to be 100% secure but to be more secure than your neighbor.” Criminals tend to attack the easiest target.
Andy Patel, senior researcher at WithSecure
But it may already be too late. “It will never be possible to create a large language model that cannot be abused,” suggests Andy Patel, senior researcher at WithSecure. “Prompt injection aside, many underhanded uses of NLG [natural language generator] models rely on generating text designed to persuade or trick people. Prompts designed to create this type of content, by nature, don’t trigger safety filters or the model’s refusal policy.”
He points to developments outside of OpenAI. “Recent ML advances have precipitated models such as Alpaca, a 6B parameter model that works approximately as well as GPT-3, and that can be run on a laptop. Those models are good enough to generate content that could be used for malicious purposes. I would expect bad guys to be more interested in those models than GPT-4 – at least in the near future. Eventually GPT-4-strength models will be available to all in the same way.”
Privacy is one of the areas considered most at risk from an unfettered use of AI, and an ethical or unethical implementation of the technology will drive the extent of privacy abuses. SecurityWeek spoke to Christina Montgomery, chief privacy & trust officer, and AI ethics board chair at IBM. “The technology is clearly moving faster than society’s ability to build reasonable guardrails around it, and there’s still not enough transparency around how other tech companies are protecting the privacy of data that interacts with their systems,” she said.
Christina Montgomery, chief privacy & trust officer, and AI ethics board chair at IBM
The solution must come from both government and industry: government in strong regulations, and industry in a clear ethical use of AI. “There is a real need for our government leaders to work with the private sector on effective, risk-based AI regulation – where the tightest regulatory control focuses on the AI applications with the greatest risk of societal harm.” But while AI must be regulated, privacy must also be protected. “We need a consistent, national privacy law in this country,” she added.
She acknowledges that ChatGPT has guardrails in place, but with clear shortcomings. “That makes proper oversight even more important, especially in a consumer context.” At the enterprise level, she believes the solution must lie in greater use of the principles of ethical use. “We’re at an early stage in public exposure to AI – and I firmly believe that every company involved in this work has an obligation to strengthen trust in the technology.”
She notes that IBM has established ethical principles to the development and use of AI, including questioning not whether something can be done, but whether it should be done. “Our focus is on developing technologies, including generative AI tools, with responsibility and ethics at the forefront and then urging other private sector developers to do the same.”
Finally, she adds, “People need to see more companies leading by example, putting ethics, responsibility, and people’s interests first. ChatGPT is a reminder that these technologies are getting more powerful and that the era of move fast and break things must end.”
Absent regulation, however, privacy abuses will likely continue. “The reason big tech companies collect so much data is to have the training data to create tools like GPT4 in the first place,” comments Bambenek. “We are only scratching the surface of the risks this poses, mostly because the primary use case for these companies is advertising. As long as people are willing to tolerate the privacy invasions for cheap/free service, there isn’t much that will slow this down.”
Aceves is more optimistic. “The short answer is yes, something can be done,” she says. “Organizations like The Cyber Collective are leading the way in educating the average person and initiating change in our current policies.”
Steve Wilson, CPO at Contrast Security, tasked ChatGPT to answer some of the questions SecurityWeek was asking, as if it were an ethics professor. “While it might not create entirely new ideas, it can produce novel combinations of existing knowledge and concepts,” responded ChatGPT about ChatGPT.
But that ‘existing knowledge may be false and incomplete. “AI models like ChatGPT can indeed learn and propagate inaccuracies or biases present in the training data,” continued the fake professor. The old adage of ‘garbage in, garbage out’ still applies to the new technologies.
“To address this issue, AI developers must continuously improve the training process by curating diverse, high-quality datasets and incorporating methods to mitigate bias.” But this is the conundrum underlying all ethical attempts to remove bias — it is led by people with existing, perhaps unconscious, biases of their own.
On March 31, 2023, the Italian data protection regulator blocked ChatGPT over concern that it is unlawfully processing personal data protected by GDPR. OpenAI has 20 days to respond, but the potential to hallucinate (give out false information) adds a further complication if it spreads false information about European residents. Then comes the issue of the European ‘right to be forgotten’. How does AI forget?
A further complication comes through other software that uses OpenAI’s GPT. Microsoft has invested billions in OpenAI, and is clearly very close to the company. ChatGPT has been incorporated into Bing and can be used via Skype.
If it is found that these internet searches gather PII on GDPR-protected citizens from untrusted internet websites, will Italy block Microsoft?
The Future of Life Institute published an open letter on March 29, 2023: “We call on all AI labs to immediately pause for at least 6 months the training of AI systems more powerful than GPT-4.” The letter cites the Asilomar AI Principles, a recognized list of AI governance principles – #20 of which states, “Advanced AI could represent a profound change in the history of life on earth, and should be planned for and managed with commensurate care and resources.”
Response from within the security industry to the letter has been varied. Dan Shiebler, head of machine learning at Abnormal Security, commented, “Personally, I don’t think this letter will achieve much. The cat is out of the bag on these models. The limiting factor in generating them is money and time, and both will fall rapidly. We need to prepare businesses to use these models safely and securely, not try to stop the clock on their development.”
Chenxi Wang, founder and general partner at Rain Capital, is in favor of a pause. “A pause in the AI fever is needed, not just from the business standpoint, but also from the point of view of security and privacy. Until we understand how to assess data privacy, model integrity, and the impact of adversarial data, continued development of AI may lead to unintended social, technical, and cyber consequences.”
But in the final analysis, we will not be able to halt the continued evolution of AI. Two of humanity’s greatest drivers, greed and curiosity, will push AI development forward. Our only hope is that we can control it. The genie is out of the bottle, and isn’t offering any wishes.
TikTok Attorney: China Can’t Get U.S. Data Under Plan
Under intense scrutiny from Washington that could lead to a potential ban, the top attorney for TikTok and its Chinese parent company ByteDance defended the social media platform’s plan to safeguard U.S. user data from China.
“The basic approach that we’re following is to make it physically impossible for any government, including the Chinese government, to get access to U.S. user data,” said general counsel Erich Andersen during a wide-ranging interview with The Associated Press at a cybersecurity conference in Sausalito, California, on Friday sponsored by the Hewlett Foundation and Aspen Digital and featuring top government officials, tech executives and journalists.
ByteDance will continue to develop its new app called Lemon8, Andersen said.
“We’re obviously going to do our best with the Lemon8 app to comply with U.S. law and to make sure we do the right thing here,” Andersen said, referring to the new social app developed by ByteDance that resembles Instagram and Pinterest. “But I think we got a long way to go with that application — it’s pretty much a startup phase.”
ByteDance’s most known app, TikTok, is under intense scrutiny over concerns it could hand over user data to the Chinese government or push pro-Beijing propaganda and misinformation on its behalf. Lemon8 was introduced across app stores in Japan in April 2020 and has been rolled out in more countries since then. It’s available for download in the U.S. and could face similar scrutiny to TikTok.
Leaders at the FBI, CIA and officials at other government agencies have warned that ByteDance could be forced to give user data — such as browsing history, IP addresses and biometric identifiers — to Beijing under a 2017 law that compels companies to cooperate with the government for matters involving China’s national security. Another Chinese law, implemented in 2014, has similar mandates.
To assuage concerns from U.S. officials, TikTok has been emphasizing a $1.5 billion proposal, called Project Texas, to store all U.S. user data on servers owned and maintained by the software giant Oracle. Under the plan, access to U.S. data would be managed by U.S. employees through a separate entity called TikTok U.S. Data Security, which is run independently of ByteDance and monitored by outside observers.
Some lawmakers have said that’s not enough. But despite skepticism about the project, TikTok says it is moving forward anyway.
“We’re investing in a system where people don’t have to believe the Chinese government and they don’t have to believe us,” Andersen said.
He also wondered if the skepticism was being driven by something else.
“Where are we falling short here?” he said. “At some point you get beyond the cybersecurity risk assessment, etcetera, and you get to ‘We don’t like your nationality.’”
TikTok CEO Shou Zi Chew has said the company started deleting all historic U.S. user data from non-Oracle servers this month and expects that process to be completed this year. During a congressional hearing held last week, Chew said migrating the data to Oracle will keep it out of China’s hands, but also acknowledged China-based employees may still have access to it before the process wraps up.
TikTok maintains it has never been requested to turn over any kind of data and won’t do so if asked. But whether those promises, or Project Texas, will allow it to stay operating in the U.S. remains to be seen.
The U.S., as well as Britain, the European Union and others, have banned TikTok on government devices. And the Biden administration is reportedly threatening a U.S. ban on the app unless its Chinese owners divest their stakes in the company.
On Friday, Andersen said a ban would be “basically giving up”.
“Banning a platform like TikTok is a defeat, it’s a statement that we aren’t creative enough to find another way,” he said.
China has said it would oppose a possible sale, a declaration that makes it more difficult for TikTok to position itself and ByteDance as a global enterprise instead of a Chinese company. In 2020, the country had also come out in fierce opposition to executive orders by then President Donald Trump that sought to ban TikTok and the messaging app WeChat.
“They were clear about their point of view back in 2020 timeframe when we faced an existential challenge from executive orders under the Trump administration,” Andersen said.
Courts blocked Trump’s efforts, and President Joe Biden rescinded Trump’s orders after taking office. The company has since been in talks about privacy concerns with the Committee on Foreign Investment in the United States, a multi-agency panel that sits under the Treasury department.
Meanwhile, lawmakers on Capitol Hill have been pushing bills that would effectively ban TikTok or give the administration more authority to do so. One bill by U.S. Sen. Josh Hawley was blocked this week by Sen. Rand Paul, the only Republican who has come out in opposition to a TikTok ban. A small number of progressive lawmakers have also said they would oppose a ban, and argued the U.S. should implement a national privacy law to curtail the problem.
Andersen said Friday TikTok would support broad-based privacy legislation.
“Our view is that we would really welcome broad-based legislation that applies broadly and evenly,” he said. “What we don’t like, frankly, is legislation that is sort of targeted at one company.”
TikTok could also be banned through another bill, called the RESTRICT Act, that has garnered broad bipartisan support in the Senate and backing from the White House. The legislation does not call out TikTok but would give the Commerce Department power to review and potentially restrict foreign threats to technology platforms.
Italy Temporarily Blocks ChatGPT Over Privacy Concerns
Italy is temporarily blocking the artificial intelligence software ChatGPT in the wake of a data breach as it investigates a possible violation of stringent European Union data protection rules, the government’s privacy watchdog said Friday.
The Italian Data Protection Authority said it was taking provisional action “until ChatGPT respects privacy,” including temporarily limiting the company from processing Italian users’ data.
U.S.-based OpenAI, which developed the chatbot, said late Friday night it has disabled ChatGPT for Italian users at the government’s request. The company said it believes its practices comply with European privacy laws and hopes to make ChatGPT available again soon.
While some public schools and universities around the world have blocked ChatGPT from their local networks over student plagiarism concerns, Italy’s action is “the first nation-scale restriction of a mainstream AI platform by a democracy,” said Alp Toker, director of the advocacy group NetBlocks, which monitors internet access worldwide.
The restriction affects the web version of ChatGPT, popularly used as a writing assistant, but is unlikely to affect software applications from companies that already have licenses with OpenAI to use the same technology driving the chatbot, such as Microsoft’s Bing search engine.
The AI systems that power such chatbots, known as large language models, are able to mimic human writing styles based on the huge trove of digital books and online writings they have ingested.
The Italian watchdog said OpenAI must report within 20 days what measures it has taken to ensure the privacy of users’ data or face a fine of up to either 20 million euros (nearly $22 million) or 4% of annual global revenue.
The agency’s statement cites the EU’s General Data Protection Regulation and pointed to a recent data breach involving ChatGPT “users’ conversations” and information about subscriber payments.
OpenAI earlier announced that it had to take ChatGPT offline on March 20 to fix a bug that allowed some people to see the titles, or subject lines, of other users’ chat history.
“Our investigation has also found that 1.2% of ChatGPT Plus users might have had personal data revealed to another user,” the company had said. “We believe the number of users whose data was actually revealed to someone else is extremely low and we have contacted those who might be impacted.”
Italy’s privacy watchdog, known as the Garante, also questioned whether OpenAI had legal justification for its “massive collection and processing of personal data” used to train the platform’s algorithms. And it said ChatGPT can sometimes generate — and store — false information about individuals.
Finally, it noted there’s no system to verify users’ ages, exposing children to responses “absolutely inappropriate to their age and awareness.”
OpenAI said in response that it works “to reduce personal data in training our AI systems like ChatGPT because we want our AI to learn about the world, not about private individuals.”
“We also believe that AI regulation is necessary — so we look forward to working closely with the Garante and educating them on how our systems are built and used,” the company said.
The Italian watchdog’s move comes as concerns grow about the artificial intelligence boom. A group of scientists and tech industry leaders published a letter Wednesday calling for companies such as OpenAI to pause the development of more powerful AI models until the fall to give time for society to weigh the risks.
The president of Italy’s privacy watchdog agency told Italian state TV Friday evening he was one of those who signed the appeal. Pasquale Stanzione said he did so because “it’s not clear what aims are being pursued” ultimately by those developing AI.
If AI should “impinge” on a person’s “self-determination” then “this is very dangerous,″ Stanzione said. He also described the absence of filters for users younger than 13 as ”rather grave.”
San Francisco-based OpenAI’s CEO, Sam Altman, announced this week that he’s embarking on a six-continent trip in May to talk about the technology with users and developers. That includes a stop planned for Brussels, where European Union lawmakers have been negotiating sweeping new rules to limit high-risk AI tools, as well as visits to Madrid, Munich, London and Paris.
European consumer group BEUC called Thursday for EU authorities and the bloc’s 27 member nations to investigate ChatGPT and similar AI chatbots. BEUC said it could be years before the EU’s AI legislation takes effect, so authorities need to act faster to protect consumers from possible risks.
“In only a few months, we have seen a massive take-up of ChatGPT, and this is only the beginning,” Deputy Director General Ursula Pachl said.
Waiting for the EU’s AI Act “is not good enough as there are serious concerns growing about how ChatGPT and similar chatbots might deceive and manipulate people.”
UK Introduces Mass Surveillance With Online Safety Bill
The Online Safety Bill is the enactment of two long held UK government desires: the removal of harmful internet content, and visibility into end-to-end (E2E) content. The latter is just a byproduct of enforcing the former. Both are justified on national security (terrorism) and protection of children requirements (child pornography).
At the time of writing, this bill (PDF) has passed through the House of Commons, and is currently at committee stage in the House of Lords. It is likely (not certain) that it will become law. While this would be a UK law, its reach expands to any internet platform providing services to people in the UK.
The primary gist of the bill is that platform providers are responsible for the content available on their platforms, irrespective of who generates the content. If content is deemed harmful (child pornography, terrorist recruitment, revenge porn, bullying, self-harm, and anything the government defines as ‘illegal’), the provider can be required to remove that content.
All of this sounds reasonable; but the problems start with visibility and enforcement. Enforcement is to be undertaken by the government’s own Office of Communications regulator, Ofcom. To be able to determine compliance with the law, Ofcom must have visibility on the content. That, in simple terms, implies mass government surveillance of any internet available to users within the UK.
But what if the information on the platform is protected with end-to-end encryption within a messaging or communications application? That doesn’t matter; it is still subject to the law, and Ofcom must be provided access to the cleartext content. In short, the Online Safety Bill will require messaging app providers to implement some form of backdoor into the encrypted data – although the government asserts this isn’t a ban on E2E encryption itself.
Ofcom’s weapons include fines up to £18 million ($22 million) or 10% of global revenue (GDPR’s maximum is 4% of global revenue), blocking the platform, and even criminal liability for senior managers.
The problem is that end-to-end encryption and backdoors are mutually exclusive. The UK government insists that this needn’t be so, but it is a technological reality. In his Primer for E2E Encryption Policy in 2022, Alec Muffett (formerly a security policy advisor to the Open Rights Group) wrote: “You can’t be ‘a little bit surveilled’. Either a non-participant has independently determined a message which Alice has spoken to the other participants, or else they have not. If such has occurred, then surveillance has occurred and the guarantee of E2E has been broken.”
The anomaly is that if the government can access the content, criminals and foreign governments will almost certainly be able to use the same backdoor.
There are three questions to consider. Is the law legal? How will the messaging app providers respond? Can users do anything to ensure private communication despite the new law?
A legal law
The first page of the bill states (referring to the European Convention on Human Rights – ECHR): “Lord Parkinson of Whitley Bay has made the following statement under section 19(1)(a) of the Human Rights Act 1998: ‘In my view the provisions of the Online Safety Bill are compatible with the Convention rights’.” The bill is sponsored by the Department for Digital, Culture, Media & Sport. Parkinson is Parliamentary Under Secretary of State for the Department for Digital, Culture, Media & Sport. Fox and hen house?
SecurityWeek asked Monica Horten, policy manager, freedom of expression at the Open Rights Group, is this bill compatible with ECHR. “No,” she replied. “It lacks procedural safeguards for users whose content is restricted, and the potential for disproportionate surveillance of private chats, are two reasons why this would be the case.” We asked, could the bill be challenged in the courts. Yes, she said. “The over-broad text, and the minimal definition leaves it open challenge.”
This law will already affect US firms. The real danger is its arguments may spread like a contagion to be used by other governments.
Response from messaging app companies:
WhatsApp has stated very clearly that it will not provide a backdoor for Ofcom, and accepts that it may be blocked in the UK (as has already happened in Iran). It worries about the message being sent by ‘a liberal democracy’ to more authoritarian regimes.
Signal president Meredith Whittaker told the BBC the organization “would absolutely, 100% walk” rather than abide by government surveillance requirements.
Tutanota takes a slightly different stance. In a blog posted on February 28, 2023, the firm states, “‘Walking out’ is not the solution here. We at Tutanota say the opposite: We will not ‘walk’ from the UK. If Prime Minister Rishi Sunak and his government want to stop people in the UK to use strong encryption – like that provided by our secure email service Tutanota – he must block access to Tutanota – just like Russia and Iran are already doing.”
The effect is that if an ‘E2E-encrypted’ product is available in the UK after the bill becomes law, it should no longer be considered to be truly safe from prying eyes.
Circumvention
SecurityWeek talked to Jeff Williams, CTO and co-founder at Contrast Security. “This is about the third time we’ve been through this exact conflict,” he said. The first was the epic battle over the Clipper Chip in 1993, containing a backdoor to enable law enforcement to access encrypted communications. There have been other similar battles in the intervening 30 years that have all ended the same way — the failure to enable government access to keys (GAK) was one attempt.
Jeff Williams, CTO and co-founder of Contrast Security
He accepts the dichotomy between wishing to keep children safe from online harm while simultaneously wishing to keep personal communications private and secure. “The problem,” he says, “is these two goals are impossible to reconcile.” Any backdoor will weaken encryption. “And it’s all but certain that unwanted individuals will use that backdoor to undermine everyone’s security and privacy. What’s worse is these backdoors can be easily circumvented by bad guys’, privacy loving good guys, and anyone else who wants to communicate securely over this insecure channel.”
An easy option, he suggests, is superencryption. “Essentially, you use your own non-backdoored encryption first, and then send it through the system using the backdoored encryption. Government will only have access to what’s been encrypted with your own non-backdoored encryption. This is easily possible for communications formats as well as files.”
He has little time for this new battle for law enforcement access to encrypted messages. “The Clipper Chip was dead in three years. This Bill won’t last nearly that long. At this point, strong cryptography is well understood and public. The idea that governments can prevent secure communications between the denizens of the world is long dead. Apparently, there are some in the government that didn’t read the obituary.”
The most effective circumvention will come when liberal governments understand you cannot beat cybercriminality by becoming a legal cybercriminal. They conflate two wishes: greater control over the internet, and the protection of children. They use the latter to justify the former, but the two wishes are not compatible. The first is a technology issue, while the second is a social issue.
In a paper (PDF) published last year, Ross Anderson, professor of security engineering at Cambridge university first debunks many of the statistics and arguments used to justify the need for the Online Safety Bill, and then concludes: “The idea that complex social problems are amenable to cheap technical solutions is the siren song of the software salesman and has lured many a gullible government department on to the rocks. Where ministers buy the idea of a magical software ‘solution’, as the industry likes to call its products, the outcomes are often disappointing and sometimes disastrous.”
Marcus Ranum put it more succinctly in Ranum’s Law: “You can’t solve social problems with software.”
Facebook ‘Unlawfully’ Used Dutch Personal Data: Court
Social media platform Facebook unlawfully processed Dutch users’ personal details without consent for advertising purposes for almost a decade, Amsterdam-based judges ruled on Wednesday.
The judgement by the Amsterdam District Court said Facebook Ireland — custodians of Dutch users’ personal details — not only used the data for advertising, but also passed it to third parties without properly informing people or having legal grounds to do so.
“Facebook Ireland has broken the law when processing personal data of Dutch Facebook users in the period from April 1, 2010 to January 1, 2020,” the judges said in a statement issued by the court.
It “processed personal data without a legal basis — such as consent — for this,” the judges said.
Judges however turned down a third claim that the use of online “cookies”, digital trackers used to target advertising, was unlawful on third party sites, as the responsibility to inform users was transferred to the relevant website operators by Facebook Ireland.
The ruling comes after a 2019 class action lawsuit brought by the Dutch-based Data Privacy Foundation, an internet watchdog which says it fights for the rights of online users against companies “exploiting our personal details”, backed by the country’s Consumers Association.
The judges did not rule on claims in the case, but the Data Privacy Foundation’s chairman Dick Bouma said it opened the door for consumers to receive “compensation for years of privacy violations by Facebook.”
“It’s now up to Facebook to provide this (compensation). We would like to discuss this with them,” he told the NU.nl Dutch news website.
Facebook owner Meta in a statement to AFP said it was “pleased the court ruled in favour of Meta for multiple of these historic claims, some of which took place over a decade ago.”
“We intend to appeal other aspects of this case,” the statement said.
The Dutch court’s finding followed a mega fine against Meta in early January, totalling 390 million ($410 million) euros, for breaching EU personal data laws on Facebook and Instagram.
Diritto all’oblio, Italiani fra i primi in Europa per richieste di rimozione dai motori di ricerca
Dal 2015 al 2021, oltre 1 milione di richieste sono state inviate a Google e Microsoft Bing, la metà delle quali proveniva dai paesi dell’Europa occidentale. L’Italia è al 5° posto con 89,1 mila richieste totali. E’ quanto emerge da un report realizzato da Surfshark, che analizza le richieste di “diritto all’oblio”. Un dato sutti: da quando è cominciata la pandemia di Covid-19, i casi di “diritto all’oblio” sono aumentati di quasi il 30% nel 2020. Complessivamente, le richieste francesi hanno rappresentato quasi un quarto di tutte le richieste presentate, mentre l’Estonia ha registrato il maggior numero di richieste per 10mila persone. I paesi dell’Europa orientale hanno esercitato meno del loro “diritto alla cancellazione”.
Cos’è il diritto all’oblio
Il “diritto all’oblio” consente alle persone di chiedere alle pagine Web di eliminare le query relative al proprio nome dalle pagine dei risultati dei motori di ricerca europei. Si applica ai paesi coperti dal GDPR (UE e SEE) e ad altri paesi europei che hanno adottato leggi simili, come il Regno Unito e la Svizzera. Le informazioni sui richiedenti all’interno di queste pagine web variano ampiamente, dalle informazioni personali e professionali ai collegamenti con attività criminali.
<!-- ARUBA - 300x250 -->
I numeri
Dal 2015 al 2021 sono state presentate a Google e Bing 1.066.274 richieste di “diritto all’oblio” o “diritto alla cancellazione”. Dei due motori di ricerca, la stragrande maggioranza (95,8%) delle richieste è stata consegnata a Google. Il 2015 ha segnato il primo anno intero in cui la politica è stata in vigore, durante la quale sono state presentate 169.200 richieste durante tutto l’anno.
Dopo il 2015, le richieste di “diritto all’oblio” sono gradualmente diminuite fino al 2020. I casi sono aumentati di quasi il 30% nel 2020, per un totale di 161,3 mila. Ad esempio, Cipro e Portogallo hanno presentato nel 2020 quasi il triplo di richieste rispetto all’anno precedente. Anche l’Italia ha registrato un aumento del 13% nel 2020. Il 2021 ha visto un ulteriore aumento del 15% su tutta la linea, con un massimo storico di 185.700 richieste nei paesi analizzati.
Picco di richieste durante il lockdown
Il picco di richieste di cancellazione nel 2020 è legato sicuramente al maggior utilizzo del digitale a causa delle restrizioni e del lockdown, che hanno chiuso per mesi in casa le persone, spinte così a diventare più consapevoli della loro dimensione virtuale e quindi più sensibili alla loro igiene digitale in nome della privacy online. Nel contempo il GDPR accelerava e continua a farlo potenziando la consapevolezza che la data protection è un diritto fondamentale.
La Francia ha presentato il maggior numero di richieste per il “diritto all’oblio” tra il 2015 e il 2021, con 255.600 richieste che rappresentano quasi un quarto di tutte le richieste presentate in questo periodo. 176.100 richieste sono arrivate dalla Germania e 125.300 dal Regno Unito, con ogni paese che rappresenta rispettivamente circa il 17% e il 12% del totale. Le richieste presentate da questi tre paesi rappresentano circa la metà di tutte le richieste di “diritto all’oblio” presentate tra il 2015 e il 2020. L’Italia è al 5° posto (su 32) con circa 89.100 richieste totali.
L’Estonia è in testa per densità di richieste relative al “diritto all’oblio”
Per quanto riguarda la densità delle richieste (ovvero richieste per 10.000 persone), l’Estonia è al primo posto con 53 richieste per 10.000 persone (più di 2,5 volte superiore alla media). La Francia è al 2° posto con 40, e l’Olanda al 3° con 32 richieste per 10.000 persone. L’Italia è 24esima con 15 richieste ogni 10.000 individui. La Bulgaria ha presentato negli anni il minor numero di richieste per 10mila persone, con circa 6, oltre tre volte inferiori alla media di tutti i Paesi presi in esame.
L’Europa occidentale e settentrionale hanno la più alta densità di richieste, rispettivamente con 28 e 21 richieste per 10.000 persone. L’Europa meridionale ha emesso 16 richieste e l’Europa orientale ha emesso otto richieste per 10.000 persone.
1 richiesta su 10 di rimozione per “diritto all’oblio” è legata a gesti criminali
4.009.729 pagine web sono state incluse nelle richieste inviate a Google nel periodo 2015-2021. Per Google, i richiedenti europei hanno chiesto di rimuovere in media quattro pagine Web per richiesta dai suoi risultati di ricerca e gli italiani hanno richiesto lo stesso numero. In generale, circa la metà delle pagine di cui è stata richiesta la rimozione da Google non poteva essere classificata in una categoria specifica e quasi il 17% rientrava nelle informazioni professionali (contenevano l’indirizzo di lavoro del richiedente, informazioni di contatto o informazioni generali sulle sue attività commerciali).
Circa 1 pagina web su 10 fa riferimento al richiedente in merito a reati, compresi quelli nel proprio campo professionale. Circa il 6% conteneva le informazioni personali del richiedente, come l’indirizzo di casa, la residenza o altre informazioni di contatto, nonché immagini e video dell’individuo.
Circolazione dei dati personali tra contratto e responsabilità. Il nuovo libro di Emilio Tosi
Circolazione dei dati personali tra contratto e responsabilità Riflessioni sulla fragilità del consenso e sulla patrimonializzazione dei dati personali nella società della sorveglianza digitale Il nuovo libro di Emilio TosiProfessore Associato Diritto Privato Università di Milano Bicocca e Direttore esecutivo Centro Studi DIRITTO NUOVE TECNOLOGIE. Giuffrè Editore
Il presente studio monografico offre un contributo allo studio dei profili contrattuali e delle responsabilità civili correlate al fenomeno emergente dello sfruttamento commerciale dei dati personali alla base dei big data del capitalismo della società della sorveglianza digitale.
In particolare vengono indagati, da un lato la doppia natura del dato personale tra diritto della personalità e nuovo bene immateriale e dall’altra la doppia natura del consenso nel General Data Protection Regulation tra natura autorizzatoria (art. 6.1 lett.a) e natura contrattuale (art. 6.1 lett.b).
Inoltre si considerano le implicazioni della recente novella al Codice del Consumo operata dal D.Lgs. 173/2021 – che ha recepito in Italia la Direttiva 770/2019 – introducendo il nuovo art.135 octies disciplinante l’utilizzo dei dati personali quale controprestazione nei contratti di scambio di beni e servizi digitali.
Ulteriore conferma normativa dell’equivoco interpretativo sorto attorno alla qualificazione del consenso causa del ripudio, non condivisibile, del riconoscimento dell’ammissibilità della dimensione contrattuale in relazione allo sfruttamento economico dei dati personali dell’interessato.
Dati personali e contratto non sono antitetici: è proprio il GDPR ad ammettere la coesistenza bilanciata tra diritti fondamentali alla protezione dei dati e libertà di circolazione degli stessi.
Emerge, dunque, la meritevolezza di tutela e ammissibilità del contratto di trattamento dati quale contratto regolamentato conformato ai precetti inderogabili del GDPR.
L’analisi dell’utilizzo dei dati personali altrui da parte di società che fondano l’attività d’impresa proprio sul trattamento dei dati è completata dallo studio del rimedio risarcitorio ai sensi dell’art. 82 del GDPR e dei profili sanzionatori.
Si registrano, infine, in capo all’interessato-contraente debole del rapporto asimmetrico di trattamento dati personali le convergenze protettive tra normativa consumeristica e dati personali, sia in prospettiva individuale che collettiva.
Internet Access, Privacy ‘Essential for Freedom’: Proton Chief
Internet privacy company Proton can spot attacks on democracy in a country before they hit the headlines, simply by watching demand for its services explode, its chief told AFP.
When Russia blocked access to independent news sites following its invasion of Ukraine a year ago, the small company which provides virtual private networks (VPNs) saw “a 9,000 percent increase in sign-ups over just a period of a few days”, company chief executive Andy Yen said in an interview last week.
Switzerland-based Proton also saw a huge surge in demand for its VPNs, which are used to skirt online restrictions, in Iran last October as authorities cracked down harder on internet access amid flaring protests following Mahsa Amini’s death in custody.
“It was a factor of 10 at least,” Yen said.
Speaking at Proton’s headquarters outside Geneva, the 34-year-old particle physicist, who worked at Europe’s physics lab CERN before founding Proton in 2014, said the company had noticed that spikes in sign-ups “almost match… one-to-one” to places where democracy and freedom are under attack. “If there is a coup happening in Africa, we see it in our data before it makes the news.”
Severe consequences
Proton, perhaps best known for its encrypted email service, sees its mission of ensuring privacy and online access as a vital tool in shoring up democracy in the digital age.
“Privacy is something that is essential for freedom,” said Yen. The Proton chief, who grew up in Taiwan and says the Chinese threat hanging over the democratic island colored his world view, acknowledged the company’s mission had taken on added urgency since Russia’s invasion of its neighbor.
“We all see in Ukraine how important it is to have digital technologies that protect privacy and give people freedom of information so they can see real news sources,” he said.
There, as inside Russia, in Iran and elsewhere, Yen insisted it was “essential” for companies like Proton to stay “even if there is a financial loss”.
“If we abandon these markets, actually the consequences are quite severe.”
Privacy in focus
Proton, which began nine years ago with 10,000 users crowd-funding 500,000 euros, today counts more than 70 million users worldwide. The company, Yen said, has pursued a fundamentally different business model than that of big tech companies like Google and Facebook, which offer “free” services in exchange for selling users’ data to advertisers.
“If you’re a Google user, you’re not Google’s actual customer. What you actually are is a product,” he said.
Influenced by former US National Security Agency (NSA) contractor Edward Snowden’s disclosures of mass digital spying by US government agencies, Yen said he had been even more concerned about “corporate surveillance, which was much more massive”.
“If you consider what the NSA might have on you, it’s probably only a drop in the bucket compared to what Google and Facebook have.” Proton does offer free end-to-end encrypted email and VPN services, but instead of compensating by selling user data, it makes its money by selling monthly subscriptions for under $10 for extra features.
The company says its end-to-end encryption means that it has no access to the data transiting its servers and thus has no data to sell even if it wanted to.
It also means that it cannot turn over email content to governments that might demand it.
‘Cat and mouse’ – This, along with the VPNs helping skirt censorship, has put Proton’s team of around 400 employees in the crosshairs of powerful governments.
“We have had situations both in Russia and Iran where the entire resources of the state were thrown towards us,” Yen said, describing how the company has a team of engineers permanently on call to ensure its services “remain available and accessible”.
“It’s not rockets or missiles flying in the air, but there definitely is a fight for the future of the Russian internet, the Iranian internet and the internet in general.”
Yen recalled one sleepless Friday night in October when Proton’s VPNs came under a massive attack in Iran, and the nail-biting decision to roll out a new stealth VPN technology, aimed at making VPN traffic invisible to sensors.
The technology, which was developed to deal with issues in Russia, had yet to be tested on a large scale, and the engineers did not know for sure if it would hold up.
“We pulled the trigger, and it worked,” Yen said. “But it also could have gone the other way.”
In Iran, Russia and elsewhere, “it’s a cat and mouse game,” he said.
“I like to think that we’re a pretty fast mouse, but it’s also a pretty good cat sometimes.”
Why TikTok Is Being Banned on Gov’t Phones in US and Beyond
The United States is ratcheting up national security concerns about TikTok, mandating that all federal employees delete the Chinese-owned social media app from government-issued mobile phones. Other Western governments are pursuing similar bans, citing espionage fears.
So how serious is the threat? And should TikTok users who don’t work for the government be worried about the app, too?
The answers depend somewhat on whom you ask, and how concerned you are in general about technology companies gathering and sharing personal data.
Here’s what to know:
HOW ARE THE U.S. AND OTHER GOVERNMENTS BLOCKING TIKTOK?
The White House said Monday it is giving U.S. federal agencies 30 days to delete TikTok from all government-issued mobile devices.
Congress, the White House, U.S. armed forces and more than half of U.S. states had already banned TikTok amid concerns that its parent company, ByteDance, would give user data — such as browsing history and location — to the Chinese government, or push propaganda and misinformation on its behalf.
The European Union’s executive branch has temporarily banned TikTok from employee phones, and Denmark and Canada have announced efforts to block TikTok on government-issued phones.
China says the bans reveal the United States’ insecurities and are an abuse of state power. But they come at a time when Western technology companies, including Airbnb, Yahoo and LinkedIn, have been leaving China or downsizing operations there because of Beijing’s strict privacy law that specifies how companies can collect and store data.
WHAT ARE THE CONCERNS ABOUT TIKTOK?
Both the FBI and the Federal Communications Commission have warned that ByteDance could share TikTok user data with China’s authoritarian government.
A law China implemented in 2017 requires companies to give the government any personal data relevant to the country’s national security. There’s no evidence that TikTok has turned over such data, but fears abound due to the vast amount of user data it collects.
Concerns were heightened in December when ByteDance said it fired four employees who accessed data on two journalists from Buzzfeed News and The Financial Times while attempting to track down the source of a leaked report about the company. TikTok spokesperson Brooke Oberwetter said the breach was an “egregious misuse” of the employees’ authority.
There is also concern about TikTok’s content and whether it harms teenagers’ mental health. Researchers from the nonprofit Center for Countering Digital Hate said in a report released in December that eating disorder content on the platform had amassed 13.2 billion views. Roughly two-thirds of U.S. teens use TikTok, according to the Pew Research Center.
WHO HAS PUSHED FOR TIKTOK RESTRICTIONS?
In 2020, then-President Donald Trump and his administration sought to force ByteDance to sell off its U.S. assets and ban TikTok from app stores. Courts blocked Trump’s efforts, and President Joe Biden rescinded Trump’s orders after taking office but ordered an in-depth study of the issue. A planned sale of TikTok’s U.S. assets was shelved.
In Congress, concern about the app has been bipartisan. Congress passed the “No TikTok on Government Devices Act” in December as part of a sweeping government funding package. The legislation does allow for TikTok use in certain cases, including for national security, law enforcement and research purposes.
House Republicans are expected to move forward Tuesday with a bill that would give Biden the power to ban TikTok nationwide. The legislation, proposed by Rep. Mike McCaul, looks to circumvent the challenges the administration would face in court if it moved forward with sanctions against the company.
The bill has received pushback from civil liberties organizations. In a letter sent Monday to McCaul and Rep. Gregory Meeks, D-N.Y., ranking member of the Foreign Affairs Committee, the American Civil Liberties Union said a nationwide TikTok ban would be unconstitutional and would “likely result in banning many other businesses and applications as well.”
HOW RISKY IS TIKTOK?
It depends on who you ask.
U.S. Deputy Attorney General Lisa Monaco has expressed concerns that the Chinese government could gain access to user data.
“I don’t use TikTok, and I would not advise anyone to do so,” Monaco said earlier this month at the policy institute Chatham House in London.
TikTok said in a blog post in June that it will route all data from U.S. users to servers controlled by Oracle, the Silicon Valley company it chose as its U.S. tech partner in 2020 in an effort to avoid a nationwide ban. But it is storing backups of the data in its own servers in the U.S. and Singapore. The company said it expects to delete U.S. user data from its own servers, but it did not provide a timeline as to when that would occur.
But the amount of information TikTok collects might not be that different from other popular social media sites, experts say.
In an analysis published in 2021, the University of Toronto’s nonprofit Citizen Lab said TikTok and Facebook collect similar amounts of user data, including device identifiers that can be used to track a user and other information that can piece together a user’s behavior across different platforms. It’s valuable information for advertisers.
“If you are not comfortable with that level of data collection and sharing, you should avoid using the app,” the Citizen Lab report said.
WHAT ARE OTHER EXPERTS SAYING?
While the potential abuse of privacy by the Chinese government is concerning, “it’s equally concerning that the US government, and many other governments, already abuse and exploit the data collected by every other U.S.-based tech company with the same data-harvesting business practices,” said Evan Greer, director of the nonprofit advocacy group Fight for the Future.
“If policy makers want to protect Americans from surveillance, they should advocate for a basic privacy law that bans all companies from collecting so much sensitive data about us in the first place, rather than engaging in what amounts to xenophobic showboating that does exactly nothing to protect anyone,” Greer said.
Others say there is legitimate reason for concern.
People who use TikTok might think they’re not doing anything that would be of interest to a foreign government, but that’s not always the case, said Anton Dahbura, executive director of the Johns Hopkins University Information Security Institute. Important information about the United States is not strictly limited to nuclear power plants or military facilities; it extends to other sectors, such as food processing, the finance industry and universities, Dahbura said.
WHAT DOES TIKTOK SAY?
Its unclear how much the government-wide TikTok ban might impact the company. Oberwetter, the TikTok spokesperson, said it has “no way” of knowing whether its users are government employees.
The company, though, has questioned the bans, saying it has not been given an opportunity to answer questions and that governments were cutting themselves off from a platform beloved by millions.
“These bans are little more than political theater,” Oberwetter said.
TikTok CEO Shou Zi Chew is set to testify next month before Congress. The House Energy and Commerce Committee will ask about the company’s privacy and data-security practices, as well as its relationship with the Chinese government.
