Il Garante Privacy ha il dente avvelenato sul ddl Intelligenza Artificiale varato dal Governo per regolare lo sviluppo della nuova tecnologia. L’Autorità non ha preso bene la mancata designazione da parte del Governo come autorità di controllo dell’AI, attribuita invece in maniera bipartita ad Agid e ACN.

Non pochi i rilievi dell’Autorità nel parere rilasciato sul pacchetto di norme del ddl, che secondo il Garante dovrà essere integrato in più parti con precisazioni varie a tutela dei dati dei cittadini.

Tutela dati personali va specificata in premessa

In primo luogo, secondo il garante va specificato subito in premessa un nuovo articolo che a priori vincoli la norma alla conformità dei trattamenti di dati personali. L’obiettivo di base è salvaguardare “la competenza del Garante anche rispetto alle norme, del disegno di legge stesso, suscettibili di avere implicazioni in termini di protezione dei dati”. Una questione di principio, questa, per chiarire da subito che la stella polare da seguire è quella della tutela dei dati personali e che il Garante ne deve gioco forza essere il nume tutelare.

Age verification va comprovata

Un altro aspetto sensibile è quello della age verification, della soglia di età fissata a 14 anni sotto la quale l’uso di sistemi di AI da parte dei minori deve essere illegittima. Servono sistemi comprovati per evitare una facile elusione del controllo dell’età da parte dei minori.

Dati sanitari, adeguarsi al regolamento Ue. Perplessità sull’Agenas

Per quanto riguarda i dati sanitari, va esplicitata la preferenza all’anonimizzazione e il divieto di trasferimento o comunicazione di dati sanitari nonché limiti precisi di conservazione, in linea con quanto previsto dal dal Regolamento sullo Spazio europeo dei dati sanitari, approvato il 24 aprile scorso.

Sempre in tema sanitario, il Garante sottolinea che “suscita perplessità l’attribuzione della titolarità dei trattamenti effettuati attraverso la piattaforma a un ente strumentale quale AGENAS anziché al Dicastero cui il trattamento è complessivamente imputabile e che dispone dei correlati poteri provvedimentali e, lato sensu, decisori”.

AI e mondo del lavoro, regole parziali

L’articolo 10 volto a regolare l’uso dell’AI in ambito lavorativo, “appare in certa misura parziale, riferendosi alla sola fase successiva all’instaurazione del rapporto di lavoro, laddove i sistemi di i.a. sono spesso utilizzati in fase preassuntiva, a fini di selezione del personale, cui pertanto le garanzie introdotte vanno estese. Si dovrebbe, pertanto, introdurre un comma ulteriore, dal tenore seguente: “Le disposizioni del presente articolo si applicano, ove compatibili, anche ai trattamenti effettuati in fase preassuntiva””.

Chiarire e potenziare il ruolo del Garante. Gli obblighi di ACN e Agid

Il ruolo centrale del garante va poi chiarito ed esplicitato “ai sensi degli artt. 8 CDFUE e 16 TFUE, per la protezione dei dati personali: diritto, appunto fondamentale, le cui istanze di tutela si sovrappongono pressoché costantemente con l’applicazione dei sistemi di i.a.”, si legge nel parere.

“Anche al fine di evitare antinomie o dubbi in sede applicativa, è opportuno perfezionare il comma 3 dell’articolo 18, estendendo più esplicitamente la clausola di salvaguardia delle funzioni del Garante anche alle norme del disegno di legge che abbiano implicazioni in termini di protezione dei dati.

Garante partecipi al comitato di coordinamento sull’AI

Per tale ragione, è anche opportuno prevedere la partecipazione del Garante al Comitato di coordinamento di cui all’articolo 18, c.2, secondo periodo, per realizzare pienamente quella leale cooperazione tra autorità competenti prevista dall’AI Act. Declinando in maniera più articolata le implicazioni di tale cooperazione, è inoltre opportuno integrare l’articolo prevedendo, infine, che AgID e ACN trasmettano al Garante gli atti dei procedimenti in relazione ai quali emergano profili suscettibili di rilevare in termini di protezione dati, richiedendo altresì il parere dell’Autorità rispetto a fattispecie, al loro esame, che coinvolgano aspetti di protezione dei dati. Il Garante trasmetterà, per parte sua, elementi informativi in ordine a profili di competenza di AgID o ACN suscettibili di emergere nella trattazione di propri procedimenti”.

Un ulteriore obbligo di designazione del Garante, quale autorità competente per i “sistemi di i.a. ad alto rischio”.

Leggi anche: Ddl AI, ok del Garante, ma con correttivi: “Siamo noi Autorità competente su AI ad alto rischio”

Garante Privacy, Stanzione: ‘AI ormai nella nostra vita. Ma non sappiamo ancora bene cosa vuol dire’

Leggi le altre notizie sull’home page di Key4biz

https://www.key4biz.it/ddl-ai-il-garante-privacy-boccia-lagenas-sui-dati-sanitari-chiesti-paletti-per-acn-e-agid/501471/

It’s no secret that despite significant investment from companies like OpenAI and Runway, AI-generated videos still struggle to achieve convincing realism at times. Some of the most amusing fails end up on social media, which has led to a new response trend on Chinese social media platforms TikTok and Bilibili where users create videos that mock the imperfections of AI-generated content. The trend has since spread to X (formerly Twitter) in the US, where users have been sharing the humorous parodies.

In particular, the videos seem to parody image synthesis videos where subjects seamlessly morph into other people or objects in unexpected and physically impossible ways. Chinese social media replicate these unusual visual non-sequiturs without special effects by positioning their bodies in unusual ways as new and unexpected objects appear on-camera from out of frame.

This exaggerated mimicry has struck a chord with viewers on X, who find the parodies entertaining. User @theGioM shared one video, seen above. “This is high-level performance arts,” wrote one X user. “art is imitating life imitating ai, almost shedded a tear.” Another commented, “I feel like it still needs a motorcycle the turns into a speedboat and takes off into the sky. Other than that, excellent work.”

While these parodies poke fun at current limitations, tech companies are actively attempting to overcome them with more training data (examples analyzed by AI models that teach them how to create videos) and computational training time. OpenAI unveiled Sora in February, capable of creating realistic scenes if they closely match examples found in training data. Runway’s Gen-3 Alpha suffers a similar fate: It can create brief clips of convincing video within a narrow set of constraints. This means that generated videos of situations outside the dataset often end up hilariously weird.

It’s worth noting that actor Will Smith beat Chinese social media users to this trend in February by poking fun at a horrific 2023 viral AI-generated video that attempted to depict him eating spaghetti. That may also bring back memories of other amusing video synthesis failures, such as May 2023’s AI-generated beer commercial, created using Runway’s earlier Gen-2 model.

While imitating imperfect AI videos may seem strange to some, people regularly make money pretending to be NPCs (non-player characters—a term for computer-controlled video game characters) on TikTok.

For anyone alive during the 1980s, witnessing this fast-changing and often bizarre new media world can cause some cognitive whiplash, but the world is a weird place full of wonders beyond the imagination. “There are more things in Heaven and Earth, Horatio, than are dreamt of in your philosophy,” as Hamlet once famously said. “Including people pretending to be video game characters and flawed video synthesis outputs.”

https://arstechnica.com/?p=2043808

Neural processing units (NPUs) are becoming commonplace in chips from Intel and AMD after several years of being something you’d find mostly in smartphones and tablets (and Macs). But as more companies push to do more generative AI processing, image editing, and chatbot-ing locally on-device instead of in the cloud, being able to measure NPU performance will become more important to people making purchasing decisions.

Enter Primate Labs, developers of Geekbench. The main Geekbench app is designed to test CPU performance as well as GPU compute performance, but for the last few years, the company has been experimenting with a side project called Geekbench ML (for “Machine Learning”) to test the inference performance of NPUs. Now, as Microsoft’s Copilot+ initiative gets off the ground and Intel, AMD, Qualcomm, and Apple all push to boost NPU performance, Primate Labs is bumping Geekbench ML to version 1.0 and renaming it “Geekbench AI,” a change that will presumably help it ride the wave of AI-related buzz.

“Just as CPU-bound workloads vary in how they can take advantage of multiple cores or threads for performance scaling (necessitating both single-core and multi-core metrics in most related benchmarks), AI workloads cover a range of precision levels, depending on the task needed and the hardware available,” wrote Primate Labs’ John Poole in a blog post about the update. “Geekbench AI presents its summary for a range of workload tests accomplished with single-precision data, half-precision data, and quantized data, covering a variety used by developers in terms of both precision and purpose in AI systems.”

In addition to measuring speed, Geekbench AI also attempts to measure accuracy, which is important for machine-learning workloads that rely on producing consistent outcomes (identifying and cataloging people and objects in a photo library, for example).

Geekbench AI supports several AI frameworks: OpenVINO for Windows and Linux, ONNX for Windows, Qualcomm’s QNN on Snapdragon-powered Arm PCs, Apple’s CoreML on macOS and iOS, and a number of vendor-specific frameworks on various Android devices. The app can run these workloads on the CPU, GPU, or NPU, at least when your device has a compatible NPU installed.

On Windows PCs, where NPU support and APIs like Microsoft’s DirectML are still works in progress, Geekbench AI supports Intel and Qualcomm’s NPUs but not AMD’s (yet).

“We’re hoping to add AMD NPU support in a future version once we have more clarity on how best to enable them from AMD,” Poole told Ars.

Geekbench AI is available for Windows, macOS, Linux, iOS/iPadOS, and Android. It’s free to use, though a Pro license gets you command-line tools, the ability to run the benchmark without uploading results to the Geekbench Browser, and a few other benefits. Though the app is hitting 1.0 today, the Primate Labs team expects to update the app frequently for new hardware, frameworks, and workloads as necessary.

“AI is nothing if not fast-changing,” Poole continued in the announcement post, “so anticipate new releases and updates as needs and AI features in the market change.”

https://arstechnica.com/?p=2043053

On Tuesday, Elon Musk’s AI company, xAI, announced the beta release of two new language models, Grok-2 and Grok-2 mini, available to subscribers of his social media platform, X (formerly Twitter). The models are also linked to the recently released Flux image-synthesis model, which allows X users to create largely uncensored photorealistic images that can be shared on the site.

“Flux, accessible through Grok, is an excellent text-to-image generator, but it is also really good at creating fake photographs of real locations and people, and sending them right to Twitter,” wrote frequent AI commentator Ethan Mollick on X. “Does anyone know if they are watermarking these in any way? It would be a good idea.”

In a report posted earlier today, The Verge noted that Grok’s image-generation capabilities appear to have minimal safeguards, allowing users to create potentially controversial content. According to their testing, when prompted, Grok produced images depicting political figures in compromising situations, copyrighted characters, and scenes of violence.

The Verge found that while Grok claims to have certain limitations, such as avoiding pornographic or excessively violent content, these rules seem inconsistent in practice. Unlike other major AI image generators, Grok does not appear to refuse prompts involving real people or add identifying watermarks to its outputs.

Given what people are generating so far—including images of Donald Trump and Kamala Harris kissing or giving a thumbs-up on the way to the Twin Towers in an apparent 9/11 attack—the unrestricted outputs may not last for long. But then again, Elon Musk has made a big deal out of “freedom of speech” on his platform, so perhaps the capability will remain (until someone likely files a defamation or copyright suit).

People using Grok’s image generator for shock value brings up an old question in AI at this point: Should misuse of an AI image generator be the responsibility of the person who creates the prompt, the organization that created the AI model, or the platform that hosts the images? So far, there is no clear consensus, and the situation has yet to be resolved legally, although a new proposed US law called the NO FAKES Act would presumably hold X liable for the creation of realistic image deepfakes.

On Thursday morning, Musk weighed in on people using the AI image generator on X. In response to an open question from Mollick about why xAI chose to use Flux to allow users to generate images without extra content filters, Musk wrote, “We have our own image generation system under development, but it’s a few months away, so this seemed like a good intermediate step for people to have some fun.”

With Grok-2, the GPT-4 ceiling still holds

Looking beyond images, in a release blog, xAI claims that Grok-2 and Grok-2 mini represent significant advancements in capabilities, with Grok-2 supposedly outperforming some leading competitors in recent benchmarks and what we call “vibemarks.” It’s always wise to approach those claims with a dose of skepticism, but it appears that the “GPT-4 class” of AI language models (those with similar capability to OpenAI’s model) has grown larger, but the GPT-4 barrier has not yet been smashed.

“There are now five GPT-4 class models: GPT-4o, Claude 3.5, Gemini 1.5, Llama 3.1, and now Grok 2,” wrote Ethan Mollick on X. “All of the labs are saying there is room left for continued giant improvements, but we haven’t seen any models truly leap above GPT-4… yet.”

xAI says it recently introduced an early version of Grok-2 to the LMSYS Chatbot Arena under the name “sus-column-r,” where it reportedly achieved a higher overall Elo score than models like Claude 3.5 Sonnet and GPT-4 Turbo. Chatbot Arena is a popular subjective vibemarking website for AI models, but it has been the subject of controversy recently when people disagreed with OpenAI’s GPT-4o mini placing so highly in the rankings.

According to xAI, both new Grok models show improvements over predecessor Grok-1.5 in areas like graduate-level science knowledge, general knowledge, and math problem-solving in benchmarks that have similarly proved controversial. The company also highlighted Grok-2’s performance on visual tasks, claiming state-of-the-art results in visual math reasoning and document-based question answering.

The models are now available to X Premium and Premium+ subscribers through an updated app interface. Unlike some of its competitors in the open-weights space, xAI isn’t releasing the model weights for download or independent verification. This closed approach stands in stark contrast to recent moves by Meta, which recently released its Llama 3.1 405B model for anyone to download and run locally.

xAI plans to release both models through an enterprise API later this month. The company says this API will feature multi-region deployment options and security measures, like mandatory multifactor authentication. Details on pricing, usage limits, or data-handling policies have not yet been announced.

Photorealistic image generation aside, perhaps Grok-2’s biggest liability is its deep link to X, which gives it a tendency to pull inaccurate information from tweets. It’s a bit like if you had a friend who insisted on checking the social media site before answering any of your questions, even when it wasn’t particularly relevant.

As Mollick pointed out on X, this close link can be annoying: “I only have access to Grok 2 mini right now, and it seems like a solid model, but often seems ill-served by its RAG connection to Twitter,” he wrote. “The model is fed results from Twitter that seem irrelevant to the prompt, and then desperately tries to connect them into something coherent.”

This story was updated at 9:30 AM on August 15, 2024 to include a quote from Elon Musk about Grok’s image generator.

https://arstechnica.com/?p=2043070

Over the past few days, a software package called Deep-Live-Cam has been going viral on social media because it can take the face of a person extracted from a single photo and apply it to a live webcam video source while following pose, lighting, and expressions performed by the person on the webcam. While the results aren’t perfect, the software shows how quickly the tech is developing—and how the capability to deceive others remotely is getting dramatically easier over time.

The Deep-Live-Cam software project has been in the works since late last year, but example videos that show a person imitating Elon Musk and Republican Vice Presidential candidate J.D. Vance (among others) in real time have been making the rounds online. The avalanche of attention briefly made the open source project leap to No. 1 on GitHub’s trending repositories list (it’s currently at No. 4 as of this writing), where it is available for download for free.

“Weird how all the major innovations coming out of tech lately are under the Fraud skill tree,” wrote illustrator Corey Brickley in an X thread reacting to an example video of Deep-Live-Cam in action. In another post, he wrote, “Nice remember to establish code words with your parents everyone,” referring to the potential for similar tools to be used for remote deception—and the concept of using a safe word, shared among friends and family, to establish your true identity.

Face-swapping technology is not new. The term “deepfake” itself originated in 2017 from a Reddit user called “deepfakes” (combining the terms “deep learning” and “fakes”), who posted pornography that swapped a performer’s face with the face of a celebrity. At that time, the technology was expensive and slow and did not operate in real time. However, due to projects like Deep-Live-Cam, it’s getting easier for anyone to use this technology at home with a regular PC and free software.

The dangers of deepfakes aren’t new, either. In February, we covered an alleged heist in Hong Kong where someone impersonated a company’s CFO over a video call and walked off with over $25 million dollars. Audio deepfakes have led to other financial fraud or extortion schemes. We might expect instances of remote video fraud to increase with easily available real-time deepfake software, and it’s not just celebrities or politicians who might be affected.

Using face-swapping software, someone could take a photo of you from social media and impersonate you to someone not entirely familiar with how you look and act—given the current need to imitate similar mannerisms, voice, hair, clothing, and body structure. Techniques to clone those aspects of appearance and voice also exist (using voice cloning and video image-to-image AI synthesis) but have not yet reached reliable photorealistic real-time implementations. But given time, that technology will likely also become readily available and easy to use.

How does it work?

Like many open source GitHub projects, Deep-Live-Cam wraps together several existing software packages under a new interface (and is itself a fork of an earlier project called “roop“). It first detects faces in both the source and target images (such as a frame of live video). It then uses a pre-trained AI model called “inswapper” to perform the actual face swap and another model called GFPGAN to improve the quality of the swapped faces by enhancing details and correcting artifacts that occur during the face-swapping process.

The inswapper model, developed by a project called InsightFace, can guess what a person (in a provided photo) might look like using different expressions and from different angles because it was trained on a vast dataset containing millions of facial images of thousands of individuals captured from various angles, under different lighting conditions, and with diverse expressions.

During training, the neural network underlying the inswapper model developed an “understanding” of facial structures and their dynamics under various conditions, including learning the ability to infer the three-dimensional structure of a face from a two-dimensional image. It also became capable of separating identity-specific features, which remain constant across different images of the same person, from pose-specific features that change with angle and expression. This separation allows the model to generate new face images that combine the identity of one face with the pose, expression, and lighting of another.

Deep-Live-Cam is far from the only face-swapping software project out there. Another GitHub project, called facefusion, uses the same face-swapping AI model with a different interface. Most of them rely heavily on a nested web of Python and deep learning libraries like PyTorch, so Deep-Live-Cam isn’t as easy as a one-click install yet. But it’s likely that this kind of face-swapping capability will become even easier to install over time and will likely improve in quality as people iterate and build on each other’s work in the open source AI development space.

https://arstechnica.com/?p=2042710

Donald Trump may have coined a new term in his latest false attack on Kamala Harris’ presidential campaign. In a pair of posts on Truth Social over the weekend, the former president said that Vice President Kamala Harris “A.I.’d” photos of a huge crowd that showed up to see her speak at a Detroit airport campaign rally last week.

“There was nobody at the plane, and she ‘A.I.’d’ it, and showed a massive ‘crowd’ of so-called followers, BUT THEY DIDN’T EXIST!” Trump wrote. “She’s a CHEATER. She had NOBODY waiting, and the ‘crowd’ looked like 10,000 people! Same thing is happening with her fake ‘crowds’ at her speeches.”

The Harris campaign responded with its own post saying that the image is “an actual photo of a 15,000-person crowd for Harris-Walz in Michigan.”

Aside from the novel use of “AI” as a verb, Trump’s post marks the first time, that we know of, that a US presidential candidate has personally raised the specter of AI-generated fakery by an opponent (rather than by political consultants or random social media users). The accusations, false as they are, prey on widespread fears and misunderstandings over the trustworthiness of online information in the AI age.

It would be nice to think that we could just say Trump’s claims here are categorically false and leave it at that. But as artificial intelligence tools become increasingly good at generating photorealistic images, it’s worth outlining the many specific ways we can tell that Harris’ crowd photos are indeed authentic. Consider this a guide for potential techniques you can use the next time you come across accusations that some online image has been “A.I.’d” to fool you.

Context and sourcing

By far the easiest way to tell Harris’ crowds are real is from the vast number of corroborating sources showing those same crowds. Both the AP and Getty have numerous shots of the rally crowd from multiple angles, as do journalists and attendees who were at the event. Local news sources posted video of the crowds at the event, as did multiple attendees on the ground. Reporters from multiple outlets reported directly on the crowds in their accounts: Local outlet MLive estimated the crowd size at 15,000, for instance, while The New York Times noted that the event was “witnessed by thousands of people and news outlets, including The New York Times, and the number of attendees claimed by her campaign is in line with what was visible on the ground.”

The Harris/Walz rally in Detroit is buzzing after a performance from the Detroit Youth Choir. #Michigan pic.twitter.com/sdFQvHhG3I

— Nora Eckert (@NoraEckert) August 7, 2024

Suffice it to say that this mountain of evidence from direct sources weighs more heavily than marked-up images from conservative commentators like Chuck Callesto and Dinesh D’Souza, both of whom have been caught spreading election disinformation in the past.

When it comes to accusations of AI fakery, the more disparate sources of information you have, the better. While a single source can easily generate a plausible-looking image of an event, multiple independent sources showing the same event from multiple angles are much less likely to be in on the same hoax. Photos that line up with video evidence are even better, especially since creating convincing long-form videos of humans or complex scenes remains a challenge for many AI tools.

It’s also important to track down the original source of whatever alleged AI image you’re looking at. It’s incredibly easy for a social media user to create an AI-generated image, claim it came from a news report or live footage of an event, then use obvious flaws in that fake image as “evidence” that the event itself was faked. Links to original imagery from an original source’s own website or verified account are much more reliable than screengrabs that could have originated anywhere (and/or been modified by anyone).

https://arstechnica.com/?p=2042525

In many cases, AIs are trained on material that’s either made or curated by humans. As a result, it can become a significant challenge to keep the AI from replicating the biases of those humans and the society they belong to. And the stakes are high, given we’re using AIs to make medical and financial decisions.

But some researchers at Washington University in St. Louis have found an additional wrinkle in these challenges: The people doing the training may potentially change their behavior when they know it can influence the future choices made by an AI. And, in at least some cases, they carry the changed behaviors into situations that don’t involve AI training.

Would you like to play a game?

The work involved getting volunteers to participate in a simple form of game theory. Testers gave two participants a pot of money—$10, in this case. One of the two was then asked to offer some fraction of that money to the other, who could choose to accept or reject the offer. If the offer was rejected, nobody got any money.

From a purely rational economic perspective, people should accept anything they’re offered, since they’ll end up with more money than they would have otherwise. But in reality, people tend to reject offers that deviate too much from a 50/50 split, as they have a sense that a highly imbalanced split is unfair. Their rejection allows them to punish the person who made the unfair offer. While there are some cultural differences in terms of where the split becomes unfair, this effect has been replicated many times, including in the current work.

The twist with the new work, performed by Lauren Treiman, Chien-Ju Ho, and Wouter Kool, is that they told some of the participants that their partner was an AI, and the results of their interactions with it would be fed back into the system to train its future performance.

This takes something that’s implicit in a purely game-theory-focused setup—that rejecting offers can help partners figure out what sorts of offers are fair—and makes it highly explicit. Participants, or at least the subset involved in the experimental group that are being told they’re training an AI, could readily infer that their actions would influence the AI’s future offers.

The question the researchers were curious about was whether this would influence the behavior of the human participants. They compared this to the behavior of a control group who just participated in the standard game theory test.

Training fairness

Treiman, Ho, and Kool had pre-registered a number of multivariate analyses that they planned to perform with the data. But these didn’t always produce consistent results between experiments, possibly because there weren’t enough participants to tease out relatively subtle effects with any statistical confidence and possibly because the relatively large number of tests would mean that a few positive results would turn up by chance.

So, we’ll focus on the simplest question that was addressed: Did being told that you were training an AI alter someone’s behavior? This question was asked through a number of experiments that were very similar. (One of the key differences between them was whether the information regarding AI training was displayed with a camera icon, since people will sometimes change their behavior if they’re aware they’re being observed.)

The answer to the question is a clear yes: people will in fact change their behavior when they think they’re training an AI. Through a number of experiments, participants were more likely to reject unfair offers if they were told that their sessions would be used to train an AI. In a few of the experiments, they were also more likely to reject what were considered fair offers (in US populations, the rejection rate goes up dramatically once someone proposes a 70/30 split, meaning $7 goes to the person making the proposal in these experiments). The researchers suspect this is due to people being more likely to reject borderline “fair” offers such as a 60/40 split.

This happened even though rejecting any offer exacts an economic cost on the participants. And people persisted in this behavior even when they were told that they wouldn’t ever interact with the AI after training was complete, meaning they wouldn’t personally benefit from any changes in the AI’s behavior. So here, it appeared that people would make a financial sacrifice to train the AI in a way that would benefit others.

Strikingly, in two of the three experiments that did follow up testing, participants continued to reject offers at a higher rate two days after their participation in the AI training, even when they were told that their actions were no longer being used to train the AI. So, to some extent, participating in AI training seems to have caused them to train themselves to behave differently.

Obviously, this won’t affect every sort of AI training, and a lot of the work that goes into producing material that’s used in training something like a Large Language Model won’t have been done with any awareness that it might be used to train an AI. Still, there’s plenty of cases where humans do get more directly involved in training, so it’s worthwhile being aware that this is another route that can allow biases to creep in.

PNAS, 2024. DOI: 10.1073/pnas.2408731121  (About DOIs).

https://arstechnica.com/?p=2042327

On Thursday, OpenAI released the “system card” for ChatGPT’s new GPT-4o AI model that details model limitations and safety testing procedures. Among other examples, the document reveals that in rare occurrences during testing, the model’s Advanced Voice Mode unintentionally imitated users’ voices without permission. Currently, OpenAI has safeguards in place that prevent this from happening, but the instance reflects the growing complexity of safely architecting with an AI chatbot that could potentially imitate any voice from a small clip.

Advanced Voice Mode is a feature of ChatGPT that allows users to have spoken conversations with the AI assistant.

In a section of the GPT-4o system card titled “Unauthorized voice generation,” OpenAI details an episode where a noisy input somehow prompted the model to suddenly imitate the user’s voice. “Voice generation can also occur in non-adversarial situations, such as our use of that ability to generate voices for ChatGPT’s advanced voice mode,” OpenAI writes. “During testing, we also observed rare instances where the model would unintentionally generate an output emulating the user’s voice.”

In this example of unintentional voice generation provided by OpenAI, the AI model outbursts “No!” and continues the sentence in a voice that sounds similar to the “red teamer” heard in the beginning of the clip. (A red teamer is a person hired by a company to do adversarial testing.)

It would certainly be creepy to be talking to a machine and then have it unexpectedly begin talking to you in your own voice. Ordinarily, OpenAI has safeguards to prevent this, which is why the company says this occurrence was rare even before it developed ways to prevent it completely. But the example prompted BuzzFeed data scientist Max Woolf to tweet, “OpenAI just leaked the plot of Black Mirror’s next season.”

Audio prompt injections

How could voice imitation happen with OpenAI’s new model? The primary clue lies elsewhere in the GPT-4o system card. To create voices, GPT-4o can apparently synthesize almost any type of sound found in its training data, including sound effects and music (though OpenAI discourages that behavior with special instructions).

As noted in the system card, the model can fundamentally imitate any voice based on a short audio clip. OpenAI guides this capability safely by providing an authorized voice sample (of a hired voice actor) that it is instructed to imitate. It provides the sample in the AI model’s system prompt (what OpenAI calls the “system message”) at the beginning of a conversation. “We supervise ideal completions using the voice sample in the system message as the base voice,” writes OpenAI.

In text-only LLMs, the system message is a hidden set of text instructions that guides behavior of the chatbot that gets added to the conversation history silently just before the chat session begins. Successive interactions are appended to the same chat history, and the entire context (often called a “context window”) is fed back into the AI model each time the user provides a new input.

(It’s probably time to update this diagram created in early 2023 below, but it shows how the context window works in an AI chat. Just imagine that the first prompt is a system message that says things like “You are a helpful chatbot. You do not talk about violent acts, etc.”)

Since GPT-4o is multimodal and can process tokenized audio, OpenAI can also use audio inputs as part of the model’s system prompt, and that’s what it does when OpenAI provides an authorized voice sample for the model to imitate. The company also uses another system to detect if the model is generating unauthorized audio. “We only allow the model to use certain pre-selected voices,” writes OpenAI, “and use an output classifier to detect if the model deviates from that.”

https://arstechnica.com/?p=2042102

Parere favorevole del Garante Privacy sullo schema di disegno di legge governativo sull’IA, recante anche delega legislativa per l’adeguamento al Regolamento Ue sull’Intelligenza Artificiale (n.2024/1689 – AI Act).

Il disegno di legge disciplina ricerca, sperimentazione, sviluppo, adozione e applicazione dei sistemi e modelli di Intelligenza Artificiale (IA) nei diversi settori della società (sanità, giustizia, lavoro e professioni, sicurezza e difesa nazionale, etc.).

Nel dare il proprio parere favorevole al testo, il Garante ha tuttavia chiesto al Governo di integrarlo in più parti per garantire una maggiore tutela dei dati personali dei cittadini.

In particolare l’Autorità ha chiesto di introdurre un nuovo articolo per precisare che i trattamenti di dati personali effettuati attraverso i sistemi di intelligenza artificiale devono rispettare la normativa privacy nazionale ed europea. Il testo, inoltre, dovrà essere integrato con uno specifico riferimento a sistemi adeguati di verifica dell’età (c.d. age verification) in grado di garantire limitazioni o divieti all’uso dei sistemi di IA da parte dei minori.

Nel caso poi di utilizzo di sistemi di IA in ambito sanitario ad alto rischio, il Garante ha chiesto di indicare particolari limitazioni per l’utilizzo dei dati (conservazione, divieto di trasmissione, trasferimento o comunicazione) e la preferenza per l’uso di dati sintetici o anonimi.

È stato inoltre richiesto di indicare il Garante – come previsto nell’AI Act – quale Autorità competente per i sistemi di Intelligenza Artificiale ad alto rischio utilizzati ad es. per le attività di law enforcement, identificazione biometrica remota, riconoscimento delle emozioni, gestione delle frontiere, amministrazione della giustizia e processi democratici.

Leggi anche:
Ddl AI, Butti: “Ecco come svilupperemo in Italia l’intelligenza artificiale generativa”
AI, ok in Cdm al Ddl. Sconti per i ricercatori che tornano e reclusione fino a 5 anni per ‘danno ingiusto’

Leggi le altre notizie sull’home page di Key4biz

https://www.key4biz.it/ddl-ai-ok-del-garante-ma-con-correttivi-siamo-noi-autorita-competente-su-ai-ad-alto-rischio/501368/

Bill Gross made his name in the tech world in the 1990s, when he came up with a novel way for search engines to make money on advertising. Under his pricing scheme, advertisers would pay when people clicked on their ads. Now, the “pay-per-click” guy has founded a startup called ProRata, which has an audacious, possibly pie-in-the-sky business model: “AI pay-per-use.”

Gross, who is CEO of the Pasadena, California, company, doesn’t mince words about the generative AI industry. “It’s stealing,” he says. “They’re shoplifting and laundering the world’s knowledge to their benefit.”

AI companies often argue that they need vast troves of data to create cutting-edge generative tools and that scraping data from the Internet, whether it’s text from websites, video or captions from YouTube, or books pilfered from pirate libraries, is legally allowed. Gross doesn’t buy that argument. “I think it’s bullshit,” he says.

So do plenty of media executives, artists, writers, musicians, and other rights-holders who are pushing back—it’s hard to keep up with the constant flurry of copyright lawsuits filed against AI companies, alleging that the way they operate amounts to theft.

But Gross thinks ProRata offers a solution that beats legal battles. “To make it fair—that’s what I’m trying to do,” he says. “I don’t think this should be solved by lawsuits.”

His company aims to arrange revenue-sharing deals so publishers and individuals get paid when AI companies use their work. Gross explains it like this: “We can take the output of generative AI, whether it’s text or an image or music or a movie, and break it down into the components, to figure out where they came from, and then give a percentage attribution to each copyright holder, and then pay them accordingly.” ProRata has filed patent applications for the algorithms it created to assign attribution and make the appropriate payments.

This week, the company, which has raised $25 million, launched with a number of big-name partners, including Universal Music Group, the Financial Times, The Atlantic, and media company Axel Springer. In addition, it has made deals with authors with large followings, including Tony Robbins, Neal Postman, and Scott Galloway. (It has also partnered with former White House Communications Director Anthony Scaramucci.)

Even journalism professor Jeff Jarvis, who believes scraping the web for AI training is fair use, has signed on. He tells WIRED that it’s smart for people in the news industry to band together to get AI companies access to “credible and current information” to include in their output. “I hope that ProRata might open discussion for what could turn into APIs [application programming interfaces] for various content,” he says.

Following the company’s initial announcement, Gross says he had a deluge of messages from other companies asking to sign up, including a text from Time CEO Jessica Sibley. ProRata secured a deal with Time, the publisher confirmed to WIRED. He plans to pursue agreements with high-profile YouTubers and other individual online stars.

The key word here is “plans.” The company is still in its very early days, and Gross is talking a big game. As a proof of concept, ProRata is launching its own subscription chatbot-style search engine in October. Unlike other AI search products, ProRata’s search tool will exclusively use licensed data. There’s nothing scraped using a web crawler. “Nothing from Reddit,” he says.

Ed Newton-Rex, a former Stability AI executive who now runs the ethical data licensing nonprofit Fairly Trained, is heartened by ProRata’s debut. “It’s great to see a generative AI company licensing training data before releasing their model, in contrast to many other companies’ approach,” he says. “The deals they have in place further demonstrate media companies’ openness to working with good actors.”

Gross wants the search engine to demonstrate that quality of data is more important than quantity and believes that limiting the model to trustworthy information sources will curb hallucinations. “I’m claiming that 70 million good documents is actually superior to 70 billion bad documents,” he says. “It’s going to lead to better answers.”

What’s more, Gross thinks he can get enough people to sign up for this all-licensed-data AI search engine to make as much money needed to pay its data providers their allotted share. “Every month the partners will get a statement from us saying, ‘Here’s what people search for, here’s how your content was used, and here’s your pro rata check,’” he says.

Other startups already are jostling for prominence in this new world of training-data licensing, like the marketplaces TollBit and Human Native AI. A nonprofit called the Dataset Providers Alliance was formed earlier this summer to push for more standards in licensing; founding members include services like the Global Copyright Exchange and Datarade.

ProRata’s business model hinges in part on its plan to license its attribution and payment technologies to other companies, including major AI players. Some of those companies have begun striking their own deals with publishers. (The Atlantic and Axel Springer, for instance, have agreements with OpenAI.) Gross hopes that AI companies will find licensing ProRata’s models more affordable than creating them in-house.

“I’ll license the system to anyone who wants to use it,” Gross says. “I want to make it so cheap that it’s like a Visa or MasterCard fee.”

This story originally appeared on wired.com.

https://arstechnica.com/?p=2042108