The decimation of local media is by now a sad, familiar tale experienced by tens of millions of Americans all over the country. In a report released today, the Senate Commerce Committee’s top Democrat is laying blame for the bloodbath squarely at the feet of Google and Facebook, claiming the companies have participated in destroying local news in the pursuit of monopolizing monetization.

Sen. Maria Cantwell (D-Wash.) released the scathing new report (PDF) today, ahead of a scheduled hearing Wednesday in which Twitter CEO Jack Dorsey, Google CEO Sundar Pichai, and Facebook CEO Mark Zuckerberg will be testifying.

It is true that the advertising streams that funded local newspapers before the advent of the Internet changed dramatically in the 21st century as news went digital. Even so, the report says, outlets could have been able to manage except that “local news has been hijacked by a few large news aggregation platforms, most notably Google and Facebook, which have become the dominant players in online advertising.”

“These trillion-dollar companies scrape local news content and data for their own sites and leverage their market dominance to force local news to accept little to no compensation for their intellectual property,” the report goes on. It explains:

The platforms use their market position to force local news into “take it or leave it” contracts, which limit the ability for local news companies to be fairly compensated. If a news outlet wants a summary of its content featured in mobile formats created by the platforms, the news outlet typically must agree to a contract that allows the platforms to obtain the content without fair financial compensation and collect user data from those who view it, among other provisions. The benefit to the news outlet is the possibility that readers will click on a link to its site to learn more. However, this arrangement also results in a disproportionate share of overall advertising revenue flowing to the platforms instead of the content creators on their individual news sites, where building a relationship and traffic is key to sustaining long-term revenue. The popularity of the major news aggregation platforms with readers means they can push local newspapers into providing local news content to them under these unfair terms.

A big and familiar problem

Cantwell’s findings echo the findings of a blockbuster antitrust report published by the House antitrust subcommittee earlier this month.

“Despite significant growth in online traffic among the nation’s leading newspapers, print and digital newsrooms across the country are laying off reporters or folding altogether,” the House report observed. “The digital advertising market is highly concentrated, with Google and Facebook controlling the majority of the online advertising market in the United States… News publishers have raised concerns that this significant level of concentration in the online market—commonly known as the digital duopoly—has harmed the quality and availability of journalism.”

The new report cites both the House report as well as several other active investigations that state, federal, and international regulators are currently conducting into tech firms’ business practices.

Antitrust investigations, however necessary, are slow, Cantwell notes. Even if the Federal Trade Commission, Department of Justice, or state attorneys general take action to mitigate the harms Google and Facebook cause to local journalism, the change might come too late. Congress, she suggests, should take steps more quickly.

So what to do?

First, the report suggests, Congress should act immediately to provide additional COVID-19 relief to individuals and institutions, including making local newspaper, radio, and television outlets eligible for support. A COVID relief bill would help stanch the immediate bleeding: in journalism as in many sectors, COVID-19 took what was a building crisis and made it a catastrophe. The Poynter Institute has kept a running list of layoffs, furloughs, and closures in print, broadcast, and digital media this year, and the numbers are ugly.

Cantwell’s report also suggests that Congress “should consider requiring that news aggregation platforms enter into good-faith negotiations with local news organizations and pay them fair market value for their content.”

Competition regulators in other nations have tried enforcing similar mandates in the past decades, and Google has fought furiously against them all. France and Google, for example, have been in a protracted fight for years over a law in that country that requires Google to pay to link to articles on French news sites.

Most recently, the Australian Competition and Consumer Commission—that nation’s equivalent of the FTC—proposed a draft rule that would allow Australian media outlets to bargain with Google and Facebook to get paid for their news content. Google responded by launching a campaign targeting consumers who visited Google or YouTube from inside Australia that popped up warning labels about the proposed law and the effects it could have on users.

On October 1, Google seemingly tried to mollify regulators worldwide with a $1 billion investment in a “news showcase” that compensates outlets for their content. The tool is debuting in Germany, Brazil, Argentina, Canada, the United Kingdom, and Australia—all places where regulators have tried to force Google to pay for content before.

The Republican-majority Federal Communications Commission took another vote against net neutrality rules today in its last meeting before a presidential election that could swing the FCC back to the Democratic Party.

Today’s vote came a year after a federal appeals court upheld FCC Chairman Ajit Pai’s repeal of net neutrality rules and deregulation of the broadband industry. Though Pai was mostly victorious in the case, the judges remanded portions of the repeal back to the FCC because the commission “failed to examine the implications of its decisions for public safety,” failed to “sufficiently explain what reclassification [of ISPs] will mean for regulation of pole attachments,” and did not address concerns about the effect deregulation would have on the FCC’s Lifeline program, which subsidizes phone and Internet access for low-income Americans.

The FCC approved its response to the court’s remand instructions in a 3-2 vote today but didn’t make any significant changes. “After thoroughly reviewing the record compiled in response to its request for additional comment on these issues, the FCC found no basis to alter the FCC’s conclusions in the Restoring Internet Freedom Order,” the commission said in its announcement. “The Order on Remand finds that the Restoring Internet Freedom Order promotes public safety, facilitates broadband infrastructure deployment by Internet service providers, and allows the FCC to continue to provide Lifeline support for broadband Internet access service.” A draft version of the decision is available here.

Democrats dissent, predict court failure for Pai

FCC Democrats Jessica Rosenworcel and Geoffrey Starks dissented today, saying the FCC majority didn’t provide a good enough justification for sticking with the repeal as is. Rosenworcel called the order “a set of three cobbled-together arguments designed to tell the court to go away, the public that we are not interested in their opinion, and history that we lack the humility to admit our mistake.” Starks said he doubts that the FCC’s response to the remand will satisfy judges at the US Court of Appeals for the District of Columbia Circuit, which could take up the case again if the FCC faces another lawsuit.

The FCC majority claimed that “regulatory certainty,” meaning deregulation, results in increased innovation and investment that “will enhance public safety.” The FCC also said the repeal “is unlikely to harm public safety communications” and that any impact “would be limited because broadband Internet access service, while important, is only a part of the broader public safety communications ecosystem.” While the FCC gave up its Title II authority over broadband providers, the commission claims that antitrust and consumer-protection laws enforced by other regulators will ensure that public safety agencies have appropriate access to communications services.

That wasn’t good enough for Commissioner Rosenworcel. She said:

The very first sentence of the Communications Act tasks the FCC with “promoting safety of life and property.” In other words, public safety is fundamental to our mission. But the agency disregards it here and sidesteps the concerns of the court by insisting that removing net neutrality increases network investment, which will accrue to the benefit of public safety. The evidence for this is less than clear. But more importantly, it doesn’t adequately explain why this is the case when lives are on the line. Nor does it detail in any meaningful way how first responders will manage when emergency communications are throttled or blocked. This concern is not just theoretical. Among those opposing the FCC’s rollback of net neutrality are firefighters who found their service throttled when they were responding to a raging blaze. But here their fears are given short shrift. The agency simply concludes that the elimination of net neutrality is worth the risk, even when lives are at stake. This is irresponsible.

Pai argued that “there is no evidence the Restoring Internet Freedom order has harmed public safety. By employing a light-touch, market-driven approach to regulation, broadband providers are better able to build stronger and more resilient networks that enhance public safety, including through services like next-generation 911.”

If Democratic nominee Joe Biden defeats President Trump in the presidential election, he’ll be able to appoint a chair to replace Pai and form a Democratic majority that could reinstate net neutrality rules and Title II regulation of broadband providers.

ISPs lost legal protections

The FCC’s reclassification of broadband, changing it from a common-carrier telecommunications service to an information service, removed some legal protections for Internet providers. Starks said this is a problem for broadband-only providers that need access to utility poles to expand their networks, and he added that the FCC remand ignores this:

The Remand Order acknowledges that access to these poles is a “competitive bottleneck,” and observes that cable operators, wireless Internet service providers and others have filled the record with stories about the difficulties in obtaining reasonable access to poles. Nevertheless, the Remand Order finds that reclassification does not significantly limit new entrants to the marketplace, and in an exercise in circular reasoning, simply restates the RIF [Restoring Internet Freedom] Order’s claim that most ISPs will remain entitled to FCC protection because their broadband service will come bundled with Title II or cable services. As for broadband-only ISPs, the Remand Order blithely suggests that they could receive FCC protection if they simply began providing video or telecom services, notwithstanding their own business plans or financial circumstances… I fail to see how such a response will satisfy the DC Circuit.

There’s a similar problem for companies that provide broadband service to poor people with subsidies from the FCC’s Lifeline program, Starks said. “In response to the DC Circuit, the majority engages in a strained legal reading to find that a provider may continue to receive Lifeline support for broadband service as long as that provider remains an Eligible Telecommunications Carrier offering telecom services to some customers,” Starks said. “This would be tricky on its own terms, but voice service in the Lifeline program will be phased out next year. What will happen to Lifeline providers who may not have any remaining voice customers after the phase-out?”

The FCC’s majority decision conceded that Lifeline support for broadband could be threatened, though it called such an outcome unlikely and said that “the benefits of reclassification would outweigh the removal of broadband Internet access service from the Lifeline program.”

Pai today pointed out that, when the Obama-era FCC added broadband to the Lifeline program, broadband was not yet classified as a Title II common-carrier service. “It is the common-carrier status of the provider, not the service, that governs whether the provider is eligible to receive Lifeline support for services provided over its network,” Pai said. “If a common carrier offers voice service and qualifies as an ETC [eligible telecommunications provider], the Lifeline program can support affordable broadband Internet access and service.”

Fight to protect net neutrality “will continue”

Today’s remand response did not address preemption of state net neutrality laws. The FCC tried to preempt all such laws, but last year’s court decision said the commission cannot do that. An ongoing court case will decide whether California’s net neutrality law can be enforced.

The remand order could be met with lawsuits from some of the same advocacy groups, state attorneys general, and tech companies that previously challenged the repeal. “This remand order callously dismisses the valid concerns of public safety officials, competitive broadband providers and millions of disconnected low-income families who can’t afford to get online,” Free Press Policy Manager Dana Floberg said. “But Pai goes even further, insisting that if the agency’s decisions on these issues harm these constituencies, that harm is justified by the supposed benefits of repealing Title II.”

Free Press, one of the FCC’s opponents in the net neutrality court case, criticized Pai for claiming that the repeal spurred new broadband development. “Approximately 92 percent of the fiber deployments made during Pai’s chairmanship were actually planned and announced during the last few years of the Obama administration, when Title II was securely in place,” Floberg said. “Chairman Pai is trading away critical public protections for a bag of magic beans, and a wink and a nod from cable lobbyists. We need public servants who will actually listen to people, consider the data and serve community needs instead of long-debunked ideologies.”

Public Knowledge, another advocacy group that fought the repeal, said the fight will continue. “Ironically, the FCC is now claiming sweeping authority over social media and other online platforms without a legitimate legal basis—after claiming it has no authority to regulate broadband, which clearly falls under the agency’s traditional telecommunications jurisdiction,” Public Knowledge Legal Director John Bergmayer said, referring to Pai helping Trump impose a crackdown on Twitter and Facebook.  “Nevertheless, the fight to protect net neutrality and promote good broadband policy is not over, and Public Knowledge will continue to engage in that fight.”

Link previews are a ubiquitous feature found in just about every chat and messaging app, and with good reason. They make online conversations easier by providing images and text associated with the file that’s being linked.

Unfortunately, they can also leak our sensitive data, consume our limited bandwidth, drain our batteries, and, in one case, expose links in chats that are supposed to be end-to-end encrypted. Among the worst offenders, according to research published on Monday, were messengers from Facebook, Instagram, LinkedIn, and Line. More about that shortly. First a brief discussion of previews.

When a sender includes a link in a message, the app will display the conversation along with text (usually a headline) and images that accompany the link. It usually looks something like this:

For this to happen, the app itself—or a proxy designated by the app—has to visit the link, open the file there, and survey what’s in it. This can open users to attacks. The most severe are those that can download malware. Other forms of malice might be forcing an app to download files so big they cause the app to crash, drain batteries, or consume limited amounts of bandwidth. And in the event the link leads to private materials—say, a tax return posted to a private OneDrive or DropBox account—the app server has an opportunity to view and store it indefinitely.

The researchers behind Monday’s report, Talal Haj Bakry and Tommy Mysk, found that Facebook Messenger and Instagram were the worst offenders. As the chart below shows, both apps download and copy a linked file in its entirety—even if it’s gigabytes in size. Again, this may be a concern if the file is something the users want to keep private.

It’s also problematic because the apps can consume vast amounts of bandwidth and battery reserves. Both apps also run any JavaScript contained in the link. That’s a problem because users have no way of vetting the security of JavaScript and can’t expect messengers to have the same exploit protections modern browsers have.

Haj Bakry and Mysk reported their findings to Facebook, and the company said that both apps work as intended. LinkedIn performed only slightly better. Its only difference was that, rather than copying files of any size, it copied only the first 50 megabytes.

Meanwhile, when the Line app opens an encrypted message and finds a link, it appears to send the link to the Line server to generate a preview. “We believe that this defeats the purpose of end-to-end encryption, since LINE servers know all about the links that are being sent through the app, and who’s sharing which links to whom,” Haj Bakry and Mysk wrote.

Discord, Google Hangouts, Slack, Twitter, and Zoom also copy files, but they cap the amount of data at anywhere from 15MB to 50MB. The chart below provides a comparison of each app in the study.

All in all, the study is good news because it shows that most messaging apps are doing things right. For instance, Signal, Threema, TikTok, and WeChat all give the users the option of receiving no link preview. For truly sensitive messages and users who want as much privacy as possible, this is the best setting. Even when previews are provided, these apps are using relatively safe means to render them.

Still, Monday’s post is a good reminder that private messages aren’t always, well, private.

“Whenever you’re building a new feature, always keep in mind what sort of privacy and security implications it may have, especially if this feature is going to be used by thousands or even millions of people around the world,” the researchers wrote. “Link previews are a nice feature that users generally benefit from, but here we’ve showcased the wide range of problems this feature can have when privacy and security concerns aren’t carefully considered.”

A popular tool used for archiving YouTube videos, YouTube-dl, is gone from GitHub after the Recording Industry Association of America filed a claim arguing that the code is inherently illegal under copyright law.

GitHub, which is owned by Microsoft, removed 18 projects on Friday that previously hosted versions of YouTube-dl, a Python library that allows for the downloading of YouTube video and audio files. Those repositories now display a message reading, “This repository is currently disabled due to a DMCA takedown notice. We have disabled public access to the repository.”

Although the notice is framed as a DMCA issue, the takedown notice from the RIAA, dated Friday, does not make claim that YouTube-dl is an act of copyright infringement. Instead, it alleges that the code itself is a violation of a different section of Us copyright law (as well as German copyright law), because the “clear purpose of this source code is to… circumvent the technological protection measures used by authorized streaming services such as YouTube, and [to] reproduce and distribute music videos and sound recordings owned by our member companies without authorization for such use.”

The RIAA notice specifically cites three examples provided in YouTube-dl documentation and source code that make reference to copyrighted music videos by Icona Pop, Justin Timberlake, and Taylor Swift—all of whom are represented by RIAA-member labels—to back up the claim.

The memory hole

The RIAA is not wrong that YouTube-dl exists as a technological workaround for streaming-only video, and it would be a lie to claim that nobody uses it for helping themselves less-than-lawfully to music tracks—people absolutely do. But YouTube-dl is not solely designed for that purpose, as the RIAA alleges. Activists, historians, and content creators widely use the tool for valid archival purposes as well.

Parker Higgins, with the Freedom of the Press Foundation, dug into the takedown claim in a twitter thread. YouTube-dl is “super popular among archivists and has incredibly broad fair-use applications,” Higgins wrote. “The RIAA stance here is pretty aggressive and out there.”

Higgins added, “The ‘activity’ that RIAA is claiming is unauthorized is hypothetical. And if a rightsholder were to authorize copying,” such as by distributing their content under a creative commons license that allows it, “that copying would (obviously) be authorized.”

The legal approach the RIAA took, however, does not necessarily have to demonstrate that YouTube-dl is an infringement instrument. Under the slice of copyright law known as Section 1201, creating any kind of end-run around “a technological measure that effectively controls access to a work,” i.e., DRM, is itself unlawful. The Copyright Office does a review of section 1201 every three years to update the list of which kind of DRM circumvention is allowable or not.

The Electronic Frontier Foundation filed a lawsuit in 2016 arguing that section 1201 is unconstitutional; that case is still winding its way through the courts.

The EFF called the RIAA’s notice a “disappointing and counterproductive move,” a sentiment echoed by activists who work in the fields of civil rights, human rights, and hate speech.

“Hang back for a second so I can show you the course,” Dean DiGiacomo says over the radio as we approach the skid pad in a pair of 610hp (455kW), Skittle-colored Huracáns.

A professional racer and the chief instructor for Lamborghini’s various performance schools—which range from customer track days to an intensive training program for the automaker’s Super Trofeo wheel-to-wheel racing series—DiGiacomo takes a moment to explain the vehicle settings I’ll need to select before he sets off on a demonstration pass.

The matte-purple machine arcs gracefully from one cone of the figure eight to the next, V10 wailing as it turns rubber into smoke. Before I know it, DiGiacomo is already back in the pit area and it’s my turn to give it a go. “Now, do it just like that,” a photographer says to me with a knowing grin. We share a laugh. But how hard can it be, right?

Although the Huracán has become the standard-bearer for Lamborghini performance and is the best-selling sports car in the company’s history, there were some who complained that it was a bit too buttoned-down when it debuted back in 2014. Some of that character could be attributed in to its electronically controlled all-wheel-drive system, which was not only designed to provide maximum traction during acceleration, but also predictable, drama-free handling at the limit.

But demands for more rowdiness soon became impossible to ignore, so in 2016, Lamborghini introduced the rear-wheel-drive LP 580-2. Though slightly down on power versus its all-wheel-drive sibling, the LP 580-2 was proclaimed by many to be the more earnest driver’s car of the two due to its lower curb weight and willingness to wag the tail around corners. Yet it too was soon superseded by the Huracán Performante, a no-holds-barred track-tuned machine that took the Nürburgring lap record for production cars in 2017 by besting the Porsche 918 Spyder hypercar on the demanding German circuit.

Noble lineage

In many ways, the Huracán Evo RWD can be considered the offspring of the LP 580-2 and Performante, taking the former’s rear-drive approach to performance and blending it with a number of the upgrades that were introduced in the latter. Output is up to 610hp (455kW) thanks to a Performante-derived iteration of the naturally aspirated, 5.2-liter V10 and its raucous center-mounted exhaust system. Meanwhile, removing the front differential hardware takes roughly 75lbs (34kg) off of the front end of the car versus a standard Huracán Evo.

There’s also 73-percent more downforce on tap here versus an LP 580-2, and although the Huracán Evo RWD lacks the standard Evo’s sophisticated LDVI stability-control system, Lamborghini developed a P-TCS (Performance Traction Control System) specifically for the RWD. The P-TCS was made with the express intent of not only maximizing the car’s performance potential, but also its fun factor.

DiGiacomo tells me to put the car in Sport mode for the first pass on the skid pad. Lamborghini considers this the fun driving mode in a track setting, as it is the only option that enables the car’s “constant throttle oversteer” technology. The feature is designed to ostensibly modulate the driver’s throttle inputs to allow the car to step out and hold a powerslide when asked to do so. It also, however, dials things back a bit to prevent a spin, if needed, while the driver keeps the go pedal buried.

On the other hand, I soon learned that it’s not a “drift mode” in the traditional sense. Sport will immediately bring the hammer down on purposeful shenanigans at lower speeds, resulting in nothing but understeer on the skid pad as the system does everything it can to bring the car back in line. But DiGiacomo also has a cheat code to share.

“Put it in Corsa mode and turn off ESC on the center console,” he says. “And once you’re moving, put it in second gear, turn in and give it some throttle.”

Off the leash

With the electronic assistants fully disabled, as they are in this setting, it doesn’t take long to understand why Lamborghini is hesitant to let drivers off the leash. This ain’t no muscle car: torque ramps up from “hmm, nothing’s happening” to “OK, we’re spinning now” in literally a fraction of a second. Without the nannies there to keep everything in check, it’s just your right foot and your flailing Kermit arms keeping this Italian brute on the intended path. It’s definitely doable, but you’d best bring your A-game for this one.

Earlier in the day, we’d spent some quality time flogging the Huracán Evo RWD around the big track at Willow Springs International Raceway, and the story was a bit different out on a proper road course. At higher speeds the system can take into account the car’s speed, pitch, yaw, and driver inputs more effectively, and it’s here where Sport mode loosens the reins, allowing the Huracán Evo RWD to dance under supervision.

Turn 3 is an uphill left with a slightly decreasing radius and banking that falls away just past the apex of the corner. It’s all too easy to turn in prematurely for this one, setting up a perfect storm for oversteer while you attempt to wrangle the car back onto the racing line as power goes up and mechanical grip goes down.

After a few recon laps and the blessing of the instructor in the lead car, I switch from Strada—the car’s most electronically guarded mode—over to Sport, and we increase the pace in turn. Against my better judgment I decide to tease fate and give it a bit more throttle than I normally would going up the hill. The back steps out predictably and stays there for as long as my courage will allow, and the car quickly collects itself as I roll off the throttle. I can’t help but smile wide.

Switching up to Corsa puts the Evo RWD in its most track-focused setting, but the goal is clean, fast laps rather than photoshoot heroics, so the system is always working to put down the power as effectively as possible rather than allowing it to go up in smoke. Lamborghini touts the new P-TCS system as providing a 30-percent improvement in “torque intervention smoothness” versus the LP 580-2. While that’s difficult to quantify at speed, the fact that I was never distracted by the system working to keep the car going where I pointed it speaks volumes.

After a few more passes around the skid pad (and nearly an equal amount of spins), DiGiacomo buzzes me over the radio to come into the pits. But I’m already staging for another pass.

“Oh—you want to go again?” he says.

You’re damn right I do.

Listing image by Lamborghini

Russian state nationals accused of wielding life-threatening malware specifically designed to tamper with critical safety mechanisms at a petrochemical plant are now under sanction by the US Treasury Department.

The attack drew considerable concern because it’s the first known time hackers have used malware designed to cause death or injury, a prospect that may have actually happened had it not been for a lucky series of events. The hackers—who have been linked to a Moscow-based research lab owned by the Russian government—have also targeted a second facility and been caught scanning US power grids.

Now the Treasury Department is sanctioning the group, which is known as the State Research Center of the Russian Federation FGUP Central Scientific Research Institute of Chemistry and Mechanics or its Russian abbreviation TsNIIKhM. Under a provision in the Countering America’s Adversaries Through Sanctions Act, or CAATSA, the US is designating the center for “knowingly engaging in significant activities undermining cybersecurity against any person, including a democratic institution, or government on behalf of the Government of the Russian Federation.”

Dangerous cyber activities

“The Russian Government continues to engage in dangerous cyber activities aimed at the United States and our allies,” said Treasury Secretary Steven T. Mnuchin, in a release published on Friday. “This Administration will continue to aggressively defend the critical infrastructure of the United States from anyone attempting to disrupt it.”

Under the sanctions, all property of TsNIIKhM that is or has come within the possession of a US person is blocked, and US persons are generally prohibited from engaging in transactions with anyone in the group. What’s more, any legal entity that’s 50-percent or more owned by one of the center members is also blocked. Some non-US persons who engage in transactions with TsNIIKhM may be subject to sanctions.

The malware used in the petrochemical manufacturer attack generated so much concern because it zeroed in on processes known as the safety instrumented systems. An SIS is a combination of hardware and software that critical infrastructure sites use to prevent unsafe conditions from arising. When gas fuel pressures or reactor temperatures rise to potentially unsafe thresholds, for instance, an SIS will automatically close valves or initiate cooling processes to prevent health- or life-threatening accidents. The malware is generally known as either Triton or Trisis because it targeted the Triconex product line made by Schneider Electric.

Triton came to be installed through a phishing message that targeted someone working at the petrochemical maker. Once the attackers burrowed further and installed their malware in the operation technology part of the facility, they attempted to manipulate the industrial controllers. An error caused facility equipment to automatically shut down, an event that prevented Triton from executing fully.

Had it not been for the accident, Triton could have caused loss of life, injuries, and mass property damage. The failure not only prevented those outcomes; it also allowed researchers to recover the malware and, ultimately, the research lab that designed and operated it.

Security firm FireEye Mandiant was the first to disclose the malware and its origins at TsNIIKhM. John Hultquist, senior director of intelligence analysis at Mandiant, applauded Friday’s sanctions.

“The government confirming the attribution is absolutely important,” he said in an interview. “Given the danger of the tools these guys are using, tying their hands is an excellent outcome, even if it’s somewhat limited.”

The Federal Communications Commission has finished investigating T-Mobile for a network outage that Chairman Ajit Pai called “unacceptable.” But instead of punishing the mobile carrier, the FCC is merely issuing a public notice to “remind” phone companies of “industry-accepted best practices” that could have prevented the T-Mobile outage.

After the 12-hour nationwide outage on June 15 disrupted texting and calling services, including 911 emergency calls, Pai wrote that “the T-Mobile network outage is unacceptable” and that “the FCC is launching an investigation. We’re demanding answers—and so are American consumers.”

Pai has a history of talking tough with carriers and not following up with punishments that might have a greater deterrence effect than sternly worded warnings. That appears to be what happened again yesterday when the FCC announced the findings from its investigation into T-Mobile. Pai said that “T-Mobile’s outage was a failure” because the carrier didn’t follow best practices that could have prevented or minimized it, but he announced no punishment. The matter appears to be closed based on yesterday’s announcement, but we contacted Chairman Pai’s office today to ask if any punishment of T-Mobile is forthcoming. We’ll update this article if we get a response.

FCC details T-Mobile mistakes

The staff-investigation report identified several mistakes made by T-Mobile during the outage, which began as T-Mobile was installing new routers in the Southeast US. When a fiber transport link in the region failed, T-Mobile’s network should have transferred traffic across a different link. But the carrier “had misconfigured the weight of the links to one of its routers,” which “prevented the traffic from flowing to the new active router as intended.” T-Mobile hadn’t implemented any fail-safe process to prevent the misconfiguration or to alert network engineers to the problem.

The Atlanta market “became isolated” from the rest of the network, causing all LTE users in the area to lose connectivity. A software error made things worse by preventing mobile devices in the Atlanta area from re-registering with the IP Multimedia Subsystem over Wi-Fi. Instead of routing device-registration attempts to a different node, “the registration system repeatedly routed re-registration attempts for each mobile device to the last node retained in its records, which was unavailable due to the market isolation.”

The software error had existed in T-Mobile’s network for months. “This software error likely did not cause problems before this outage occurred because the outage was the first notable market isolation since T-Mobile integrated this software into its network,” the FCC said. Regular testing “could have discovered the software flaw and routing misconfiguration before they could impact live calls,” the FCC also said.

After the trouble on June 15 began, T-Mobile engineers “ended up exacerbating [the outage’s] impact because they misdiagnosed the problem.” The FCC report continued:

T-Mobile believed that the fiber transport link that failed earlier in the day was continuing to cause the ongoing outage. Acting on this belief, T-Mobile manually shut down the link in an attempt to transfer traffic away from it. Due to the still-misconfigured Open Shortest Path First weights, however, these steps recreated the outage’s initial conditions. LTE customers in the Atlanta market were again disconnected from the LTE network and forced to establish calls over Wi-Fi, and their registration attempts again failed and created a registration storm that added further congestion to T-Mobile’s IP Multimedia Subsystem.

T-Mobile engineers almost immediately recognized that they had misdiagnosed the problem. However, they were unable to resolve the issue by restoring the link because the network management tools required to do so remotely relied on the same paths they had just disabled. When T-Mobile engineers were able to access the equipment on site and correct their mistake by restoring the link an hour later, customers in the Atlanta market were again able to attempt to register to VoLTE [Voice over LTE]. However, this again created additional congestion because T-Mobile engineers had not yet addressed the software error that prevented registrations from completing.

Outage goes nationwide

The FCC report explained how the outage spread from the Atlanta market, going nationwide. External traffic destined for the Atlanta system was redirected to other regions, which “created enough congestion in those registration systems to cause the T-Mobile network to send the registration attempts to other nodes. The software error again routed re-registration attempts to the last node on record, which was likely already experiencing severe congestion.” Shortly after, “IP Multimedia Subsystem, VoLTE, and Voice over Wi-Fi registrations began to fail nationwide.”

The vast majority of T-Mobile customers were unable to connect to Voice over LTE or Voice over Wi-Fi networks and thus “fell back to T-Mobile’s 3G and 2G circuit-switched networks to make and receive calls while the device continued its registration attempts to the VoLTE network.” This resulted in 3G and 2G congestion, causing many phone calls to fail. Network nodes continued to hold resources for these call sessions after the calls terminated, overwhelming the nodes’ computing resources and causing even more call failures.

911 calls can typically be made even when mobile devices can’t complete registration with the IP Multimedia Subsystem, but in this case, 911 was affected by the 3G and 2G network congestion “because the same network nodes that choose gateways for calls destined for 2G and 2G networks also choose gateways for 911 calls. When those nodes’ computing resources became overwhelmed by abandoned call sessions’ resource reservations, it also caused many 911 calls to fail,” the FCC said.

T-Mobile told the FCC that 23,621 calls to 911 didn’t reach public safety answering points [PSAPs] due to congestion during the outage. Another 111,253 emergency calls were successfully completed. Including both 911 and non-emergency calls, at least 41 percent of calls on T-Mobile’s network failed during the outage, the FCC said.

This could have been avoided or minimized if T-Mobile had implemented “reasonable 911 network monitoring,” which “would have revealed to T-Mobile in real time that the outage was causing call blocking on PSAP administrative lines,” the FCC said.

T-Mobile has since corrected technical problems identified due to the outage and made other changes to prevent or reduce the severity of future outages, the commission report said.

Hey T-Mobile—please don’t do that again

In a press release yesterday, Pai again criticized T-Mobile. “T-Mobile’s outage was a failure,” Pai said. “Our staff investigation found that the company did not follow several established network reliability best practices that could have either prevented the outage or at least mitigated its impact. All telecommunications providers must ensure they are adhering to relevant industry best practices, and I encourage network reliability standards bodies to apply their expertise to the issues identified in this report for further study.”

Despite that, Pai announced no punishment.

“In keeping with past practice, the [FCC’s Public Safety and Homeland Security] Bureau plans to release a Public Notice, based on its analysis of this and other recent outages, reminding companies of industry-accepted best practices, including those recommended by the FCC’s Communications Security, Reliability, and Interoperability Council, and their importance,” the FCC said. “In addition, the Bureau will contact major transport providers to discuss their network practices and offer assistance to smaller providers to help ensure that our nation’s communications networks remain robust, reliable, and resilient.”

This is similar to what happened last year when an FCC investigation into mobile carriers’ response to Hurricane Michael in Florida found that carriers failed to follow their own voluntary roaming commitments, unnecessarily prolonging outages. Pai called the carriers’ responses to the hurricane “completely unacceptable” but imposed no punishment related to the bad hurricane response and continued to rely on voluntary measures to prevent recurrences.

Pai’s FCC also let Verizon, T-Mobile, and US Cellular off without any punishment after finding they exaggerated their 4G coverage in official filings. Pai has proposed fines for AT&T, Verizon, T-Mobile, and Sprint to punish the carriers’ illegal sales of phone-location data, but the penalties of $12 million to $91 million per carrier were criticized by Democrats as not big enough relative to the harm to consumers.