At least 26 of the top 100 most popular websites – and 242 of the top 1,000 –  are now blocking GPTBot,  the web crawler OpenAI introduced Aug. 7, according to an updated analysis.

Why we care. To block or not to block ChatGPT? That has been a big question for many SEOs because ChatGPT does not cite or link to its sources. We have let search engines crawl our content because there is a clear potential benefit – we get traffic through direct links/citations. Clearly, even more of the most popular websites have decided to block GPTBot, presumably because they don’t want OpenAI scraping their data to help train its models – at least not without some form of compensation.

12 popular websites now blocking GPTBot. Among the new additions from the top 100 most popular sites in the past month, the majority of which publish news and information:

• pinterest.com
• indeed.com
• theguardian.com
• sciencedirect.com
• usatoday.com
• stackexchange.com
• alamy.com
• webmd.com
• dictionary.com
• washingtonpost.com
• npr.org
• cbsnews.com

One big reversal. Interestingly, Foursquare, which was blocking GPTBot last month, no longer is. 

What about CCbot? Common Crawl’s web crawler is still blocked less – by just 130 websites. As a reminder, Common Crawl provides part of the training data used by OpenAI, Google and others. 

• 109 of the top 1,000 websites block both GPTBot and CCbot.

Limitations. 67 robots.txt files out of the 1,000 websites were not identified/inspected as part of this analysis. (That’s why I wrote “at least” in the opening sentence.)

Originality.ai’s updated analysis. Websites That Have Blocked OpenAI’s GPTBot – 1000 Website Study

Dig deeper. Should you block ChatGPT’s web browser plugin from accessing your website?

The search landscape is expanding rapidly. People increasingly turn to various platforms for their search needs instead of relying solely on traditional Google searches.

In this evolving landscape, a brand’s visibility is crucial, as consumers are bombarded with messaging at every scroll, click, and swipe.

With recent developments like Search Generative Experience and Google’s Perspectives feeds, dominating the SERPs is a top priority for SEO professionals.

From Google’s familiar SERPs to the dynamic realms of TikTok and Instagram reels, each platform offers a unique opportunity for brands to share their stories.

However, just having a presence isn’t enough. It’s about crafting compelling, optimized, relevant content that resonates with your audience, aligns with your search strategies, and serves your business goals.

This article will explore how your brand can conquer the search universe by creating authentic, engaging, and valuable content that attracts and retains consumers’ attention.

1. Hone in on experience, expertise, authority and trustworthiness (E-E-A-T)

There can be as many discussions around emerging platforms for search as you like, but Google is still the head honcho when it comes to search.

Because of this, sending the correct signals to Google within your content remains of the highest importance. It just so happens that their E-E-A-T guidelines also reflect users’ needs from content, so they’re a great starting point.

E-E-A-T must be at the core of content strategy development. When choosing content topics, always assess whether your brand has direct experience or unique expertise. This ensures you produce content to inspire, educate, or entertain users.

Conducting gap analysis and keyword research helps you see how well-positioned your brand is to send these signals. From here, you’ll begin to:

• Understand where your brand lacks first-hand experience and where to fill an expertise gap.
• Strategize ways to develop trustworthiness with an audience and establish topical authority.

At this point, you can look to build relationships with content creators, conduct an internal search for topic experts, or even connect with subject matter experts who can fill the gaps.

While E-E-A-T is a Google consideration, it also reflects the thought process of users and audiences when they are searching.

Leveraging this step as almost a “right-to-rank” activity is a foundational stage for any content strategy. 

2. Understand what content needs to be created and where to place it

Once you have established which topics you feel you are well-positioned to discuss, think of how to effectively discuss them.

This used to be relatively easy as the default was to create long-form, text-based content, typically as a blog or guide. 

However, in 2023, your users are engaging with multiple content types, meaning you need to consider how to effectively satisfy the intent behind the search query.

The goal here is to understand what signals your users are giving us.

You will likely lean into video content if you recognize that content around a query is popular on YouTube and TikTok. Similarly, if you notice a trend on Pinterest, you will explore image content as a solution.

As Google Perspectives enters the conversation, I anticipate this step getting simpler, as you can leverage Google data to understand the types of queries that carry ample search volume. 

After establishing what content users leverage Perspectives for, you can inspect content types and forms in further detail.

In the short term? Leverage tools like Glimpse – a Chrome extension that provides wider knowledge around trending topics – on Google Trends.

Look at the signals in the tool’s “Channels” feed.  This lets you understand users’ expectations around content solutions for given keywords, hashtags and ideas.

This step has become crucial in my content strategy for brands. In the past, I primarily analyzed Google SERP features, People also ask questions, and competitors’ content. 

Today, understanding where queries are most prominent lets me determine where content needs to be placed.

This step allows us to identify video content opportunities, creator-led considerations or reaffirm that Google is the go-to destination.

Overall, this approach ensures that your strategy reflects your users instead of shoe-horning content solutions around a Google-only approach. 

3. Platform-specific content customization

In the evolving search universe, different content platforms act like distinct planets – each with its own personality, audience, and algorithms. 

For brands aiming to succeed and connect with their audience on these platforms, merely having a presence isn’t sufficient. 

The content created must feel native to each platform, underscoring the importance of platform-specific content customization. 

For example, if users are searching on TikTok, your content should align with the platform’s style (i.e., vertical, short-form videos) to resonate with the audience effectively.

Let’s explore how to ensure you’re working toward platform-specific content creation and customization. 

Native content creation

Platforms are not mere distribution channels; they are unique stages. 

Content shouldn’t be created once and distributed everywhere. Instead, it should be crafted or tailored to align with the platform’s nature. 

Think of it like translating a message into different languages. The core message remains the same, but the delivery varies to cater to the specific audience and the solutions they crave on a specific platform.

Understand platform nuances

Each platform has its own set of rules and preferences. 

From content duration and format to engagement metrics and peak activity times, myriads of factors can influence your content’s success. 

Recognizing and respecting these nuances ensures that your content feels organic and relevant.

Audience expectations

Users approach platforms with different mindsets. For instance, what a user seeks on a video-centric platform might differ from a community-driven forum. 

Brands should be attuned to these expectations, ensuring content aligns with the audience’s intent and platform experience. 

When brands do not achieve this, this can be a real turn-off for users.

Consistency in diversity

While the medium and method might change, your brand voice and core message should remain consistent. 

This strengthens brand identity and ensures that while the delivery might vary, the essence is recognizable across platforms.

Stay updated

The search universe is ever-evolving. Platform preferences shift, algorithms get updated, and user behaviors most certainly change. 

To truly dominate the content game, brands must stay informed and be ready to adapt their strategies based on platform-specific insights.

4. Future-proof your strategies

A successful content strategy isn’t limited to the present but looks ahead to the future. 

By staying open to learning and ongoing content improvement, brands stay current with changing trends and tools, avoiding being stuck in outdated practices.

But it’s not only about knowing; it’s also about putting that knowledge into action.

The key lies in cultivating a strategy that is rooted yet flexible. As new platforms emerge and user preferences shift, brands should be agile enough to pivot without reinventing the wheel.

This agility is complemented by an eagerness to harness the latest technological advancements, whether that’s AI-driven analytics or immersive VR experiences, Google Perspectives or SGE.

However, even with these shifts, it’s crucial to prioritize the audience’s perspective.

Stay tuned with evolving search behaviors and consistently review analytics and feedback to gain valuable insights about current sentiments and future expectations.

While the future may be uncertain, you can be ready with scenario planning. Envision different paths for the search landscape and strategize how to position your brand in this evolving environment.

In doing so, you build a content strategy responsive to today’s digital rhythm and ready to dance to tomorrow’s tune.

5. Creator-led SEO: Foster collaborations and partnerships 

“If content is king, experience is queen,” as I highlighted in a Search Engine Land article, “Why creator-led content marketing is the future of search.” 

Partnerships and collaborations have emerged as powerful tools to amplify reach, diversify content offerings, and tap into new audiences – all while showcasing experience.

More than just a means to pool resources, strategic collaborations can:

• Bring a fresh perspective.
• Add credibility.
• Build trust among audiences

All this is vital for the other considerations we have covered above.

By aligning with influencers who share similar values and audiences with your brand, you can co-create content that offers diverse viewpoints while maintaining consistency in messaging.

These partnerships let you explore innovative content formats and platforms, reaching new audiences or re-engaging current ones. 

Brands can benefit from these collaborations by expanding their online presence beyond just Google search results, ensuring a broader digital dominance.

Collaborations with influencers bring a human touch to a brand’s digital presence.

Authentic endorsements or shared content from trusted figures can significantly elevate brand perception and reach, particularly among younger audiences who value peer recommendations – E-E-A-T signals sent!

Furthermore, partnerships can pave the way for cross-promotion, where both parties benefit from shared audiences, leading to mutual growth. 

This extends beyond content creation. Joint webinars, shared research reports, or collaborative events can deepen engagement and position both entities as thought leaders.

Navigating the new search universe: Enhancing your brand’s visibility beyond Google

Optimizing for search is no longer limited to traditional engines like Google. As consumers explore new platforms, we must adapt and refine our strategies. 

Prioritizing expertise, trustworthiness, and genuine connections can help you stand out and build authority.

These considerations go beyond steps. They form a holistic approach to address changing user behaviors. 

In a world where attention is precious, brands must move past visibility to connect and engage effectively. SEO in 2024 and beyond is all about building relationships.

Remember, adaptability is crucial. The digital world is ever-changing, with new platforms emerging and old ones fading.

You can navigate and excel in this vast and evolving search landscape by grounding your strategy in understanding and authenticity.

Local businesses must ensure they can be found by those looking for them – both online and on the road.

Location pages on your website are the perfect place to direct people to your place of business while giving additional information that might be helpful to them.

Location pages also signal important information about your business to search engines, which helps make your website more relevant for certain search queries.

Let’s dive into what makes a perfect location page for SEO and visitors:

• Contact information (NAP + W)
• Available services/products/features
• Map and directions
• Parking information
• Visuals
• Hours of operation
• Marketing copy
• Reviews
• More helpful information
• Page optimization
• How to handle multiple locations

Think about your target audience

Before choosing which strategies to implement in this article, think about your target audience. What information is going to be the most helpful for them?

For example, if you are a moving company, your location pages may direct people to your location and give vital information about the city you’re located in. 

After all, people who are moving in may want to know more about the local community they are entering into. 

In this scenario, you might dedicate some text to things to do in the area, the schools, the crime rate, etc.

Always start with the page’s purpose – what do you want it to do for your visitors? Then, the content strategies will follow.

If you decide not to implement every suggestion in this article, that’s OK. Do what is best for your business and your audience.

What would a location page be without an address? 

A physical address is a signal to Google for ranking, and the search engine wants to rank businesses closest to the point of search (where the searcher is).

But what is often forgotten is to make sure the name, address, phone number and website (NAP + W) have the most up-to-date information. 

(This can get trickier to keep up to date when you have multiple locations.)

And when I say up to date, I also mean that the NAP + W on the site is the leading example for the NAP + W across all other online channels like online directories, social media profiles, etc. 

Search engines like Google want to be able to trust a business, and making sure the NAP + W information is consistent everywhere is key.

Available services/products/features at the location

Letting people know what to expect at your location can be helpful.

For example:

• If you’re a coffee shop, you might include a menu.
• If you’re a storage facility, you might show the availability of storage units.
• If you’re a boutique hotel, perhaps you might show which amenities are available at that location.

The following screenshot from a location page at a storage facility business at StoragePost.com shows what types of storage are available at the specific location.

So think about what you can tell people to give them an at-a-glance view of what you offer at your location. 

This helps them know what to expect (and they don’t have to navigate to other pages to find the most important information).

A map and directions

Embed a map (preferably a Google map) that shows your location. 

A dynamic embedded Google map versus a static map makes it easy for people to click the map and get directions to your location, like how we do at BruceClay.com:

Here’s a Google help file on sharing a map to get you started.

It’s hard to imagine a world where someone doesn’t rely on GPS to get there, but just in case, you might consider giving text directions from the major directional points (from the north, south, east and west, and citing any landmarks).

Locals may be able to glance at those quickly and know where they are going without the map. And it could be helpful if the map or someone’s GPS isn’t working.

Here’s an example from StoragePost.com, where I pulled a blurb about directions to the facility:

Parking information

Nothing stresses out some people more than having a nice GPS-guided tour to your location, only to be faced with the chaos of figuring out where they can stop their car and get out. 

Giving parking information is a nice way to prepare people for what to do when they get there. 

For example:

• Do you have parking on-site, or is there only street parking? 
• Do you have to pay for parking or is it free? 
• Where do they turn to get to the parking lot? 
• What times of day are the busiest for parking? 
• Is there a parking garage nearby if need be? 

The answers to these questions can be a lifesaver for some. And for some locations with tricky parking and some city streets, this is an absolute must.

Visuals of your business

Another way to help people know what to expect is to have visuals of your business. 

You can do this by:

Posting photos of your building and the inside of your business

Make sure they are high-resolution, good-quality photos. You can use the same photos in your Google Business Profile. (Hopefully, you have one set up as a local business.) You can also optimize these images so that they have a better chance of showing up in the search results.

Embedding a 360-degree virtual tour

Give people a virtual tour of your business. Google “how to embed a Google virtual tour on your website” for more tips.     

Adding a video

Add a video that gives an overview of your business. You can create the video, upload it onto YouTube, and then embed it on the location page.

This way, it will do double duty for you. It serves as interesting content on your YouTube channel, helps your location page visitors and has the ability to rank in the search results.  

Dig deeper: 5 top tips to boost your local SEO efforts

Hours of operation

One of the most essential bits of information you will have on your location page is the hours of operation. Let people plan their trip to you.

This one is self-explanatory – be sure to include the days and times you are open. 

And if you want to add a little extra, you can say when your peak hours of operation are (as in what days and times you tend to be the busiest). Some people appreciate this so they can avoid the crowds.

Marketing copy

Why not use the location page as yet another place to differentiate your business and get people interested? 

Consider adding marketing copy that shows what makes your business great, how you are different from the competition and why people won’t be disappointed in giving you their business.

Reviews

A location page is a great place on the website to display reviews of your business. 

You have creative reign when it comes to how you want to display your reviews. You can display star ratings, or you can add testimonials.

For testimonials, consider:

• Displaying localized reviews that mention the local services, products or experiences you provide.
• Including the person’s name and perhaps things like the city, town, or neighborhood the customer resides in.
• Adding the person’s picture so you can to make it more personal.

More helpful information 

You can add value to your location page in many different ways. Here are some ideas:

Write about the community

Take some time to say a little bit about the surrounding areas. This can infuse the page with localized content, which is good for SEO but also provides value to visitors.

For instance, if you have a business that caters to customers who are potentially new to the area or looking to move to the area, you can talk about the community, what it has to offer and maybe a little bit about its history.

Here’s an example from StoragePost.com right on one of their location pages:

Transportation, accommodations, attractions

If you have a lot of out-of-town customers, you might tell them how to get to and from Point A to Point B, where they can stay the night and what they can do while there.

That’s what we do on BruceClay.com:

Helpful articles

If you have written any relevant blog articles on your site that relate to the location, then link to them from the location page.

This can additionally drive more traffic to your blog and keep visitors on your site longer.

Here’s what StoragePost.com does:

Dig deeper: How to use location-specific authority content for SEO

Page optimization 

As with any other key page on your website, you want to optimize your location page. 

There are a ton of things you can do to improve the relevancy of your page. Some of the basics include:

• Making sure the content feels as if it has been written with a certain degree of expertise or experience, depending on the topic. For example, if you’re writing about local stuff, you’d better have local experience in that place (or at least be able to research it really well).
• Ensuring the content is updated as needed to keep it evergreen.  
• Having a page that offers a good user experience, such as those factors outlined in Google’s page experience concept (mobile-friendliness, a secure site, no intrusive pop-ups, Core Web Vitals).
• Meta tags, heading tags.
• Image optimization, including alt text.
• Schema, including NAP+W schema and review schema. See schema.org for LocalBusiness (scroll down to the bottom for more specific business categories) and Review.

How to handle multiple locations

If you’re lucky enough to expand your business to multiple locations, you need a strategy for handling location pages.

For this, you must create a “Locations” section on the site where each location will have a corresponding webpage.

Here’s how StoragePost.com handles it on the homepage:

When you click through, you are brought to a main location landing page, where you can click on states or individual locations:

Choosing a Bronx location brings you to that specific location page:

You must ensure that the content is unique to each page, using some of the tips outlined earlier in this article on creating content for location pages.

Dig deeper: Optimizing local landing pages for search: 8 best practices

Boost your local visibility by nailing down these elements

While location pages might seem like one of those less-important webpages on your site, nothing could be further from the truth.

Location pages provide useful information to website visitors and customers and improve your website’s relevancy with search engines. 

How has it only been 25 years? Am I the only one who feels like there can’t have been a world without Google?! 

The search and technology behemoth has undoubtedly impacted most human experience across the globe in some form or another, but none more so than for the SEOs of the world. 

The SEO industry effectively shares its history with Google.

Once search engines like Google started to become the primary method through which people accessed information online, it became crucial for businesses to have a digital presence and be listed on search engine results pages (SERPs). 

Evolution of the algorithm: Changes in SEO focus 

Thanks to ever-evolving algorithms and expectations, search is a relentless journey of adaptation and resilience for SEOs. 

Each update feels like a new puzzle, urging the industry to delve deeper, understand better and refine strategies to ensure sites continue to shine organically.

It’s about trying to anticipate the subtleties of algorithm changes, predicting what will come next and constantly innovating to keep content relevant, user-friendly and, importantly, visible. 

Every change is a new opportunity to learn. Here are just some of the many algorithm changes that SEO has (mostly) survived, as told by industry veterans themselves. 

Penguin update

When we talk about Google shaking things up in the SEO world, the Penguin update of April 2012 is one for the books. 

It was the kind of wave that reshaped how we all thought about links and rankings and made us rethink our game plans.

Mark Williams-Cook, co-owner of search agency Candour, has some pretty vivid memories about this time. “The April 2012 Penguin v1 Update… it made such profound and lasting changes in the industry,” he said. 

Before Penguin rolled out, the more, the merrier was the mantra regarding links. It was all about piling on those links, and aggressive link building was the way to go. 

There were places where you could buy links, and services promising to boost your rankings actually worked! But then, Penguin happened. It was like a switch flipped overnight. 

Williams-Cook saw it all – big names in affiliate marketing saw their traffic go down the drain, agencies were getting the boot from clients whose rankings tanked, and suddenly, folks were charging money to remove links. Major web players like Expedia didn’t escape the penalties either.

He shares his own brush with Penguin quite candidly. “I personally had a few of my money websites tank (deservedly), and it was a great lesson in ‘it works until it doesn’t’.” 

It’s a reminder of how strategies that seem to work can suddenly backfire if they aren’t sustainable or authentic.

Medic update

Let’s hop back to August 2018, which is likely to be well remembered by many SEO enthusiasts with health-related sites – all thanks to the Medic update.

Brady Madden, founder at Evergreen Digital, had a CBD commerce website under his wing. He enjoyed the fruits of ranking well, with sales soaring over $100,000 monthly. 

Then Medic hit, and it was like the rug was pulled out from under him, leading to a loss of around $50,000 in sales in just a month.

“Since this ‘Medic’ update was so new, we didn’t know where or what to address,” Madden said 

He and his team took about six months to revamp their strategies, reassess their content, strengthen their link building campaigns and make several other crucial adjustments to bounce back.

“At the time, I had been around SEO and digital marketing for 5+ years… This whole event really got me going with SEO and has become my top marketing channel to do work for since then,” Madden shares as he reflected on the journey that this update had led him on.

Medic was termed a “broad, global, core update” by Google, with a peculiar focus on health, medical, and Your Money Your Life (YMYL) sites. 

Google’s advice was straightforward yet frustrating for many – there’s no “fix,” just continue building great content and improving user experience. 

The update was a learning curve and a revelation of SEO rankings’ dynamics and core algorithm updates’ impacts.

“I also learned that while they’re tedious to read, docs like Google’s Quality Raters Guidelines provide a wealth of knowledge for SEOs to use,” Madden said, highlighting the invaluable lessons to be learned from Google’s documentation. 

Diversity update

Olivian-Claudiu Stoica, senior SEO specialist at Wave Studio, has been in the SEO game since 2015. He fondly remembers this transformative update. 

“The Diversity update from 2019 was a nice touch from Google… This clearly helped smaller sites gain visibility,” Stoica said.

The Diversity update ensured a more equitable representation by limiting the presence of the same domain to no more than two listings per search query, striving to bring balance to search results. 

It was a shift toward a richer, more diverse search experience, offering users a wider array of perspectives and information sources.

Stoica also sheds light on the updates that profoundly impact businesses today. He highlights the Helpful Content, Spam, and Medic updates as pivotal changes defining the contemporary SEO landscape. 

“Holistic marketing, brand reputation, PR-led link building, well-researched content, customer experience, and proven expertise are now the norm to succeed in Google Search,” he said, outlining the multifaceted and often multi-disciplined approach required from modern SEOs.

Product reviews update, May 2022 core update and more

After Penguin, things didn’t get any smoother, and Ryan Jones, marketing manager at SEOTesting, has quite the tale about rolling with the punches of Google’s relentless period of updates.  

“The site hovered between 1,000 and 1,500 clicks per day between November 2021 and December 2021,” he shares.

But then, it was like a series of rapid-fire updates came crashing down: December 2021 Product Reviews Update, May 2022 Core Update, and more.

And the impact? It was like watching the rollercoaster take a steep dive. 

The traffic went from those nice highs to around 500 clicks per day.

“Between half and more than half of the traffic was gone,” Jones said.

But it wasn’t just about the numbers but about figuring out where the ship was leaking. Turns out, most of the traffic loss was from the blog. 

“Google was no longer loving the blog and was not ranking it,” Jones said. 

It was a wake-up call, leading to some tough decisions, like removing around 75 blog posts – a whopping 65% of the blog.

It wasn’t just about cutting the losses but about rebuilding and re-strategizing. Diving deep into content marketing, investing in quality writing, and focusing on truly competitive content became the new game plan.

“We also invested in ‘proper’ link building… which we believed helped also,” Jones added.

And the journey didn’t end there. The uncertainty and stress continued with the next series of updates, including the July 2022 Product Reviews Update and the October 2022 Spam Update. 

“It took a while, but we achieved a 100% recovery from this update,” Jones said.

Link spam and helpful content updates

Jonathan Boshoff, SEO manager at Digital Sisco, recounts the experience of turbulence when algorithms threw curveballs one after the other. 

“In my previous role, I ran SEO for one of Canada’s largest online payday loan companies. We got hit by three algorithm updates in a row (2022 link spam and helpful content, and one in Feb.) This was the comedown from all-time highs. The company was thrilled to be making lots of money with SEO, but the algorithm updates destroyed all our rankings. Essentially, the whole company was scrambling. Everyone became an SEO all of a sudden.”

Boshoff’s story is a snapshot of the collective experiences of the SEO community. 

The excitement of top rankings. The scramble when they plummet. And the continuous endeavor to decode and master the changing goalposts. 

“Ultimately, another algorithm update rolled out and fixed everything. Since then they have continued on to have even better results than ever before. It was a very hectic couple of months!” he shares, underscoring the resilience and adaptability that define the spirit of SEO professionals.

E-A-T (and later E-E-A-T)

When Google was still in its infancy, the core mission was clear and simple: Create the best search experience possible. 

In an era where links ruled the kingdom of page ranking, SEO enthusiasts explored every nook and cranny to master the game.

Enter E-A-T (Expertise, Authoritativeness and Trustworthiness), a concept that’s been in Google’s quality rater guidelines since 2014. 

It has been a compass guiding the journey of search engine evolution, with the goal of delivering diverse, high-quality, and helpful search results to the users.

Helene Jelenc, SEO consultant at Flow SEO, who was then exploring the realms of SEO with her travel blog, shares her journey and reflections on the inception of E-A-T. 

“When EAT was added to the guidelines, that’s when things began to ‘click’ with SEO for me,” she said. 

Jelenc, like many content creators, was striving to offer unique and original content and insights. Her determination to produce well-researched blog posts finally earned recognition and reward from Google.

E-A-T isn’t just about the visible and tangible elements like backlinks and anchor text; it’s about building genuine high-level E-A-T, a journey that requires time, effort and authenticity. 

Google values quality, relevance and originality over quick fixes and instant gains. It’s about promoting the “very best” and ensuring that they get the spotlight they deserve.

It’s important to understand that E-A-T and the Quality Rater Guidelines are not technically part of Google’s algorithms. 

However, they represent the aspirations of where Google wants the search algorithm to go. 

They don’t reveal the intricacies of how the algorithm ranks results but illuminate its aims. 

By adhering to E-A-T principles, content creators have a better chance of ranking well, and users get the right advice. 

Jelenc’s insights and experiences mirror the broader evolution in the SEO world. In an era riddled with misinformation, especially in areas impacting Your Money or Your Life (YMYL) queries, the principles of E-A-T have become more important than ever. 

“Our humanness and the experiences we have inform our content and add layers that make it unique,” she reflects. 

Jelenc sees the advancements in E-A-T and the introduction of E-E-A-T (the added “E” being Experience) as steps toward acknowledging the value of human experiences in content creation. 

With this, she’s optimistic about Google’s future directions “[I]n light of AI-generated everything, Google dropped E-E-A-T, which, in my opinion, is the single most important update in years.”

Legacy SEO strategies and tactics: It works until it doesn’t

Exact match domains

A pivotal moment in Google Search’s history was the devaluation of exact match domains (EMDs). This move reshaped domain strategies and brought a renewed focus on content relevance and quality.

Daniel Foley, SEO evangelist and director at Assertive Media, shared a memorable encounter that perfectly encapsulates the significance of this shift. 

“One of my most memorable moments in SEO was when Matt Cutts announced the devaluation of exact match domains,” he said. 

The timing of this announcement was awkward for Foley. He had just secured a number one ranking on Google for an EMD he had purchased, using “free for all” reciprocal links and a page filled with nothing but Lorem Ipsum content!

His sharing of this achievement with then-Googler Matt Cutts on Twitter didn’t earn applause but instead highlighted the loopholes the devaluation aimed to address.

“He wasn’t impressed,” Foley said, recounting how the interaction went “semi-viral at the time.”

This anecdote is more than just a glimpse into the cat-and-mouse game between SEO professionals and Google’s updates. It reflects the continual evolution of SEO strategies in response to Google’s quest for relevance and quality. 

The devaluation of EMDs was a clear message from Google – relevance and quality of content are paramount, and shortcuts to high rankings are fleeting.

Backlink profile manipulation

SEO is a wild ride. Sometimes, you can’t help but look at some tactics and just – laugh. Preeti Gupta, founder at Packted.com, has some thoughts on this, especially when it comes to old-school strategies like profile creation and social bookmarking. 

“While there are a bunch of outdated things SEOs do, the most fascinating one to me definitely has to be profile creation, social bookmarking, and things like that,” she said.

It’s a funny thing, really. Agencies and SEO folks put in so much time and effort to build these links, thinking it’s the golden ticket. 

“Agencies and SEOs waste a lot of their time and resources to build these links and I guess this clears the path for genuine sites,” Gupta said. “It’s both funny and sad at the same time.”

This became even more amusing when the December 2022 link spam update rolled out. It pretty much neutralized the impact of this type of link building. 

And the irony? The same folks offering these services were now offering to disavow them! 

Negative SEO

Let’s jump back to 2015 with Ruti Dadash, founder at Imperial Rank SEO, who was just dipping her toes into SEO at the time. 

“What does SEO stand for?” That was her, a complete newbie at a startup, a small fish in a pond teeming with gigantic, well-established competitors with bottomless marketing budgets. SEO became a secret weapon, helping her rank against the giants.

Dadash shares a vivid memory of waking up one day to find that rankings had skyrocketed. Detective work revealed a sneaky move from a rival, thousands of backlinks pointing straight to Dadash’s site. 

But this sudden gift was a Trojan horse as all these links were from sketchy gambling and adult sites, a disaster waiting to happen. 

“The scramble to put together a new disavow file, and the relief/disappointment as we lost that sudden surge…” You can imagine how this experience shaped her SEO career.

“It’s been a wild ride, from watching how Google has evolved to moving from in-house SEO to freelance, and then to founding my own agency,” Dadash said. 

This wild, unpredictable journey makes SEO such a fascinating field, a constant game of cat and mouse with Google’s ever-evolving algorithms.

Hidden text

There was a time when the “coolest” SEO trick in the book was like a magic act – now you see it, now you don’t. 

Brett Heyns, a freelance SEO, can’t help but reminisce about those days.

“All time best was white text on a white background, filling all the white space with keywords. Boggles the mind that there was a time that this used to work and was ‘standard practice’!” Heyns said.

Hidden text was the stealthy ninja of old-school SEO strategies. In the times when search engines were more like text-matching machines, SEOs could publish one content piece for site visitors and hide another for search engines. 

Yes, you heard it right! Web pages were stuffed with invisible keywords, creating long, invisible essays, all in the name of ranking.

Heyns’s memory of this practice takes us back to when consumers would see a conversion-optimized webpage, blissfully unaware of the keyword-stuffed content hidden behind the scenes. 

SEOs would ingeniously position white text on white backgrounds or even place images over the text, keeping it concealed from the human eye but visible to the search engine’s gaze.

Some even explored more sophisticated schemes, like cloaking, where scripts would identify whether a site visitor was a search engine or a human, serving different pages accordingly. It was like a clandestine operation, showing keyword-optimized pages to search engines while hiding them from users.

These techniques might sound like relics from the past. Still, they’re a reminder of the evolution of SEO strategies – from the overtly stealthy to the authentically user-centered approaches we value today. 

Content scraping and cloning

In the Wild West days of the early internet, SEO was more of an arcane art and less of a science, and it was fertile ground for some pretty wild tactics. 

Sandy Rowley, SEO professional and web designer, reminisces about a time when the rules were, well, there weren’t really any rules. 

“Back in the ’90s, an SEO expert could clone a high traffic website like CNN, and outrank every one of its pages in a week.” Yes, you heard that right – cloning entire high-traffic websites!

This practice, known as scraping, involved using automated scripts to copy all the content from a website, with intentions ranging from stealing content to completely replicating the victim’s site. 

It’s a prime example of black hat SEO tactics where the cloned sites would appear in search results instead of the original ones, exploiting Google’s ranking algorithms by sending fake organic traffic and modifying internal backlinks.

This advanced form of content theft wasn’t just about plagiarism.

It was about manipulating search engine results and siphoning off the web traffic and, by extension, the ad revenue and conversions from the original sites. 

It was a tactic that threw fairness and ethical considerations to the wind, leveraging the loopholes in search engine algorithms to gain a competitive advantage.

New features, plus ones we loved and lost

Seasoned SEOs will all remember (and shed a tear for) at least one tool or feature that has become no more than a relic. 

Equally, each will have a story about a new innovation or tactic that they’ve had to quickly comprehend. 

Let’s continue our journey through the anecdotes and rifle through Google’s figurative trash can while finding interesting tidbits about current search attributes.

Remember Orkut? 

Let’s take a nostalgic stroll down memory lane, back to the days of Orkut with Alpana Chand, a freelance SEO. 

Ah, Orkut. Remember it? It was one of the first of its kind, a social media platform even before the era of Facebook, named after its creator, Orkut Büyükkökten. 

Orkut was a hub of innovation and connection, especially popular in Brazil and India, offering unique features like customizing themes, having “crush lists,” and even rating your friends.

“I remember Orkut. I used it when I was in engineering college,” Chand said, recalling the days of shared weak Wi-Fi in dorm rooms and big, heavy IBM computers. 

Chand paints a picture of when being hooked to Orkut was the norm. “Addicted is understating our obsession,” she said. 

And let’s not forget those Yahoo forwards filled with quotable quotes shared on Orkut. Seeing your wall and the new feed popping on it was a different vibe.

“Today’s social media features change in the blink of an eye and Orkut’s slow stable feature – the wall, the scrapbook, the direct messages were etched on my mind,” Chand shares.

But, like all good things, Orkut had its sunset moment. As platforms like Facebook offered more simplicity, reliability, and innovative features like a self-updating News Feed, Orkut struggled to keep up with its increasing complexity and limited user range. 

Eventually, Google shifted its focus to newer ventures, like Google+, and in 2014, it was time to bid Orkut farewell.

“So many memories and an entire memorabilia was buried along with Orkut’s discontinuation,” she said, wishing for some saved screenshots of those times.

Hello, featured snippets!

Who doesn’t love a quick answer? That’s what Google’s featured snippets aimed to do, and have done for years now. 

Featured snippets, those handy blocks of content directly answering our queries at the top of SERPS, made their grand entrance in January 2014, much to the dismay of SEOs that relied on traffic for even the shortest answers. 

They quickly became known as Google’s “answer boxes” or “quick answers,” pulling the most relevant information from web pages and placing it right at the top of our search results.

Megan Dominion, a freelance SEO, remembers the early days of Featured Snippets and the missteps that followed. 

“I remember when the below featured snippets were being spoken about in the SEO industry as well as outside of it,” she said, reminiscing about the time Google got it hilariously wrong.

Just imagine searching for “How many legs does a horse have?” and being told they have six!

“There’s a nice spike in search trends for when it happened to commemorate the occasion and the beginning of ‘Google is not always right’,” Dominion points out.

It was a reminder that even the ‘big dogs’ can have their off days. If you fancy a bit of a giggle, check out some more examples of Google’s featured snippets getting it wrong.

Featured snippets, with their prominence and direct answers, became a coveted spot for SEOs and website owners, sparking discussions, strategies, and a slew of articles on optimizing for them. 

They were officially named “featured snippets” in 2016, distinguishing them from short answers pulled from Google’s database.

Hreflang havoc

Hreflangs, introduced in 2011, are like signposts for Google’s bots, telling them what language and region a page is intended for. 

They help avoid duplicate content issues when creating content that’s essentially the same but for different markets. 

When they work, it’s smooth sailing, but when they decide to go off-script, the outcome can be hilarious. 

Nadia Mojahed, international SEO consultant at SEO Transformer, had a front-row seat to when hreflang attributes decided to have a bit of fun.

Mohajed recalls when a frantic call revealed an ecommerce site in chaos. This site, which had been doing great globally, was suddenly offering luxury handbags like gourmet dishes. 

It’s all thanks to some cheeky hreflang tags. “Imagine Italian pages talking about pasta and French ones describing sneakers as delicacies!” she laughs.

Google turned page titles and descriptions into hilarious misinterpretations. Think “Women’s Fashion” in Spanish being read as “Exotic World of Llamas and Ponchos,” and German “Men’s Accessories” transforming into “Bratwurst-Tasting Bow Ties.”

Getting hreflang attributes right can be a bit of a puzzle, especially for mega-sites with tons of pages. A 2023 study by fellow SEO Dan Taylor even found that 31% of international sites were getting tangled up in hreflang errors.

Mojahed quickly found and fixed the issues, turning the blunder into a learning moment about the quirky challenges of SEO tech.

Next, please! The rel=prev/next drama

Remember the collective gasp in the SEO community when Google dropped the bomb about rel=prev/next for pagination? Yeah, that was a moment. 

Lidia Infante, senior SEO manager at Sanity.io, paints the picture vividly. 

“The time Google announced that they hadn’t been using rel=prev/next for a while! There was a range of reactions. From freaking out over how to handle pagination now to mourning all the developer hours we sank into managing it through rel=prev/next.”

It turns out this signal had been deprecated for some time before the SEO world was in the know. 

Infante and many others soon learned that it had other purposes beyond indexing, and well, Google isn’t the only player in the game. 

Other search engines were still using it. It was a reminder moment for many, understanding that Google was not the only search engine they were optimizing for.

This revelation came just before a BrightonSEO edition, and the atmosphere was charged when John Mueller was asked about it on stage during the keynote. “It was a whole vibe,” Infante reminisces.

Realizing the mix-up, Google apologized for not proactively communicating this change. 

They assured they’d aim for better communication about such changes in the future, stating:

“As our systems improve over time, there may be instances where specific types of markup are not as critical as it once was, and we’re committed to providing guidance when changes are made.” 

Oops! 

Google+ or else?!

You’ll be hard-pushed to find an SEO who doesn’t remember Google+ unless they’re super green. 

Reflecting on its turbulent journey, Mersudin Forbes, portfolio SEO and agency advisor, shares, “Google+ was a flash-in-the-pan, social media platform pushed on us by Google. So much so that they appended the +1 functionality in search results to make us believe that upvoting the search results increased rankings.”

Google+ emerged with a promise, a beacon of new social media interaction backed by Google’s colossal influence. 

Forbes recalls how Google firmly asserted that this feature would be a new signal in determining page relevance and ranking. “This was the start of spammageddon of +1 farming and just as quick as it arrived, it was gone,” he remarks.

The bold venture seemed promising, with innovative features like Circles and Hangouts and rapid user acquisition, reaching 90 million by the end of the launch year. 

However, the spammy notifications and the platform’s failure to maintain active user engagement led to its eventual demise.

“I wonder if the same will happen to SGE, one can only hope,” Forbes said. 

Google+ now stands as a reminder of the transient nature of tech innovations, even those stamped with the Google seal.

UA who? The reluctant adoption of GA4

When Google Analytics 4 (GA4) debuted, it wasn’t exactly met with a standing ovation from marketers. 

It was more like a hesitant applause, with some even hosting metaphorical funerals for its predecessor, Universal Analytics (UA). Yes, it was that dramatic!

Irina Serdyukovskaya, an SEO and analytical consultant, found a silver lining in the upheaval while grappling with the new intricacies of GA4. 

“The first months with this were tough,” she admits. But delving into the unknown of GA4 opened up new career paths and opportunities for her. 

“After solving GA4 tracking questions within the SEO field, I started offering this as a separate service and found myself enjoying learning new tools and understanding the data better.”

Serdyukovskaya’s journey resonates with many who initially struggled but eventually found potential growth in adapting to the new system. 

“Changes are hard, and we get used to the way things are, but this can also be an opportunity to grow professionally,” she reflects. 

The collective groan about another change in the digital world symbolizes both resistance to change and a concealed desire for novelty and learning.

Despite the reluctant adoption and the initial hiccups, the majority eventually boarded the GA4 train, with more than 90% of marketers making the transition, albeit with varying levels of readiness and acceptance. 

It wasn’t just a switch but a complete migration to a new platform built from scratch, requiring strategic planning and expert implementation.

GA4 wasn’t merely a new “version.” It was a paradigm shift in how data was collected, processed, and reported. 

And with Google sunsetting UA, the reluctant adaptation became necessary, pushing many to leave the comfort of the familiar and step into the unknown, just like Irina did.

Reflections and looking forward

That’s a wrap on 25 years – a trip down memory lane with the seasoned SEO pros who’ve been through the thick and thin of Google’s ever-evolving world. 

We’ve dived deep into the archives, laughed over the old wild days, and scratched our heads over the relentless changes and updates we’ve all seen and felt.

Huge thanks to all the SEO pros who chipped in with their tales, insights, and lessons for this article. Your shared experiences give us all a closer, more colorful look at the twists and turns that have made the world of SEO what it is today. It’s been a wild ride, and it’s awesome to hear about it from the folks who’ve lived it!

So, what about the next 25 years?

Will AI and Search Generative Experience (SGE) revolutionize how we search and measure SEO success? Will SEO undergo drastic transformations, or heaven forbid, will it finally die as has been foretold many times? Or, in a plot twist, will we all just surrender and switch allegiances to Bing?

It’s a fascinating future to contemplate – a future where the integration of AI could reshape our interaction with online spaces, making searches more intuitive, personalized, and responsive. 

SGE could redefine user engagement, creating more immersive and dynamic experiences.

Will we still be chasing algorithms, or will the advancements enable a more harmonious coexistence where the user experience is paramount? SEO is more about enhancing this experience than outsmarting the system.

Whatever the future holds, here’s to embracing the challenges, exploring new frontiers, and navigating another 25 years of dancing with Google.

Happy 25th, Google! Keep those surprises coming, and keep us guessing!

Google will soon block Bard’s shared conversations from appearing in Google Search. Google Bard recently came out with shared conversations that let users publicly share the chats they had with Bard. Soon after Google Search started to discover those URLs, crawl them, and index them.

The issue. Gagan Ghotra posted on X a screenshot showing how a site command for site:bard.google.com/share returned results from Bard. Those results were shared conversations. Here is a screenshot of Bard showing up in the Google Search index:

Google will block Bard results from Google Search. I asked Google’s Search Liason, Danny Sullivan, about this and he responded on X saying, “Bard allows people to share chats, if they choose. We also don’t intend for these shared chats to be indexed by Google Search. We’re working on blocking them from being indexed now.”

Google will soon remove Bard’s shared conversations from showing in Google Search.

Why we care. So if you see Bard’s shared conversations showing up in Google Search, don’t fret it, they should go away in the coming days. Google said it does not want them to show up in Google Search’s index and they are working on blocking them now.

Apple is not planning to create its own search engine to rival Google.

Eddy Cue, Apple’s senior vice president of services, is expected to testify at the federal antitrust trial that there are no plans for an “Apple Search” model because its partnership with Google works better for its customers, Bloomberg reported.

Cue notably helped negotiate Apple’s multibillion-dollar deal with Google, which took four months “working every single day” to finalize.

Why we care. The U.S. Justice Department argues that Google’s deal with Apple to become the default search engine on its products has played a major role in creating an unfair search landscape as it’s prevented rival search engines from being able to compete seriously. The testimony from Cue, which Apple tried to block per Reuters, provides a significant insight into Google’s relationship with Apple.

Apple’s deal with Google. Google has a financial agreement in place with Apple to be the default search engine on its products at a cost of around $10 billion a year. In addition, Google pays Apple advertising revenue – which is one of the search engine’s biggest costs. See also: Report: Google sharing Chrome iOS search revenue with Apple.

Apple reports its income from Google as advertising revenue, which is categorized under its services division. This amounted to $78.1 billion in sales during Apple’s fiscal year 2022.

Google could pay Apple as much as $19 billion this fiscal year, according to an estimate from Bernstein analyst Toni Sacconaghi.

Why Google is Apple’s default search engine. Cue is expected to testify that Apple chose Google as the default search engine for its product because it is the best search engine. He will also state that Apple has financial deals with other search engines such as Yahoo, Microsoft Bing, DuckDuckGo and Ecosia, as well as Google.

In addition, Cue is set to testify that Apple customers with Google as their default search engine can easily change it, echoing comments made by Google’s lead lawyer, John Schmidtlein.

However, DuckDuckGo founder and CEO Gabriel Weinberg told the court last week that changing a default search engine is “way harder than it needs to be.” He said:

• “If you switch some of these defaults eventually you’re just going to be switched back to Google if you do nothing.”

Who else has testified? Cue will be the second executive from Apple to testify, following Apple AI head (and former Google executive) John Giannandrea last week. However, most of Giannandrea’s testimony was given in a closed courtroom, so the details are not publicly available.

What Google is saying. Schmidtlein said Google dominates the search market with a 90% share because it is a superior product – not because it has financial deals that give its rivals an unfair advantage.

In contrast, Google highlights the default inclusion of Microsoft’s Bing on the Windows operating system, which has not significantly helped Bing’s market position.

Years of Apple Search rumors. Every couple of years, it seems we hear rumors about Apple getting serious about search or even that the company is ready to debut its own search engine – most recently in May 2022. Maybe it’s correlation, but it’s entirely possible Apple uses this to negotiate better financial terms with Google. Either way, last we heard it will be at least four years before we see an Apple search engine.

Dig deeper. Read our Google antitrust trial updates for all the latest developments in this landmark case.

Generative AI, a subset of artificial intelligence, has emerged as a revolutionary force in the tech world. But what exactly is it? And why is it gaining so much attention? 

This in-depth guide will dive into how generative AI models work, what they can and can’t do, and the implications of all these elements.

What is generative AI?

Generative AI, or genAI, refers to systems that can generate new content, be it text, images, music, or even videos. Traditionally, AI/ML meant three things: supervised, unsupervised, and reinforcement learning. Each gives insights based on clustering output. 

Non-generative AI models make calculations based on input (like classifying an image or translating a sentence). In contrast, generative models produce “new” outputs such as writing essays, composing music, designing graphics, and even creating realistic human faces that don’t exist in the real world. 

The implications of generative AI

The rise of generative AI has significant implications. With the ability to generate content, industries like entertainment, design, and journalism are witnessing a paradigm shift. 

For instance, news agencies can use AI to draft reports, while designers can get AI-assisted suggestions for graphics. AI can generate hundreds of ad slogans in seconds – whether or not those options are good or not is another matter. 

Generative AI can produce tailored content for individual users. Think of something like a music app that composes a unique song based on your mood or a news app that drafts articles on topics you’re interested in.

The issue is that as AI plays a more integral role in content creation, questions about authenticity, copyright, and the value of human creativity become more prevalent. 

How does generative AI work?

Generative AI, at its core, is about predicting the next piece of data in a sequence, whether that’s the next word in a sentence or the next pixel in an image. Let’s break down how this is achieved.

Statistical models

Statistical models are the backbone of most AI systems. They use mathematical equations to represent the relationship between different variables. 

For generative AI, models are trained to recognize patterns in data and then use these patterns to generate new, similar data. 

If a model is trained on English sentences, it learns the statistical likelihood of one word following another, allowing it to generate coherent sentences.

Data gathering

Both the quality and quantity of data are crucial. Generative models are trained on vast datasets to understand patterns. 

For a language model, this might mean ingesting billions of words from books, websites, and other texts. 

For an image model, it could mean analyzing millions of images. The more diverse and comprehensive the training data, the better the model will generate diverse outputs.

How transformers and attention work

Transformers are a type of neural network architecture introduced in a 2017 paper titled  “Attention Is All You Need” by Vaswani et al. They have since become the foundation for most state-of-the-art language models. ChatGPT wouldn’t work without transformers.

The “attention” mechanism allows the model to focus on different parts of the input data, much like how humans pay attention to specific words when understanding a sentence. 

This mechanism lets the model decide which parts of the input are relevant for a given task, making it highly flexible and powerful.

The code below is a fundamental breakdown of transformer mechanisms, explaining each piece in plain English.

class Transformer: # Convert words to vectors # What this is: turns words into "vector embeddings" –basically numbers that represent the words and their relationships to each other. # Demo: "the pineapple is cool and tasty" -> [0.2, 0.5, 0.3, 0.8, 0.1, 0.9] self.embedding = Embedding(vocab_size, d_model) # Add position information to the vectors # What this is: Since words in a sentence have a specific order, we add information about each word's position in the sentence. # Demo: "the pineapple is cool and tasty" with position -> [0.2+0.01, 0.5+0.02, 0.3+0.03, 0.8+0.04, 0.1+0.05, 0.9+0.06] self.positional_encoding = PositionalEncoding(d_model) # Stack of transformer layers # What this is: Multiple layers of the Transformer model stacked on top of each other to process data in depth. # Why it does it: Each layer captures different patterns and relationships in the data. # Explained like I'm five: Imagine a multi-story building. Each floor (or layer) has people (or mechanisms) doing specific jobs. The more floors, the more jobs get done! self.transformer_layers = [TransformerLayer(d_model, nhead) for _ in range(num_layers)]  # Convert the output vectors to word probabilities # What this is: A way to predict the next word in a sequence. # Why it does it: After processing the input, we want to guess what word comes next. # Explained like I'm five: After listening to a story, this tries to guess what happens next. self.output_layer = Linear(d_model, vocab_size) def forward(self, x):  # Convert words to vectors, as above x = self.embedding(x)  # Add position information, as above x = self.positional_encoding(x)  # Pass through each transformer layer # What this is: Sending our data through each floor of our multi-story building. # Why it does it: To deeply process and understand the data. # Explained like I'm five: It's like passing a note in class. Each person (or layer) adds something to the note before passing it on, which can end up with a coherent story – or a mess. for layer in self.transformer_layers: x = layer(x)  # Get the output word probabilities # What this is: Our best guess for the next word in the sequence. return self.output_layer(x)

In code, you might have a Transformer class and a single TransformerLayer class. This is like having a blueprint for a floor vs. an entire building. 

This TransformerLayer piece of code shows you how specific components, like multi-head attention and specific arrangements, work. 

class TransformerLayer:  # Multi-head attention mechanism # What this is: A mechanism that lets the model focus on different parts of the input data simultaneously. # Demo: "the pineapple is cool and tasty" might become "this PINEAPPLE is COOL and TASTY" as the model pays more attention to certain words. self.attention = MultiHeadAttention(d_model, nhead)  # Simple feed-forward neural network # What this is: A basic neural network that processes the data after the attention mechanism. # Demo: "this PINEAPPLE is COOL and TASTY" -> [0.25, 0.55, 0.35, 0.85, 0.15, 0.95] (slight changes in numbers after processing) self.feed_forward = FeedForward(d_model) def forward(self, x):  # Apply attention mechanism # What this is: The step where we focus on different parts of the sentence. # Explained like I'm five: It's like highlighting important parts of a book. attention_output = self.attention(x, x, x) # Pass the output through the feed-forward network # What this is: The step where we process the highlighted information. return self.feed_forward(attention_output)

A feed-forward neural network is one of the simplest types of artificial neural networks. It consists of an input layer, one or more hidden layers, and an output layer.

The data flows in one direction – from the input layer, through the hidden layers, and to the output layer. There are no loops or cycles in the network.

In the context of the transformer architecture, the feed-forward neural network is used after the attention mechanism in each layer. It’s a simple two-layered linear transformation with a ReLU activation in between.

# Scaled dot-product attention mechanism
class ScaledDotProductAttention: def __init__(self, d_model):  # Scaling factor helps in stabilizing the gradients # it reduces the variance of the dot product. # What this is: A scaling factor based on the size of our model's embeddings. # What it does: Helps to make sure the dot products don't get too big. # Why it does it: Big dot products can make a model unstable and harder to train. # How it does it: By dividing the dot products by the square root of the embedding size. # It's used when calculating attention scores. # Explained like I'm five: Imagine you shouted something really loud. This scaling factor is like turning the volume down so it's not too loud. self.scaling_factor = d_model ** 0.5 def forward(self, query, key, value): # What this is: The function that calculates how much attention each word should get. # What it does: Determines how relevant each word in a sentence is to every other word. # Why it does it: So we can focus more on important words when trying to understand a sentence. # How it does it: By taking the dot product (the numeric product: a way to measure similarity) of the query and key, then scaling it, and finally using that to weigh our values. # How it fits into the rest of the code: This function is called whenever we want to calculate attention in our model. # Explained like I'm five: Imagine you have a toy and you want to see which of your friends likes it the most. This function is like asking each friend how much they like the toy, and then deciding who gets to play with it based on their answers. # Calculate attention scores by taking the dot product of the query and key. scores = dot_product(query, key) / self.scaling_factor # Convert the raw scores to probabilities using the softmax function. attention_weights = softmax(scores) # Weight the values using the attention probabilities. return dot_product(attention_weights, value) # Feed-forward neural network
# This is an extremely basic example of a neural network.
class FeedForward: def __init__(self, d_model): # First linear layer increases the dimensionality of the data. self.layer1 = Linear(d_model, d_model * 4)
 # Second linear layer brings the dimensionality back to d_model. self.layer2 = Linear(d_model * 4, d_model) def forward(self, x): # Pass the input through the first layer,
#Pass the input through the first layer:
# Input: This refers to the data you feed into the neural network. I
#First layer: Neural networks consist of layers, and each layer has neurons. When we say "pass the input through the first layer," we mean that the input data is being processed by the neurons in this layer. Each neuron takes the input, multiplies it by its weights (which are learned during training), and produces an output.
# apply ReLU activation to introduce non-linearity, # and then pass through the second layer.
#ReLU activation: ReLU stands for Rectified Linear Unit.  # It's a type of activation function, which is a mathematical function applied to the output of each neuron. In simpler terms, if the input is positive, it returns the input value; if the input is negative or zero, it returns zero.
# Neural networks can model complex relationships in data by introducing non-linearities. # Without non-linear activation functions, no matter how many layers you stack in a neural network, it would behave just like a single-layer perceptron because summing these layers would give you another linear model. # Non-linearities allow the network to capture complex patterns and make better predictions. return self.layer2(relu(self.layer1(x))) # Positional encoding adds information about the position of each word in the sequence.
class PositionalEncoding: def __init__(self, d_model): # What this is: A setup to add information about where each word is in a sentence. # What it does: Prepares to add a unique "position" value to each word. # Why it does it: Words in a sentence have an order, and this helps the model remember that order. # How it does it: By creating a special pattern of numbers for each position in a sentence. # How it fits into the rest of the code: Before processing words, we add their position info. # Explained like I'm five: Imagine you're in a line with your friends. This gives everyone a number to remember their place in line. pass def forward(self, x): # What this is: The main function that adds position info to our words. # What it does: Combines the word's original value with its position value. # Why it does it: So the model knows the order of words in a sentence. # How it does it: By adding the position values we prepared earlier to the word values. # How it fits into the rest of the code: This function is called whenever we want to add position info to our words. # Explained like I'm five: It's like giving each of your toys a tag that says if it's the 1st, 2nd, 3rd toy, and so on. return x # Helper functions
def dot_product(a, b): # Calculate the dot product of two matrices. # What this is: A mathematical operation to see how similar two lists of numbers are. # What it does: Multiplies matching items in the lists and then adds them up. # Why it does it: To measure similarity or relevance between two sets of data. # How it does it: By multiplying and summing up. # How it fits into the rest of the code: Used in attention to see how relevant words are to each other. # Explained like I'm five: Imagine you and your friend have bags of candies. You both pour them out and match each candy type. Then, you count how many matching pairs you have. return a @ b.transpose(-2, -1) def softmax(x): # Convert raw scores to probabilities ensuring they sum up to 1. # What this is: A way to turn any list of numbers into probabilities. # What it does: Makes the numbers between 0 and 1 and ensures they all add up to 1. # Why it does it: So we can understand the numbers as chances or probabilities. # How it does it: By using exponentiation and division. # How it fits into the rest of the code: Used to convert attention scores into probabilities. # Explained like I'm five: Lets go back to our toys. This makes sure that when you share them, everyone gets a fair share, and no toy is left behind. return exp(x) / sum(exp(x), axis=-1) def relu(x): # Activation function that introduces non-linearity. It sets negative values to 0. # What this is: A simple rule for numbers. # What it does: If a number is negative, it changes it to zero. Otherwise, it leaves it as it is. # Why it does it: To introduce some simplicity and non-linearity in our model's calculations. # How it does it: By checking each number and setting it to zero if it's negative. # How it fits into the rest of the code: Used in neural networks to make them more powerful and flexible. # Explained like I'm five: Imagine you have some stickers, some are shiny (positive numbers) and some are dull (negative numbers). This rule says to replace all dull stickers with blank ones. return max(0, x)

How generative AI works – in simple terms

Think of generative AI as rolling a weighted dice. The training data determine the weights (or probabilities). 

If the dice represents the next word in a sentence, a word often following the current word in the training data will have a higher weight. So, “sky” might follow “blue” more often than “banana”. When the AI “rolls the dice” to generate content, it’s more likely to choose statistically more probable sequences based on its training.

So, how can LLMs generate content that “seems” original? 

Let’s take a fake listicle – the “best Eid al-Fitr gifts for content marketers” – and walk through how an LLM can generate this list by combining textual cues from documents about gifts, Eid, and content marketers.

Before processing, the text is broken down into smaller pieces called “tokens.” These tokens can be as short as one character or as long as one word.

Example: “Eid al-Fitr is a celebration” becomes [“Eid”, “al-Fitr”, “is”, “a”, “celebration”].

This allows the model to work with manageable chunks of text and understand the structure of sentences.

Each token is then converted into a vector (a list of numbers) using embeddings. These vectors capture the meaning and context of each word.

Positional encoding adds information to each word vector about its position in the sentence, ensuring the model doesn’t lose this order information.

Then we use an attention mechanism: this allows the model to focus on different parts of the input text when generating an output. If you remember BERT, this is what was so exciting to Googlers about BERT. 

If our model has seen texts about “gifts” and knows that people give gifts during celebrations, and it has also seen texts about “Eid al-Fitr” being a significant celebration, it will pay “attention” to these connections.

Similarly, if it has seen texts about “content marketers” needing specific tools or resources, it can connect the idea of “gifts” to “content marketers“.

Now we can combine contexts: As the model processes the input text through multiple Transformer layers, it combines the contexts it has learned.

So, even if the original texts never mentioned “Eid al-Fitr gifts for content marketers,” the model can bring together the concepts of “Eid al-Fitr,” “gifts,” and “content marketers” to generate this content.

This is because it has learned the broader contexts around each of these terms.

After processing the input through the attention mechanism and the feed-forward networks in each Transformer layer, the model produces a probability distribution over its vocabulary for the next word in the sequence.

It might think that after words like “best” and “Eid al-Fitr,” the word “gifts” has a high probability of coming next. Similarly, it might associate “gifts” with potential recipients like “content marketers.”

How large language models are built

The journey from a basic transformer model to a sophisticated large language model (LLM) like GPT-3 or BERT involves scaling up and refining various components. 

Here’s a step-by-step breakdown:

LLMs are trained on vast amounts of text data. It’s hard to explain how vast this data is.

The C4 dataset, a starting point for many LLMs, is 750 GB of text data. That’s 805,306,368,000 bytes – a lot of information. This data can include books, articles, websites, forums, comment sections, and other sources. 

The more varied and comprehensive the data, the better the model’s understanding and generalization capabilities.

While the basic transformer architecture remains the foundation, LLMs have a significantly larger number of parameters. GPT-3, for example, has 175 billion parameters. In this case, parameters refer to the weights and biases in the neural network that are learned during the training process.

In deep learning, a model is trained to make predictions by adjusting these parameters to reduce the difference between its predictions and the actual outcomes. 

The process of adjusting these parameters is called optimization, which uses algorithms like gradient descent.

• Weights: These are values in the neural network that transform input data within the network’s layers. They are adjusted during training to optimize the model’s output. Each connection between neurons in adjacent layers has an associated weight.
• Biases: These are also values in the neural network that are added to the output of a layer’s transformation. They provide an additional degree of freedom to the model, allowing it to fit the training data better. Each neuron in a layer has an associated bias.

This scaling allows the model to store and process more intricate patterns and relationships in the data.

The large number of parameters also means that the model requires significant computational power and memory for training and inference. This is why training such models is resource-intensive and typically uses specialized hardware like GPUs or TPUs.

The model is trained to predict the next word in a sequence using powerful computational resources. It adjusts its internal parameters based on the errors it makes, continuously improving its predictions.

Attention mechanisms like the ones we’ve discussed are pivotal for LLMs. They allow the model to focus on different parts of the input when generating output. 

By weighing the importance of different words in a context, attention mechanisms enable the model to generate coherent and contextually relevant text. Doing it at this massive scale enables the LLMs to work the way they do.

How does a transformer predict text?

Transformers predict text by processing input tokens through multiple layers, each equipped with attention mechanisms and feed-forward networks. 

After processing, the model produces a probability distribution over its vocabulary for the next word in the sequence. The word with the highest probability is typically selected as the prediction.

How is a large language model built and trained?

Building an LLM involves gathering data, cleaning it, training the model, fine-tuning the model, and vigorous, continuous testing.

The model is initially trained on a vast corpus to predict the next word in a sequence. This phase allows the model to learn connections between words that pick up patterns in grammar, relationships that can represent facts about the world and connections that feel like logical reasoning. These connections also make it pick up biases present in the training data.

After pre-training, the model is refined on a narrower dataset, often with human reviewers following guidelines. 

Fine-tuning is a crucial step in building LLMs. It involves training the pre-trained model on a more specific dataset or task. Let’s take ChatGPT as an example. 

If you’ve played with GPT models, you know that prompting is less “write this thing” and more like

• Prompt: Once upon a time
  • Continuation: There was an evil wizard at the top of a tower.
• Prompt: Why did the chicken join a band?
  • Continuation: Because it had the drumsticks!

To get to ChatGPT from that point involves a lot of low-paid labor. Those people create immense corpora to put a finger on the weight of GPT responses and expected behaviors. These workers create tons of prompt/continuation texts that are like:

• Prompt: Finish this story: “Once upon a time..”
  • Continuation: Sure! Once upon a time, in a land far, far away, there was a small village nestled between two majestic mountains. 
• Prompt: Tell me a joke about a chicken.
  • Continuation: Why did the chicken join a band? Because it had the drumsticks!

This fine-tuning process is essential for several reasons:

• Specificity: While pre-training gives the model a broad understanding of language, fine-tuning narrows its knowledge and behavior to align more with specific tasks or domains. For instance, a model fine-tuned on medical data will better answer medical questions.
• Control: Fine-tuning gives developers more control over the model’s outputs. Developers can use a curated dataset to guide the model to produce desired responses and avoid unwanted behaviors.
• Safety: It helps in reducing harmful or biased outputs. By using guidelines during the fine-tuning process, human reviewers can ensure the model doesn’t produce inappropriate content.
• Performance: Fine-tuning can significantly improve the model’s performance on specific tasks. For example, a model that’s been fine-tuned for customer support will be much better at it than a generic model.

You can tell ChatGPT has been fine-tuned in particular in some ways.

For example, “logical reasoning” is something LLMs tend to struggle with. ChatGPT’s best logical reasoning model – GPT-4 – has been trained intensely to recognize patterns in numbers explicitly. 

Instead of something like this:

• Prompt: What’s 2+2?
• Process: Oftentimes in math textbooks for children 2+2 =4. Occasionally there are references to “2+2=5” but there is usually more context to do with George Orwell or Star Trek when that is the case. If this was in that context the weight would be more in favor of 2+2=5. But that context doesn’t exist, so in this instance the next token is likely 4.
• Response: 2+2=4

The training does something like this:

• training: 2+2=4
• training: 4/2=2
• training: half of 4 is 2
• training: 2 of 2 is four

…and so on. 

This means for those more “logical” models, the training process is more rigorous and focused on ensuring that the model understands and correctly applies logical and mathematical principles. 

The model is exposed to various mathematical problems and their solutions, ensuring it can generalize and apply these principles to new, unseen problems.

The importance of this fine-tuning process, especially for logical reasoning, cannot be overstated. Without it, the model might provide incorrect or nonsensical answers to straightforward logical or mathematical questions. 

Image models vs. language models

While both image and language models might use similar architectures like transformers, the data they process is fundamentally different:

Image models

These models deal with pixels and often work in a hierarchical manner, analyzing small patterns (like edges) first, then combining them to recognize larger structures (like shapes), and so on until they understand the entire image.

Language models

These models process sequences of words or characters. They need to understand the context, grammar, and semantics to generate coherent and contextually relevant text.

How prominent generative AI interfaces work

Dall-E + Midjourney

Dall-E is a variant of the GPT-3 model adapted for image generation. It’s trained on a vast dataset of text-image pairs. Midjourney is another image generation software that is based on a proprietary model.

• Input: You provide a textual description, like “a two-headed flamingo.”
• Processing: These models encode this text into a series of numbers and then decode these vectors, finding relationships to pixels, to produce an image. The model has learned the relationships between textual descriptions and visual representations from its training data.
• Output: An image that matches or relates to the given description.

Fingers, patterns, problems

Why can’t these tools consistently generate hands that look normal? These tools work by looking at pixels next to each other. 

You can see how this works when comparing earlier or more primitive generated images with more recent ones: earlier models look very fuzzy. In contrast, more recent models are a lot crisper. 

These models generate images by predicting the next pixel based on the pixels it has already generated. This process is repeated millions of times over to produce a complete image.

Hands, especially fingers, are intricate and have a lot of details that need to be captured accurately. 

Each finger’s positioning, length, and orientation can vary greatly in different images. 

When generating an image from a textual description, the model has to make many assumptions about the exact pose and structure of the hand, which can lead to anomalies.

ChatGPT

ChatGPT is based on the GPT-3.5 architecture, a transformer-based model designed for natural language processing tasks.

• Input: A prompt or a series of messages to simulate a conversation.
• Processing: ChatGPT uses its vast knowledge from diverse internet texts to generate responses. It considers the context provided in the conversation and tries to produce the most relevant and coherent reply.
• Output: A text response that continues or answers the conversation.

Specialty

ChatGPT’s strength lies in its ability to handle various topics and simulate human-like conversations, making it ideal for chatbots and virtual assistants.

Bard + Search Generative Experience (SGE)

While specific details might be proprietary, Bard is based on transformer AI techniques, similar to other state-of-the-art language models. SGE is based on similar models but weaves in other ML algorithms Google uses. 

SGE likely generates content using a transformer-based generative model and then fuzzy extracts answers from ranking pages in search. (This may not be true. Just a guess based on how it seems to work from playing with it. Please don’t sue me!)

• Input: A prompt/command/search
• Processing: Bard processes the input and works the way other LLMs do. SGE uses a similar architecture but adds a layer where it searches its internal knowledge (gained from training data) to generate a suitable response. It considers the prompt’s structure, context, and intent to produce relevant content.
• Output: Generated content that can be a story, answer, or any other type of text.

Applications of generative AI (and their controversies)

Art and design

Generative AI can now create artwork, music, and even product designs. This has opened up new avenues for creativity and innovation.

Controversy

The rise of AI in art has sparked debates about job losses in creative fields. 

Additionally, there are concerns about:

• Labor violations, especially when AI-generated content is used without proper attribution or compensation.
• Executives threatening writers with replacing them with AI is one of the issues that spurred the writers’ strike.

Natural language processing (NLP)

AI models are now widely used for chatbots, language translation, and other NLP tasks. 

Outside the dream of artificial general intelligence (AGI), this is the best use for LLMs since they are close to a “generalist” NLP model. 

Controversy

Many users find chatbots to be impersonal and sometimes annoying. 

Moreover, while AI has made significant strides in language translation, it often lacks the nuance and cultural understanding that human translators bring, leading to impressive and flawed translations.

Medicine and drug discovery

AI can quickly analyze vast amounts of medical data and generate potential drug compounds, speeding up the drug discovery process. Many doctors already use LLMs to write notes and patient communications

Controversy

Relying on LLMs for medical purposes can be problematic. Medicine requires precision, and any errors or oversights by AI can have serious consequences. 

Medicine also already has biases that only get more baked in using LLMs. There are also similar issues, as discussed below, with privacy, efficacy, and ethics.

Gaming

Many AI enthusiasts are excited about using AI in gaming: they say that AI can generate realistic gaming environments, characters, and even entire game plots, enhancing the gaming experience. NPC dialogue can be enhanced through using these tools. 

Controversy

There’s a debate about the intentionality in game design. 

While AI can generate vast amounts of content, some argue it lacks the deliberate design and narrative cohesion that human designers bring. 

Watchdogs 2 had programmatic NPCs, which did little to add to the narrative cohesion of the game as a whole. 

Marketing and advertising

AI can analyze consumer behavior and generate personalized advertisements and promotional content, making marketing campaigns more effective. 

LLMs have context from other people’s writing, making them useful for generating user stories or more nuanced programmatic ideas. Instead of recommending TVs to someone who just bought a TV, LLMs can recommend accessories someone might want instead.

Controversy

The use of AI in marketing raises privacy concerns. There’s also a debate about the ethical implications of using AI to influence consumer behavior.

Dig deeper: How to scale the use of large language models in marketing

Continuing issues with LLMS

Contextual understanding and comprehension of human speech

• Limitation: AI models, including GPT, often struggle with nuanced human interactions, such as detecting sarcasm, humor, or lies.
• Example: In stories where a character is lying to other characters, the AI might not always grasp the underlying deceit and might interpret statements at face value.

Pattern matching

• Limitation: AI models, especially those like GPT, are fundamentally pattern matchers. They excel at recognizing and generating content based on patterns they’ve seen in their training data. However, their performance can degrade when faced with novel situations or deviations from established patterns.
• Example: If a new slang term or cultural reference emerges after the model’s last training update, it might not recognize or understand it.

Lack of common sense understanding

• Limitation: While AI models can store vast amounts of information, they often lack a “common sense” understanding of the world, leading to outputs that might be technically correct but contextually nonsensical.

Potential to reinforce biases

• Ethical consideration: AI models learn from data, and if that data contains biases, the model will likely reproduce and even amplify those biases. This can lead to outputs that are sexist, racist, or otherwise prejudiced.

Challenges in generating unique ideas

• Limitation: AI models generate content based on patterns they’ve seen. While they can combine these patterns in novel ways, they don’t “invent” like humans do. Their “creativity” is a recombination of existing ideas.

Data Privacy, Intellectual Property, and Quality Control Issues:

• Ethical consideration: Using AI models in applications that handle sensitive data raises concerns about data privacy. When AI generates content, questions arise about who owns the intellectual property rights. Ensuring the quality and accuracy of AI-generated content is also a significant challenge.

Bad code

• AI models might generate syntactically correct code when used for coding tasks but functionally flawed or insecure. I have had to correct the code people have added to sites they generated using LLMs. It looked right, but was not. Even when it does work, LLMs have out-of-date expectations for code, using functions like “document.write” that are no longer considered best practice.

Hot takes from an MLOps engineer and technical SEO

This section covers some hot takes I have about LLMs and generative AI. Feel free to fight with me. 

Prompt engineering isn’t real (for generative text interfaces)

Generative models, especially large language models (LLMs) like GPT-3 and its successors, have been touted for their ability to generate coherent and contextually relevant text based on prompts.

Because of this, and since these models have become the new “gold rush,” people have started to monetize “prompt engineering” as a skill. This can be either $1,400 courses or prompt engineering jobs.

However, there are some critical considerations:

LLMs change rapidly

As technology evolves and new model versions are released, how they respond to prompts can change. What worked for GPT-3 might not work the same way for GPT-4 or even a newer version of GPT-3.

This constant evolution means prompt engineering can become a moving target, making it challenging to maintain consistency. Prompts that work in January may not work in March.

Uncontrollable outcomes

While you can guide LLMs with prompts, there’s no guarantee they’ll always produce the desired output. For instance, asking an LLM to generate a 500-word essay might result in outputs of varying lengths because LLMs don’t know what numbers are.

Similarly, while you can ask for factual information, the model might produce inaccuracies because it cannot tell the difference between accurate and inaccurate information by itself.

Using LLMs in non-language-based applications is a bad idea

LLMs are primarily designed for language tasks. While they can be adapted for other purposes, there are inherent limitations:

Struggle with novel ideas

LLMs are trained on existing data, which means they’re essentially regurgitating and recombining what they’ve seen before. They don’t “invent” in the truest sense of the word. 

Tasks that require genuine innovation or out-of-the-box thinking should not use LLMs. 

You can see an issue with this when it comes to people using GPT models for news content – if something novel comes along, it’s hard for LLMs to deal with it.

For example, a site that seems to be generating content with LLMs published a possibly libelous article about Megan Crosby. Crosby was caught elbowing opponents in real life.

Without that context, the LLM created a completely different, evidence-free story about a “controversial comment.”

Text-focused

At their core, LLMs are designed for text. While they can be adapted for tasks like image generation or music composition, they might not be as proficient as models specifically designed for those tasks.

LLMs don’t know what the truth is

They generate outputs based on patterns encountered in their training data. This means they can’t verify facts or discern true and false information. 

If they’ve been exposed to misinformation or biased data during training, or they don’t have context for something, they might propagate those inaccuracies in their outputs. 

This is especially problematic in applications like news generation or academic research, where accuracy and truth are paramount. 

Think about it like this: if an LLM has never come across the name “Jimmy Scrambles” before but knows it’s a name, prompts to write about it will only come up with related vectors.

Designers are always better than AI-generated Art

AI has made significant strides in art, from generating paintings to composing music. However, there’s a fundamental difference between human-made art and AI-generated art:

Intent, feeling, vibe

Art is not just about the final product but the intent and emotion behind it.

A human artist brings their experiences, emotions, and perspectives to their work, giving it depth and nuance that’s challenging for AI to replicate.

A “bad” piece of art from a person has more depth than a beautiful piece of art from a prompt.

When implementing an SEO strategy, a document roadmap is crucial for several reasons:

• It serves as your daily guide.
• It represents what non-SEO professionals perceive as SEO – a practical checklist of tasks.
• It provides a structure to communicate expectations with stakeholders.

A solid, well-researched strategy is best complemented with an SEO roadmap in Google Sheets or a tool like Jira or Asana. Ideally, leverage the technologies your teams are already using. 

This article discusses why an SEO roadmap is essential for your strategy. Plus, learn how to create a solid SEO roadmap to prioritize tasks and collaborate effectively with teams.

What you need for an SEO roadmap

An SEO roadmap is a lot like SEO as a whole, it cannot successfully exist in a silo. That’s both in terms of your own research and your team within the wider business. 

Within SEO, you’d ideally have a completed SEO strategy, which includes your: 

Once you have all of that, you should be able to layer it on top of your wider business context. This includes: 

• Your business goals for the next one to five years.
• Tools, people and budget availability. This includes knowing when and how you can pitch for budget.
  • Planned product launches or marketing campaigns.
• Your technology stack and any known limitations, considerations, or planned changes (including migrations).

Gathering this information for your SEO roadmap is crucial because, initially, you’ll aim to tackle high-impact, low-effort tasks. Keep in mind that these tasks vary depending on your organization.

Understanding your business goals will:

• Help align your SEO “quick wins” with work that will actually directly benefit the bottom line of the business.
• Give you the ability to speak to that benefit when getting the work done. 

Knowing the people, tools and budget available to you may push you to reconsider where you start with your roadmap. 

Let’s say you’re launching a new product in the next three months, and your development team is totally at capacity until a month after that launches. Maybe you start with your content team and create new hub-and-spoke content within your organization’s area of expertise first.

For example, when I worked at Optus, a major telco in Australia, folks would be hard-pressed to get any new development done on the website, large or small, from about June to September because we were preparing for the iPhone launch.

(In that instance, I was a part of that launch team, so I had a bit more wiggle room to get stuff done than I would’ve otherwise, but you get the gist). 

You’ll also want to get a grasp on your tech stack. For some tech stack configurations, particularly on smaller websites or with SPAs, implementing a 404 server response or a 301 redirect might seem straightforward but is much more difficult and time-consuming than standard. 

It isn’t easy to generalize in SEO. So, while you may want to say that a high-impact, low-effort task would be to resize the 10MB images on your product pages (yes, I’ve seen this happen, and you probably have too), there may be a reason why your images are that large, or why they haven’t been compressed yet. 

Having all this context before putting together an SEO roadmap becomes your early indicator of those sticky situations. 

Steps to create your SEO roadmap

1. Prioritize

This is the first round of prioritization, the one where I estimate the SEO impact in a silo. 

If I look at nothing else, the one metric I look at to help prioritize is how much of the website that potential issue affects. 

Once I understand the scale, I look to estimate the impact. Quantifying impact can be difficult, but there are a few ways to do it: 

• Top-down using competitor analysis and historical or current website changes as a model.
• Bottom-up using keyword-level forecasting based on estimated position changes.
• Traffic estimates using historical modeling and tactically based case studies or historical competitor execution.

While we can be secure in our scale and impact estimates for SEO, the one element we’re less certain of is effort. 

I’ll still take a stab at estimating effort, but to do it well, I bring in the experts.

2. Assign and estimate

SEO can sometimes be like herding cats because you rely on other people or teams to implement your tactics. 

This is why I start my roadmap with what some may feel is that last step – figuring out who’ll be doing the work and speaking to them about how long they think it will take. 

If relevant, I’ll ask when they think they can do it and any other dependencies they’d need to do the work, like budget. 

I’m not an expert in design, UX, or front-end development for our website. They are.

So, while I can build a rough estimate of what I think a task will take based on my own experiences with other clients or what the web generally says, it’s only a T-shirt size – one that could be off-base for reasons I don’t know.

I don’t know the legacy code we have stringing through the website or the fact that it can take up to an hour sometimes to replace images on a product page because of a buggy CMS that’ll randomly not apply filename changes sometimes. 

I don’t know if we’re amidst an unofficial code migration or running 20x CRO tests on that page. 

I don’t know all that stuff, but my teams do.  

So I take my SEO strategy, which I’ve probably worked on for months, agonized and perfected, and I pull out all the tasks I think a particular team will do. 

Then I take that and do a rough grooming and sizing session for those tasks with someone who knows what they’re talking about – usually, with the product owner of that squad. 

And if you don’t know who that is, it’s a good time to make friends. I swear, 90% of being an SEO is about being curious and kind. 

Note: Depending on your organization, your team might estimate differently. Sometimes, it’s in days. Other times, it’s using an agile Fibonacci scale or something similar. 

Whatever it is, you’ll want to be consistent. A Fibonacci scale can make translating to days in order to estimate the time frame a bit more difficult. Still, I’d lean on the team to give an average execution time so you can then put in at least an estimated delivery.

This estimation process is helpful for enterprise-level companies because, assuming the conversation goes well, you’re essentially getting a soft yes from that team. 

That’ll make it easier for them to say yes for real when the work comes across their desk later.

And then once that’s done, for me, it’s mostly paperwork.  

3. Do the paperwork

For me, there’s no better format than Gantt when it comes to timed work and keeping track of it.

You can build it like I typically do in Google Sheets or use software-based versions in tools like Jira or Monday. 

Define your headings

Your headings should be your primary work areas and can be based on your strategy. 

For example, you may have buckets of work related to infrastructure, content or engineering. 

For my strategy and roadmap, I typically align my primary areas of work to the teams that will primarily be handling them because that’s how my brain keeps track of things best. 

Sometimes, aligning them differently, for example, with your business goals or team KPIs may make sense.

This could be something like “site speed, accessibility, expertise” where site speed and accessibility would primarily be development teams, and expertise would be a mix of SEO, development and content tasks. 

Define your tasks

Everyone has a different perspective on what tasks should and shouldn’t be included in an SEO roadmap. Two major considerations are: 

• How much you’re able to execute yourself.
• The level of detail you want and at what point it becomes overwhelming.

I typically work with enterprise clients, so in terms of tasks, there’s not usually much I can actually do myself. 

What this means is my SEO roadmap is often a duplication of tasks that are assigned elsewhere, like JIRA tickets for developers or Asana tasks for designers. 

The SEO roadmap becomes a centralized way for me to keep track of progress on SEO initiatives all in one place rather than a to-do list for me and my team to follow. 

Let’ say you can execute much of the work yourself, then fabulous.

If tasks you can do yourself account for more than half the tasks in your roadmap, I’d suggest culling the list to focus on SEO work only and tracking non-SEO work either in a different tab in the spreadsheet or elsewhere.  

For the granularity of tasks, aim for the middle ground – not too big and overwhelming, but also not too small and insignificant. 

You wouldn’t want to include, say, “change the title tag on the blue widget page. But “change all title tags on the website” might be too large if you’re an enterprise site. 

A middle ground could be, “Rewrite title tag formula for product page template, ” or even “Rewrite all title tag formulas.” 

Something like “Product page optimization” could and should be broken into its component tasks, like: 

• Create and share example product schema output for implementation.
• Create FAQ for top 20 revenue-driving products.
• Resize images for top 20 revenue-driving products.  
• Work with content team to re-write product descriptions so they’re different from the manufacturer-provided ones for top 20 revenue-driving products.

From a practical perspective, if this is a year-long SEO roadmap, you want a manageable number of rows rather than hundreds or even thousands. 

You’d likely get overwhelmed and end up not feeling confident getting started. Analysis paralysis is real. 

4. Re-prioritize, assign and schedule your tasks

To use the SEO roadmap well, you need to treat it with laser focus. Once you have everything bucketed and tasked and know how long they’ll take to execute, it’s time to review your priority. 

I prioritize tasks through a few different lenses after taking clear notes from my teams about dependencies and requirements for getting the work done. 

First, the Eisenhower matrix. 

This simple scale of the intersection of urgency and importance can be super grounding: more will probably end up in the “delete” bucket than you think. 

I generally put these in a list in my notebook or project management tool I call “If/When I Have Time.” 

This is after the initial grooming with execution teams, which probably discussed a few tasks or elements of tasks I realized weren’t worth the effort for the impact they would have. 

I then come back to my initial prioritization and factor in the effort estimation from my teams. 

• Does their feedback change my estimate of the impact? 
• Did their expertise change my understanding of the scale? 

Finally, we come back to the start. With tasks re-prioritized from actual feedback from the folks that’d be doing the work, it’s time to get the ball rolling. 

This is when I tap execution teams on the shoulder to let them know this work should be coming their way and confirm they’ll be the ones to execute. 

I start those “let’s make this official” for the tasks scheduled for the next six weeks, which, if all has gone well, should be those low-effort, high-impact tasks. 

Drive SEO results with a well-defined roadmap

In case you weren’t sure, yes, this is a cycle. There’s a reason I started and ended my suggested process with similar tasks around assigning and estimating. 

A strong SEO roadmap often moves and adjusts to what’s happening in the business and on the Internet. It’s an agile, living document. 

When creating an SEO roadmap, my advice is to: 

• Have a clear picture of the business and market context.
• Prioritize tasks first based on your SEO expertise.
• Re-prioritize tasks based on estimations and feedback from the executing teams.
• Organize your roadmap based on business goals or execution teams.
• Be focused on which tasks are actually valuable.
• Remember this is different from your day-to-day task list. Detailed page-level tasks should probably be kept out of the task list.
• Revisit the roadmap regularly.

With all that in mind, you’re armed to build a strong and flexible SEO roadmap for your clients, your business or your employer. 

Artificial intelligence (AI) is poised to become a vital tool for brands seeking to enhance their online presence. 

However, integrating AI into marketing strategies inevitably creates legal considerations and new regulations that agencies must carefully navigate. 

In this article, you’ll discover:

• How businesses and SEO and media agencies can minimize legal risks of implementing AI-enhanced strategies.
• Useful tools to reduce AI bias and a handy process to review AI-generated content quality.
• How agencies can navigate main AI implementation challenges to ensure efficiency and compliance for their clients.

Legal compliance considerations

Intellectual property and copyright

A crucial legal concern when using AI in SEO and media is following intellectual property and copyright laws.

AI systems often scrape and analyze vast amounts of data, including copyrighted material. 

There are already multiple lawsuits against OpenAI over copyright and privacy violations.

The company faces lawsuits alleging unauthorized use of copyrighted books for training ChatGPT and illegally collecting personal information from internet users using their machine learning models.

Privacy concerns on OpenAI’s processing and saving of user data also caused Italy to entirely block the use of ChatGPT at the end of March. 

The ban has now been lifted after the company made changes to increase transparency on the chatbot’s user data processing and add an option to opt out of ChatGPT’s conversations used for training algorithms. 

However, with the launch of GPTBot, OpenAI’s crawler, further legal considerations are likely to arise.

To avoid potential legal issues and infringement claims, agencies must ensure any AI models are trained on authorized data sources and respect copyright restrictions:

• Ensure data has been obtained legally and the agency has the appropriate rights to use it.
• Filter out data that doesn’t have the required legal permissions or is of poor quality.
• Conduct regular audits of data and AI models to ensure they comply with data usage rights and laws.
• Hold a legal consultation of data rights and privacy to ensure nothing conflicts with legal policies.

Both agency and client legal teams will likely need to be involved in the above discussions before AI models can be integrated into workstreams and projects.

Data privacy and protection

AI technologies rely heavily on data, which may include sensitive personal information. 

Collecting, storing, and processing user data must align with relevant privacy laws, such as the General Data Protection Regulation (GDPR) in the European Union. 

Moreover, the recently introduced EU AI Act also emphasizes addressing data privacy concerns associated with AI systems. 

This is not without merit. Large corporations, such as Samsung, have banned AI completely due to the exposure of confidential data uploaded to ChatGPT. 

Therefore, if agencies use customer data in conjunction with AI technology, they should: 

• Prioritize transparency in data collection.
• Obtain user consent.
• Implement robust security measures to safeguard sensitive information. 

In these cases, agencies can prioritize transparency in data collection by clearly communicating to users which data will be collected, how it will be used, and who will have access to it. 

To obtain user consent, ensure that consent is informed and freely given through clear and easy-to-understand consent forms that explain the purpose and benefits of data collection.

In addition, robust security measures include:

• Data encryption.
• Access control.
• Data anonymization (where possible).
• Regular audits and updates. 

For example, OpenAI’s policies align with the need for data privacy and protection and focus on promoting transparency, user consent and data security in AI applications.

Fairness and bias

AI algorithms used in SEO and media have the potential to inadvertently perpetuate biases or discriminate against certain individuals or groups. 

Agencies must be proactive in identifying and mitigating algorithmic bias. This is especially important under the new EU AI Act, which prohibits AI systems from unfairly affecting human behavior or displaying discriminatory behavior.

To mitigate this risk, agencies should ensure that diverse data and perspectives are included in the design of AI models and continuously monitor results for potential bias and discrimination. 

A way to achieve this is by using tools that help reduce bias, like AI Fairness 360, IBM Watson Studio and Google’s What-If Tool.

False or misleading content

AI tools, including ChatGPT, can generate synthetic content that may be inaccurate, misleading or fake. 

For example, artificial intelligence often creates fake online reviews to promote certain places or products. This can lead to negative consequences for businesses that rely on AI-generated content.

Implementing clear policies and procedures for reviewing AI-generated content before publication is crucial to prevent this risk.

Another practice to consider is labeling AI-generated content. Although Google seems not to enforce it, many policymakers support AI labeling.

Liability and accountability

As AI systems become more complex, questions of liability arise. 

Agencies utilizing AI must be prepared to take responsibility for any unintended consequences resulting from its use, including: 

• Bias and discrimination when using AI to sort candidates for hiring.
• The potential to abuse the power of AI for malicious purposes such as cyberattacks.
• The loss of privacy if information is collected without consent.

The EU AI Act introduces new provisions on high-risk AI systems that can significantly affect users’ rights, highlighting why agencies and clients must comply with the relevant terms and policies when using AI technologies. 

Some of OpenAI’s most important terms and policies relate to the content provided by the user, the accuracy of responses and the processing of personal data. 

The content policy states that OpenAI assigns the rights of the generated content to the user. It also specifies that generated content can be used for any purpose, including commercial, providing it complies with legal restrictions. 

However, it also states that output may be neither completely unique nor accurate, meaning that AI-generated content should always be thoroughly reviewed before use. 

On a personal data note, OpenAI collects all information users input, including file uploads. 

When using the service to process personal data, users must provide legally adequate privacy notices and fill out a form to request data processing.

Agencies must proactively address accountability issues, monitor AI outputs, and implement robust quality control measures to mitigate potential legal liabilities. 

AI implementation challenges for agencies

Since OpenAI released ChatGPT last year, there have been many talks on how generative AI will change SEO as a profession and its overall impact on the media industry. 

Although changes come with a mix of enhancements to the daily workload, there are some challenges agencies should consider when implementing AI into client’s strategies. 

Education and awareness

Many clients may lack a comprehensive understanding of AI and its implications. 

Agencies, therefore, face the challenge of educating clients about the potential benefits and risks associated with AI implementation. 

The evolving regulatory landscape necessitates clear communication with clients regarding the measures taken to ensure legal compliance. 

In order to achieve this, agencies must:

• Have a clear understanding of their client’s goals.
• Be able to explain the benefits.
• Show expertise in implementing AI.
• Address the challenges and risks. 

A way to do that is by having a fact sheet to share with clients containing all the necessary information and, if possible, provide case studies or other examples of how they can benefit from using artificial intelligence. 

Resource allocation

Integrating AI into SEO and media strategies requires significant resources, including financial investments, skilled personnel and infrastructure upgrades. 

Agencies must carefully assess their clients’ needs and capabilities to determine the feasibility of implementing AI solutions within their budgetary constraints, as they may require AI specialists, data analysts, SEO and content specialists that can effectively collaborate together. 

Infrastructure needs may include AI tools, data processing and analytics platforms to extract insights. Whether to provide services or facilitate external resources depends on each agency’s existing capabilities and budget. 

Outsourcing other agencies might lead to quicker implementation while investing in in-house AI capabilities might be better for long-term control and customization of the offered services. 

Technical expertise

AI implementation demands specialized technical knowledge and expertise. 

Agencies may need to recruit or upskill their teams to effectively develop, deploy, and manage AI systems in line with the new regulatory requirements. 

To make the most of AI, team members should have:

• Good programming knowledge.
• Data processing and analytical skills for managing large amounts of data.
• Practical knowledge of machine learning.
• Excellent problem-solving skills.

Ethical considerations

Agencies must consider the ethical implications of AI use for their clients. 

Ethical frameworks and guidelines should be established to ensure responsible AI practices throughout the process, addressing the concerns raised in the updated regulations. 

These include:

• Transparency, disclosure, and accountability when AI is utilized.
• Respecting user privacy and intellectual property.
• Obtaining client consent to use artificial intelligence.
• Human control over AI with ongoing commitment to improve and adapt to emerging AI technologies.

Accountability matters: Meeting the legal challenges of AI implementation

While AI presents exciting opportunities for improving SEO and media practices, agencies must navigate legal challenges and adhere to the updated regulations associated with its implementation. 

Businesses and agencies can minimize legal risks by:

• Ensuring data has been obtained legally and the agency has the appropriate rights to use it.
• Filtering out data that doesn’t have the required legal permissions or is of poor quality.
• Conducting audits of data and AI models to ensure they comply with data usage rights and laws.
• Holding a legal consultation of data rights and privacy to ensure nothing conflicts with legal policies.
• Prioritizing transparency in data collection and obtaining user consent by clear and easy-to-understand consent forms.
• Using tools that help reduce bias, like AI Fairness 360, IBM Watson Studio, and Google’s What-If Tool.
• Implementing clear policies and procedures for reviewing the quality of AI-generated content before publication.

Search engine optimization techniques keep evolving, leaving many wondering what still works and which tactic is no longer effective. With the focus on guest blogging being so intense and the opinions about it changing so fast over the years, it’s no surprise that many are uncertain about its relevance for SEO today. As for the Google algorithm, everything is stable – having links from quality niche-related sites still counts as a digital thumbs-up that can enhance your search rankings.

Of course, there are multiple ways to earn those valuable backlinks besides guest blogging. Still, even a single post on the right website can significantly change your site’s position in SERPs. That’s why it is hard to imagine any SEO specialist not implementing this tactic. If that’s you, you probably also know the pain and stress of carrying out multiple monthly outreaches, hoping to get a quality backlink that can make a big difference. So, is it even worth it?

Why do many SEO experts still use guest-blogging sites?

You may have heard some argue that guest-blogging sites are bad for SEO. If so, you might wonder why most SEO experts still use them. Some confusion can be traced back to this post on Google Search Central, which appears to aim at large-scale article campaigns done to earn links. And we all know that most guest-blogging sites want to get you as many links as possible, which can be an exact recipe for disaster. But is this always the case?

A second look at the GSC blog referenced above shows that guest blogging is permissible under certain conditions. They must be educational and non-manipulative. The truth is that many SEO techniques are not entirely black and white; many are in the “gray” area. Is paying for guest posts against Google policies? Yes. Can you find sites to publish your blogs for free? You can try (hard). Can search engines know if you paid for the posts? Hardly. That’s why guest blogging is popular. It is effective, and it can be a Google-friendly tactic.

What are the downsides of guest posting you should consider?

Still, there are downsides to guest posting that you may want to consider. While efficient for those with the know-how, it is still easy for the newbies to make costly mistakes and get penalized. The first detail you should know is that you might not encounter any issues if you act according to a well-thought-out plan based on the best practices. You are definitely off the road when you use questionable and spammy tactics. And the only thing you will face in the long run is the drawbacks of your guest blogging activities.

What are those “spammy tactics” you should avoid? I’d say that some of the most widespread are the following:

• Using too many keywords in your anchor text or only using keywords for anchors.
• Ignoring the natural dynamic of acquiring backlinks. If you have a newly created website that gets 50 backlinks in a month out of nowhere, it can do no good for your site. There is no rule of thumb regarding the number of monthly backlinks. Still, most pages with high rankings get +5%-14.5% dofollow backlinks from new websites.
• Creating low-quality content for guest-blogging sites (including AI-generated texts).
• Forgetting about the natural and diversified backlink profile. You don’t only need dofollow links from super high-DA websites. Instead of ignoring nofollow links, create an organic mix, including at least web 2.0 backlinks (forums, social media, listing platforms, etc.).

Then, again, if you outsource your guest posting to someone else, whether it is a freelancer, an agency, or a specialized online platform, you have to make sure they know what they’re doing. And, of course, if the only experience you had with guest blogging was spammy and low-quality sites, you will be convinced that this SEO tactic can only result in problems without much else to do. By low-quality sites, I mean those that have:

• Low basic SEO indicators, such as traffic and its cost, DA/DR, etc.
• More outbound than inbound links (this probably means that they publish almost any content).
• A dedicated “Write for us” page that has all the information and prices for submitting a guest post.

Effective guest blogging: The best practices from renowned SEO experts

What makes for an effective guest blogging strategy? Here is what the experts have to say.

#1 Find valuable prospects

For this stage, you must deliberate about your search for high-quality sites that will let you submit a guest post. Here’s what you should look for: the domain authority, the relevant industry and their guest blogging history. Sites with a high DA rating in your niche will give you the best quality links and boost your search rankings. Otherwise, you will get the opposite results.

Now, the main question is, how do you find those relevant and valuable prospects? There are a couple of ways you can do that:

• DIY. Of course, as with anything else, you can roll up your sleeves and do it yourself. One of the difficult parts is finding the appropriate pages that accept guest blogs. Some options are analyzing where your competitors get the links, looking for reputable sites in your niche, and finding websites with the “Write for us” section. Then, you can gather those and reach out to them.
• Hire your own team. This is one of the best options, yet there is a huge BUT – you need big budgets. If you do, you can stop reading this blog right here and hire SEO analysts, outreach specialists, professional copywriters, PR experts, their team leads, etc. If they have a proven experience with great results, you will be golden (in theory).
• Hire a freelancer. This is a budget-friendly alternative for those who don’t want to do guest blogging themselves and can’t afford a full-scale SEO team. Still, if you ever had an experience hiring freelancers, you know that it might not be as easy as it seems. Finding someone who can do exactly what you need is pretty hard. Yet, nothing is impossible, so Upwork and Fiverr are waiting for you. Please remember that if something looks too good to be true on those sites, it probably is.
• Use directories with guest blogging sites. Imagine if you could simply insert all your search requests, including niche and DA and get a full list of potential pages where you could submit a guest post. Sounds like a dream for everyone who knows all the difficulties of finding the right prospects. Yet, guest-blogging services like this actually exist. And the beauty of them is that you get connected to outreach specialists who have established bonds with the sites you select. Besides, Adsy, for example, allows you to choose from several people who can contact the potential prospects in your name.

Whatever you pick, remember that your guest blogs should not be a part of your race to get as many backlinks as possible. They should provide value and be relevant to your business.

#2 Carry out a tactical outreach and personalized pitch approach

The word “tactical” evokes images of battle-ready and skillful maneuvers by warriors in combat gear engaging in some high-stakes action. The same can be said for your pitches during your guest blogging outreach (with a little less dramatism). You’ve researched and narrowed down the best likely prospects and even have the content topic ideas. But how do you get those sites to publish them? Your next step is to reach out to the potential target pages via email or social media. Take the time to grasp how to dominate their inbox with this powerful email marketing audit checklist that can open doors for more opportunities.

You want to be persuasive, stand out and be simply irresistible in your pitches. Ensure you share your ideas for a future blog post, explaining why you think their readers will benefit from that information and why you are the best person for the task. You might find many templates online, but this easy solution might not be the best. For example, Alexandra Tachalova thinks it’s better to go for a more personalized approach that shows your true interest.

#3 Create impactful, memorable content

Content quality is often repeatedly mentioned in blogs that explore ways to earn links from other sources, and that’s because it works. And there is no way you can skip this step. Content may or not be the king but the texts you write are the king, the queen and the royal family. This is why you need to invest time and resources in doing the necessary research that will help you create memorable content. And as much as you can share something new. How? Invite someone for an interview, conduct even small research among your clients/followers, or talk about your real-life case.

You also want to ensure that people will feel something after reading your texts (it is a hard part). Erica Schneider recommends that your content elicits some form of emotional response from your audience. If your readers can connect or identify with your post, it leaves a lasting impact and makes your brand more relatable and special. And, of course, if we’re talking SEO, emotions alone won’t get you far. So, don’t forget to optimize your content and choose your headlines wisely, as 85% of titles in the top five Google search results feature keywords.

#4 Look after your personal brand and company image

You might wonder what this tip has to do with effective guest blogging. Yet, if you are targeting some of the best guest-blogging sites, you must understand that the website owners will check who you are and what your company does (if they read your pitch). Sometimes, you might not get a response simply because your online profile doesn’t look appealing or doesn’t show your expertise. It is just an impression that has nothing to do with reality, but what we post online matters nowadays, whether we like it or not.

There is another great bonus you get when improving your social media presence. Some websites, podcasts or YouTube channels can reach out to you themselves. Everyone talks about personal brands this much today because it is a real deal. People love true personalities. So, the better image you create, the more opportunities you’ll get. Don’t try to be picture-perfect, though.

#5 Don’t overdo it

Whatever you think about guest posting, it can be an ace up your sleeve or a complete disaster that will harm your overall SEO efforts. As doctors love to say – everything is fine in moderation. Treat your guest posting the same way, and you will never be spammy or go against Google policies.