The best SEO tactics aren’t high-level, complex theories, but simple, actionable insights.

All the pivot tables in the world won’t save your bacon if you fail to drive results. And results come from outputs, from action, not analysis. So close out those multi-tab spreadsheets and roll up your sleeves.

This short, simple article will help get you back to basics right in time to kick-start growth for next year.

How to drive SEO results in the next six months

SEO is a long-term game. That’s why your results next year will often come from what you’re doing this year. But what if that’s not soon enough, not fast enough or just not enough in general?

If that’s the case, you need to adjust your payback period and the strategies or tactics you pick as a result. With SEO, you front-load an investment over the first few years to create an asset with a defensible moat that appreciates in value over time. 

Could you set your sights high, then? Should you dominate the top commercial, money-making term in your space?

Definitely!

There’s only one problem: you’re probably looking at years and millions to make it happen. That won’t help you today, next week or next month.

Instead, try shifting your attention from what you don’t have and don’t rank for, to what you do have and are already ranking for. 

Here’s how.

1. Update and re-optimize content ranking outside the top five 

The vast majority of ranking content doesn’t get seen, read or clicked.

Only about the top five results on a SERP get roughly 80% of the love. (See nearly every SERP CTR study ever.)

That means two things:

1. SERPs (and SEO by extension) are a zero-sum game. To win, others must lose.
2. This means you need to win at all costs – prioritizing top five rankings ASAP.

Contrary to popular belief, new content won’t help you much here because will take weeks-to-months-to-years to go from non-indexed to top five.

Instead, start by looking at all existing content ranking in positions five to 15. Throw in a minimum volume floor to focus on the biggest bang for your buck. Then, set a keyword difficulty ceiling to prioritize the “lowest-of-low” hanging fruit. 

Voila.

Now you’ve found some easy content pieces to update. More importantly, you’ve identified a few different scalable content ideas where you’re already showing some topical authority to help increase the time to rank new content when you’re ready to triple down on new output.

2. Update and re-optimize content showing keyword cannibalization

You’re already ranking for some keywords. That’s great! But here are the common problems:

• You’re picking up rankings you shouldn’t – where you’re optimizing for one topic and sort of ranking for another, which means…
• You’ll likely never crack into the top five with that piece of content for that keyword because the competition outranking you is properly optimized for said query, which also means…
• You probably already have content you want to rank for that keyword, except it’s ranking potential is being cannibalized by that other existing page on your site.

Here’s an example:

The page in green (“/cat-health-insurance/”) is ranking for “pet insurance for cats.” That’s good! Hence, the green. But look at the terms in yellow. There are four keywords and two pages. That’s not a great sign unless you can prove significant SERP overlap.

The first keyword is branded and blurred. (We can’t share all our secrets.) Yet it’s the homepage (“/”) ranking for a branded keyword, which is fine. However, the same homepage also ranks for top unbranded or non-branded keywords (like “pet insurance” or “dog pet insurance”). That’s bad!

Unless those SERPs are already filled with homepages, which they aren’t, then you have keyword cannibalization screwing with the pages you actually want to show up here.

Right on queue, look at the last one in yellow, the “/dog-health-insurance/” – which is being cannibalized by the homepage. Where there’s smoke, there’s fire. 

If this is happening in a case like this, it’s almost certainly happening across many other areas of your site, too. No worries! Here’s all you have to do: 

This is a super easy, five-step process that can quickly (in about zero to six months) pay dividends.

3. Update and re-optimize content that used to rank well but has lost rankings in the last 1 to 2 years

SERPs are living, breathing things. They’re not static. They always evolve as searchers become more nuanced and information changes over time.

Your competition for these SERPs is also evolving at the same time, upping their game and throwing more money plus talent at the same rankings you used to covet so dearly.

The result?

After a year or so of good rankings, you get complacent. Your attention shifts to other areas. You very quickly will start to slide backward into SERP obscurity.

• This is extremely common, so expect and embrace reality.
• It’s happening all the time, literally, as we speak.
• And it’s also easy to spot and fix!

Once again, look up your existing keyword rankings. This time, add a date filter to capture the historical rankings from the last year or two. Then, filter by the keywords and/or pages with the biggest drops during that time period.

Exhibit C:

The fact that you used to rank well for this keyword is another positive underlying sign. It hints at some level of topical authority and popularity around these relevant terms.

And once again, updating and re-optimizing these pages – before worrying about publishing any brand new content – will likely yield a better ROI over the next three to six months than virtually anything else you could be prioritizing at the moment. 

To get faster results, take action now

SEO is an art and science. It can be technically challenging and confusing at times. The trick is not to lose the forest through the trees. All the thinking in the world doesn’t make up for inaction.

Resist the tunnel-vision urge, pouring endless hours into analyzing, strategizing and talking yourself in circles. Instead, if you need to drive results ASAP, get back to the basics.

Adjust your keyword research, content creation and link building efforts to fit a more appropriate short-term payback period. Double down on what you’ve already got or used to have, and let that help you drive results over the next few months.

Today’s work will also have the added bonus of buying you time to shift your attention back to more ambitious, commercial and lucrative projects that might not pay off for at least a year.

An effective SEO strategy must evolve over time to account for changes within your company and the broader digital landscape. An annual review is key to keeping your approach up to date and results-driven. 

This article outlines critical internal and external elements you need to assess when refreshing your SEO plan, from website changes to competitive shifts to algorithm updates.

You can evolve your SEO strategy with continuous shifts of priorities and changes that are reflected in a dynamic SEO roadmap, or you can develop a more defined annual or semi-annual process.

For this article, we’ll work under the assumption this is an annual review to determine if priorities have changed in the last year (i.e., waterfall, rather than agile, for those who think that way).

Internal elements to review

A year is a long time, and things can change internally, whether they are processes or priorities.

Before we shift our SEO strategy based on an external factor, we want to make sure the brand and the company can support it – and that it matters to the business. 

Your SEO strategy implementation

Start by auditing what has successfully been implemented or what projects have been approved – from the first iteration of the SEO strategy translated into tactics through an SEO roadmap – and what has struggled.

If possible, you’ll also want to see what impact that resulted from those changes. This will allow you to start mapping out the particular elements that impact your website and potentially your industry.

Your website

As one of the key elements of your SEO strategy, you must understand any changes on your website’s backend.

If you’re lucky, you’ll be familiar with this anyway, because you’ll either be embedded as a part of the development team or across their workflow and on good terms with the product or program owners. 

You also need to understand what’s changed on your website from a content perspective. A number of enterprise-level SEO tools have change records as a part of their software, or you could use tools like: 

• Visual Ping
• Hexomatic
• Sken.io

Otherwise, you may be able to use the “Last modified” or “Last edited” history within your CMS if those fields are recorded.

Your products/offering

Your product and your company go through changes as well. Maybe this year, there was a fundamental shift from focusing on product A to launching and supporting product B. 

Or maybe the goals, targets and mission of the business have shifted, and it’s less focused on gaining new customers, but about retaining existing business. 

That fundamentally could shift both the content strategy and the technical focus of the website and should inform an evolution of your SEO strategy as well.

Your processes

Maybe a process has been improved and unblocked a lot of work you couldn’t get done in the last year, or potentially the reverse. 

While not exhaustive, these are good places to start understanding how things within your control have changed in the last 12 months and how they’re likely to change in the next 12. 

External elements

Understanding external changes involves assessing factors beyond your control. Determine whether immediate action is needed or if it’s better to address the issue before it becomes critical.

This may lead to a broader business discussion requiring leadership’s approval, so be prepared for a potentially time-consuming process.

Google algorithm updates or feature changes

This is an obvious one, but it’s worth stating: SEO strategies, to a certain extent, will always be reactive.

While a good SEO strategy will be near-impervious to shifts in Google’s algorithms, the evolution comes in anticipating and acting toward future changes in how people search. It’s seeing things that may be a small consideration now, but you see the threads becoming bigger, like the “Experience” in E-E-A-T or Google’s SGE. 

What your competitors are doing

While I’m a follower of the infinite game in business, it doesn’t mean you shouldn’t pay attention to what happens in your market. Instead, it means not measuring your success by it. 

If you find that most of your market has shifted to writing long-form, thought-leadership content, that’s at least worth tucking away in the corner of your mind. But whether you evolve your SEO strategy to address it depends on your wider goals. 

Understanding the market landscape is different from reacting to it, and any evolution of your SEO strategy needs to be informed by an accurate, timely understanding of the market where you operate.  

Chaos

One recommendation that has always stuck with me from Ian Lurie’s various strategy documents is to consider the chaos. 

• What would happen if a new channel came in and disrupted the market? 
• What if there was a global pandemic? 
• What if you get fantastic media coverage? 
• What if the iPhone had a split launch? 
• What if Samsung had to recall its phones? 

This thought exercise stretches your strategic muscles. It also helps you understand the edge cases and assess how resilient your processes and tactics are. Note how these edge cases would require you to shift or entirely change your strategy. 

Look to the future

The evolution of the SEO strategy should include: 

• Updating SEO goals and targets.
• Shifting any strategic pillars that are either internal or external reactions.
• Adding new strategic pillars based on the market landscape.

Armed with this up-to-date information, you’re well-positioned to evolve your SEO strategy and future-proof it by keeping the document live and loved. 

Google Search supports discussion forum and profile page structured data. This functionality allows Google to show “first-person perspectives from social media platforms, forums, and other communities” within the search results.

In addition, Google added new Search Console reports to support those who added structured data to their pages, the company announced.

Profile page structured data. The new profile page structured data and markup is for any site where creators, either people or organizations, share first-hand perspectives. This markup will help Google highlight the creator’s name or social handle, profile photo, follower count, or the popularity of their content in the Google search results.

This information is also available in the Perspectives and Discussions and Forums search result features.

Here is what it may look like:

Discussion forum structured data. The new discussion forum structured data and markup is for any forum-style site where people collectively share first-hand perspectives. “When forum sites add this markup, Google Search can better identify online discussions across the web and make use of this markup in features such as Discussions and Forums and Perspectives,” Google explained.

Here is what it looks like:

Changes to Q&A markup. Google also updated its Q&A markup to say if you are already using Q&A markup for your question and answer user forums. The company is updating the Q&A structured data documentation to be more in line with the richness of the new discussion forum guidelines. You don’t need to use both types of structured data on the same page. You should instead use the one that’s best suited to your use case:

• Q&A forums: If your forum is structured by a question that’s followed by answers, use Q&A markup.
• For general forums: If your forum structure is more general and isn’t strictly question and answer content, use DiscussionForumPosting.

Search Console reports. Google added new Search Console reports, which maybe why the tool went down earlier today, to help site owners monitor discussion threads and profile page markup issues. These reports show you valid items related to your marked-up pages. The company also updated the Rich Results Test to help you test and validate your markup.

Here are screenshots of those reports:

Why we care. With the rise of the new personalized search features, including how Google is promoting creators more in search through perspectives and other areas, it may make sense to implement these new structured data types. Google probably won’t support these elements forever but you may get a higher click-through rate if you add these to your pages.

Google Ads mistakenly sent out an email telling some advertisers that their accounts had been suspended.

The notification, which was triggered by a technical fault, was issued just hours before the Thanksgiving holiday, causing some marketers to “freak out”.

Google confirmed the error shortly after and reassured advertisers to disregard the message.

Why we care. If you or your client received this message, rest assured that your account remains active, and your business will not be impacted. This notification was sent in error and is of no concern.

‘Clients freaking out’. The error messages were first flagged by PPC expert Kirk Williams on X. He told his followers:

• “We’re suddenly getting clients freaking out that their GMC got suspended by this email (not true, we send them back screenshots of GMC and their ads in the wild to assure them). Anyone else receive these?”
• “Seems to be some generic form email. What in the world would possess someone at Google Ads to send an email like this hours before the Thanksgiving holiday to brands so they all (rightfully) freak out thinking GMC dropped their products, only to realize this seems to be an error or promotional email?!?!”

What Google is saying. Once aware of the problem, Google Ads liaison Ginny Marvin advised her followers on X to ignore the warning, explaining:

• “We’ve identified a technical issue that incorrectly triggered this notification. The message is incorrect and can be ignored. Additional comms explaining will be sent. Apologies!”

Deep dive. Read our Google Ads Best Practices guide by Search Engine Land contributor Rémi Kerhoas for more information on using Google Ads.

Technical SEO refers to the process of improving a website’s architecture and backend elements to improve its visibility and performance on search engines.

Tackling such SEO projects often requires collaboration between technical SEO specialists and web developers, especially for large sites.

Technical SEO examines page load speed, site architecture, security, internal linking, etc., to eliminate friction points that might hinder search bots from properly crawling, indexing and rendering a website.

Technical SEO vs. on-page SEO vs. off-page SEO

In a nutshell:

• Technical SEO is a process designed to maintain your website’s technical health. 
• On-page SEO revolves more around keyword research and content efforts.
• Off-page SEO involves activities, such as link building and digital PR, that aim to boost the external authority signals pointing to your website. 

Technical SEO focuses mostly on the code-level deployment of your website to improve crawlability, allowing for quicker and more extensive content indexing.

To some extent, technical SEO elements overlap with on-page SEO. For instance, an on-page SEO specialist examines webpages and makes changes to metadata items (i.e., title tags, meta descriptions, H1s, etc.) based on keyword research.

A technical SEO specialist then focuses on factual code errors related to metadata. This includes issues like overly long page titles or missing meta descriptions without the task of writing or creating metadata for each element.

Another area of overlap is internal linking. On-page SEO focuses on adding internal links strategically to boost the link equity of particular pages. Conversely, technical SEO approaches internal links from a broader perspective, focusing on code errors and overall site link architecture.

Technical SEOs conduct a thorough site crawl to identify broken links and provide recommendations for resolution. Unlike on-page SEO, technical SEO is less concerned with the specific anchor text of individual links and prioritizes the overall health of the site’s link structure.

Technical SEO and off-page SEO are distinct practices, typically attracting different skill sets. While they generally operate at opposite ends of the SEO spectrum, there is some overlap, particularly in link auditing.

In the past, this involved assessing your backlink profile for harmful links and disavowing them to avoid penalties. However, recent critiques from Google’s John Mueller have cast doubt on the efficacy of disavowing toxic links.

Link auditing has shifted to a more positive approach, focusing on evaluating your links and those of competitors for “best in class” examples. After this analysis, conclusions are drawn.

For instance, if competitors get strong links from events, the goal is to get you to those events. Armed with these insights, the digital PR specialist aims to generate real-world activity, securing high-tier editorial coverage.

To recap:

Technical SEO

• Focuses on website’s technical aspects and code-level optimization.
• Aims to enhance the site’s crawlability and overall technical health.
• Addresses issues like metadata errors, broken links, and site architecture.

On-page SEO

• Revolves around optimizing individual webpages’ content and elements.
• Involves keyword research, content deployment, and metadata optimization.
• Typically examines on-page factors like page titles, headings, and internal linking.

Off-page SEO

• Concentrates on building external signals and authority for the website.
• Often referred to as digital PR, it seeks to boost the site’s online reputation.
• Involves activities like link building, content promotion, and earning editorial coverage to improve rankings.

Technical SEO’s role in an overall SEO strategy

Technical SEO is crucial in overall SEO strategy. Simply put, other efforts are futile if a site has significant technical issues or is not indexable.

In a comprehensive SEO campaign, the technical SEO audit is typically done first and is often the most extensive. This means technical SEO is in play right from the start for practical reasons.

If a website has issues hindering indexing, other efforts become ineffective. The on-page SEO team often depends on aspects of the technical SEO audit to guide their initial actions. 

For example, if there are large chunks of the site with no on-page content, meta descriptions or H1s, that must be tackled. It’s only once the house is so that an on-page SEO team will perform their own analysis (usually based on keyword and/or ranking data) to champion new site sections to build out.

Likewise, the off-page SEO team often waits for the technical SEO person to complete competitive link auditing. Typically, the technical SEO audit comes before analyzing competitor links. 

This places technical SEO at the center of the SEO campaign, especially in the early stages. As the campaign progresses, the technical SEO team uses tools like Ahrefs or Lumar to monitor site health, but their involvement is usually reduced by then.

Takeaways:

• Technical SEO is a crucial component of an overall SEO strategy.
• Other SEO activities like content and link building are ineffective if a website has significant technical issues.
• In a comprehensive SEO campaign, the technical SEO audit is typically the first step and the most extensive one.
• Technical SEO plays a central role in the early phases of an SEO campaign.
• On-page SEO relies on the technical SEO audit to address issues like missing content, meta descriptions, or erroneous H1 tags.
• Technical SEO monitoring is done over time using tools like Ahrefs or Lumar.
• A technical SEO’s skills are in constant demand, and they often serve as a technical bridge between various aspects of SEO.

Common technical SEO activities

Here’s a list of activities commonly completed by technical SEO professionals.

Accessibility and indexing checks

• Uncovering and tackling page indexing problems using tools such as Google Search Console.
• Ensuring that all indexing signals (e.g., robots.txt, no-index tags, canonical tags, XML sitemaps) agree with each other, without sending contradictory signals to Google.

Content management system (CMS) tasks

• Engaging in some broader CMS work, if needed. (Smaller, single-page issues might be offloaded to on-page / content SEO teams.)
• Deciding which CMS to migrate to if the current CMS isn’t very SEO-friendly.

Crawling

• Ensuring proper website crawling using industry-standard tools such as Screaming Frog.
• Analyzing crawl data to power technical SEO audits.
• Understanding how best to configure crawlers (on a site-by-site basis) and monitoring the crawl. (No one wants to accidentally DDoS their client with crawling software!)

Domain work and migrations

• Deciding on which new domain name to select for SEO benefit. 
• Carrying out large-scale website migrations, which may be domain external or domain internal.
  • A domain external migration involves moving a website (and its associated SEO rankings) to a new domain. 
  • A domain internal migration involves moving to a new CMS (which writes URLs differently) within the same domain (redirect work is still needed). 
• Tackling non-migration-related redirect work, such as the de-tangling of legacy redirect chains.

Metadata analysis and issue scoping

Top-level site security

• Understanding and implementing HTTPS / HSTS header responses to keep website users safe.
• Deploying content security policies, blocking external embeds via X-frame options, etc.

Structured data

• Deploying schema or structured data which Google uses to enrich their view of a page’s content, boosting its chances to appear for rich snippets in the SERPs. 

Usability and site speed

• Working with web designers or front-end developers to ensure that a website is SEO-friendly.
• Dissecting page speed performance issues and checking whether the site passes Core Web Vitals assessment.

Internal link architecture and wider site architecture

• Addressing internal linking errors (e.g., a nav link pointing to a redirect, resulting in tens of thousands of links to redirects on your site).
• Managing wider site architecture issues (e.g., faceted navigation filters spawning tens of thousands of URLs that need to be canonicalized).

Advanced technical SEO activities

These activities require a more seasoned technical SEO professional to tackle.

Advanced page speed checks and JavaScript analysis

Improving page speed used to be simple – compress some images, minify your scripts, and get good speed scores. But things have changed.

Today, most speed problems are tied to congestion in your users’ web browsers. Now, we optimize our websites to deliver serve content the fastest way possible, even on devices with limited capabilities.

The main issue here is that most web browsers only (by default) execute JavaScript within their “main” processing thread, meaning all JavaScript must be executed in sequence. This causes page load bottlenecks, even when JavaScript is quickly shipped to your end user. (It’s like firing a fire hose at a one-inch gap in a brick wall.

Technical SEO experts should know how to check the inside of a browser using tools like the Chrome Dev Tools Performance tab. You should be able to discuss these issues in a way that a senior web developer can grasp. An advanced technical SEO person is comfortable talking about technologies like JavaScript parallelization or server-side rendering.

Log file analysis

Log files reveal which pages and resources crawl bots have asked for and how often. Examining patterns in this data can uncover interesting anomalies.

For instance, you might observe that a specific part of your site isn’t getting any Googlebot requests. Why? It could be due to a problem with a robots.txt rule blocking Google from crawling that section.

Log file analysis is valuable. It has been part of the technical SEO toolkit but gained mainstream attention with tools like Screaming Frog’s Log File Analyzer. Despite this, it’s still seen as an advanced technical SEO method.

Edge SEO

Edge SEO is a subset of edge computing, a practice that involves making technical and architectural changes to process data closer to the source or the end user. 

This reduces latency, making technology faster, unlike cloud computing, which relies on centralized data centers, less efficient for many users.

Edge SEO is just applying edge computing with SEO in mind – for instance, aiming for faster-loading pages that may rank better on Google.

Here’s a list of some common technical SEO tools. From indexing checks to full site crawls, these are the best tools for the job. (While this list doesn’t include all technical SEO tools, it covers the fundamentals.)

Google Search Console

GSC is used for accessibility and indexing checks and other wider applications, such as viewing which Google keywords supplied the most clicks and impressions. 

Search Console can also check your schema implementation and Core Web Vitals adherence across your site. The best part? Google supplies this tool for free, so verify your website.

Screaming Frog SEO Spider

Screaming Frog is a crawler tool for website analysis that runs from your desktop environment. The size of the crawls that this can handle is only limited by the power of your local machine. 

You must keep an eye on the crawl to ensure you aren’t crawling too quickly, causing your client’s site to time out. 

It is also great at spitting out lots of data, though typically, the output will require more handling on your end.

Ahrefs 

Ahrefs’ Site Audit is great for quick top-line analysis. It crawls a site and generates a user-friendly dashboard highlighting technical SEO issues for prioritization. Compared to Screaming Frog, Ahrefs is more user-friendly. 

It’s often recommended to use a deep crawler like Screaming Frog alongside a cloud-based crawler like Ahrefs to combine detailed data and insightful analysis.

Lumar (formerly Deepcrawl)

Lumar is probably the best compromise crawler between Screaming Frog’s data spitting and Ahrefs’ more actionable insights.

Chrome DevTools

Chrome Dev Tools is particularly useful for its Performance tab and Lighthouse module, which can help to examine page speed issues in depth on an individual page basis.

The embedded Lighthouse module is foundation technology, which is utilized by the (separate) PageSpeed insights tool.

PageSpeed Insights

PageSpeed Insights is a great tool to perform a deep dive into page-speed issues on a single webpage. It can tell you whether a page passes or fails the Core Web Vitals assessment, should sufficient field data (from the CrUX / Chrome UX report) be available.

Tip: Don’t forget to expand the diagnostics area of this tool’s reports. There’s some very interesting stuff in there, which will help you to dissect issues regarding a clogged main thread.

Screaming Frog Log File Analyzer

Screaming Frog can help you understand crawl bot behavior and identify potential crawl issues.

It’s simpler than exporting raw log files and creating complex spreadsheets for analysis. The tool is cost-effective annually, making it a valuable asset. 

Microsoft Excel or Google Sheets

Most of the other listed tools can provide exported snippets of data in Excel or CSV format.

Without a spreadsheet tool that can import and manipulate that data for further analysis, they would be useless.

Large-scale CSV manipulation applications

Sites with millions of internal links might overwhelm Excel when exporting data from Screaming Frog.

Fortunately, Screaming Frog can export to CSV, and tools like Delimit by Delimitware and ModernCSV are great for handling large data volumes. You can then trim the data to make it compatible with Excel. 

Thinking of a career in technical SEO?

If you have done your time as a junior SEO associate or an SEO executive, chances are you’ll soon be looking to specialize. 

Are you thinking of a career in technical SEO? Here are some of the skills and traits you will need:

• Technical web expertise: Proficiency in understanding and optimizing website code, structure and architecture.
• Proficiency in auditing techniques: If the thought of spending multiple days on single audits bores you to tears, technical SEO is not for you.
• General awareness of HTTPS and HSTS adherence for top-level website security.
• Familiarity with structured data.
• General understanding of user experience and its relationship with SEO.
• Analytical thinking: The ability to analyze data and identify technical issues.
• Problem-solving skills: Capability to diagnose and solve complex technical challenges.
• Strong communication: The skill to communicate technical issues and recommendations to non-technical team members.
• General data analysis: Proficiency in data analysis and the use of spreadsheets to extract insights.

Technical SEO is an excellent career choice that I am very happy with.

Technical SEO: The backbone of a robust SEO strategy

Technical SEO is a crucial component of any effective SEO strategy. 

By optimizing a website’s infrastructure and eliminating friction points that hinder crawling and indexing, technical SEO lays the groundwork for other efforts like content creation and link building to be successful.

Technical SEO specialists address critical issues from site migrations to page speed, metadata, internal linking, and more using the right tools and auditing techniques. 

Requiring a specialized skillset, technical SEO is an exciting career path for those with the analytical abilities and technical know-how to diagnose and resolve complex website issues.

More than 250 websites now block Google-Extended, the “standalone product token” Google introduced Sept. 28 to let you block Bard, Vertex AI generative APIs and future generations of models from accessing your content.

That’s according to research shared exclusively with Search Engine Land by the Detailed.com team.

Why we care. There has been much debate and discussion about whether brands and businesses should block any bots (e.g., GPTBot, CCBot) that crawl content that is then used to train LLMs. Only a minority of sites have decided to block so far, but the numbers have continued to climb in recent months – as they don’t want their content to help AI companies profit and compete against them.

A big increase. As of Nov. 19, 252 websites out of a set of 3,000 popular websites had blocked Google-Extended. Just over a month earlier (Oct. 8), only 89 of those sites had blocked Google-Extended.

• That means the number of sites blocking Google-Extended jumped 180% in the past month.

Websites blocking Google-Extended crawling. They include:

• Ziff Davis properties (e.g., PC Mag, Mashable).
• Vox properties (e.g., The Verge and NYMag).
• The New York Times.
• Condé Nast (22 sites, including GQ, Vogue, Wired)
• Yelp (frequent Google critic and legal opponent).

Reminder. While you can block Google-Extended in robots.txt, that does not block your content from appearing in Google’s Search Generative Experience or prevent Google from using your content from training SGE. To opt-out fully, you’d have to block Googlebot (which would also take you out of Search). However, you can opt out of SGE overviews using nosnippet.

Google Search Console went down at about 8 am ET today and is still down at the time this story was published. Search Console was down and inaccessible for about 50 minutes. The tool went down at about 8 am ET and returned at 8:52 am ET.

Google is aware. At about 8:30 am ET Google’s Daniel Waisberg replied to some concerns on X with a bit of a joke, he said, “I was looking for a plug to charge my phone, so I had to unplug the server.” “Joking aside, we’re working on it,” he added.

Either way, we know someone who works with the Search Console team is aware of the issue and was able to get the team to fix the issue. Waisberg posted at 8:57 am ET saying, “Should be all good now.”

What it looks like. When you try to access Google Search Console you are presented with this 500 error status:

Why we care. If you are doing any reporting for clients or yourself, then you might want to check back later. The tool is down for everyone in every location, so it is not just you.

Maybe go work on other SEO-related tasks while Google works on restoring Search Console.

Postscript – it’s back. Google has fixed the issue at 8:52 am ET. You should now be able to access Google Search Console, it seems to have been fully restored after a 50 minute outage.

High-potential keywords are phrases that bring relevant traffic to your website and lead to revenue. While Google Keyword Planner and SEO tools are useful, they focus on search volume and intent, not necessarily revenue. 

To find keywords that actually lead to sales, it’s important to explore other sources that combine website traffic with actual conversions. 

Let’s look at what conversions to consider and then explore where you can discover keywords with high conversion potential tailored to your business, whether you’re a publisher, a B2B brand or a seller of consumer products.

Keywords geared toward conversions

Conversions for SEO include:

• Sales
• Leads
• Email and SMS subscriptions
• Higher pageviews that drive CPM revenue
• App and ebook or information downloads
• Phone calls
• Live chat sessions
• Free trials

If you get someone on your email list because you provided solutions, you can bring them back to shop once they’re ready. And as they’re opting in, you can ask about interests. 

If you sell gardening supplies, find out if your customers are into tools, indoor or outdoor gardening, and whether they prefer growing vegetables, flowers, or succulents. Knowing this helps you tailor your marketing to their interests. 

If you track customer lifecycle metrics, you can see that SEO attracted them initially, and email was what persuaded them to make a purchase. This approach also works for other indirect conversions, like those from social media, PPC ads or blog content.

And this is what leads us to find these high-conversion phrases. Look at non-SEO tools and data providers first.

Referring URLs

We all see the traffic backlinks we get naturally, the ones that we’ve built, and what our affiliates send us. And our companies are tracking the conversions from them. If they aren’t, start today while following all tracking laws.

If these referring URLs have traffic that converts, there is no reason you cannot try to replicate and beat them at their own game. 

Media companies advertise their own content and bring traffic to their high-conversion articles so they can make money. You cannot mimic that through SEO, but you can advertise on those pages.

Take the referring URL and plug it into your favorite SEO research tool. You’ll get a list of the keywords and phrases they show up for. Now, compile the ones most like the reasons you’re getting sales. Sometimes you’ll see zero search volume, but guess what, there actually is. 

Tip: Check out the backlink profile and, if you can, social shares. If there is no SEO traffic, the traffic has to be coming from elsewhere and converting. Creating better content and a better UX allows you to get all the traffic vs. the referral.

Customer surveys

When was the last time you surveyed your customers? If you never have or haven’t asked the customer service and PR or branding teams for the data, you’re missing out on a gold mine. 

Customer surveys let you know what solutions the people who make you money need and what they’re interested in.

Here are some questions to ask and how they can help you find high-intent SEO keywords.

What did our product not solve? 

If customers tell you what they need, and you address those needs, you can create blog posts, update product information and run ads to reach people seeking those solutions. 

If you haven’t offered this solution before and now you do, there’s a good chance that people actively looking for it will be more likely to purchase it.

Dig deeper: How to turn polls and surveys into great content

What do you do for fun? 

By knowing the person’s interests and seeing that a large percentage of your audience has the same, you can do keyword research and create a separate content funnel. Just ensure the content is topically relevant to your products or service, or you can do more damage than good.

If the topics aren’t relevant, but something you can write about, create a new entity like a blog with a forum/community (once the blog takes off) and use it for lead generation. 

Now, you can beat the places they go to engage for fun and remarket your main brand through PPC to bring them into your conversion site for revenue. 

Which websites do you visit for fun or for keeping up to date?

Even if they say mass media sites with a million categories, that is still beneficial for keyword research. 

• Reader demographics: You can find political swings, along with age, gender, income, and other skews.
• Relevant categories: Keep track of the evergreen topics covered by these sites and begin developing a content strategy as long as the topics stay relevant to your core offerings and topics.
• Advertise: Look for where they are on these sites and show ads on relevant topics to your brand and with your audience demographics in mind. If you can track the referring URL, see the topics and keywords that the page shows up for. This can result in more keyword data for you.

PPC data

I left this for last because it involves the strategies we discussed earlier. Instead of just asking about the keywords your PPC team is using (since they shouldn’t spend on unproductive traffic), also inquire about where the ads are being displayed.

Google Ads and similar platforms allow you to check where your ads appear, such as on YouTube videos and websites. If you’re getting a good number of conversions, look at the topics that brought those conversions to your site.

YouTube videos can be a goldmine for article and keyword ideas. If the videos have chapters, consider creating a series of content about each chapter. If they showcase a feature that could lead to a conversion, check if you have that covered on your product or service page.

When you get the list of referring URLs that drove at least XYZ conversions, use the research strategy from the first section. 

Dig deeper: 7 reports SEO and PPC can use to help each other succeed

Don’t start with the SEO tools for high-intent and high-conversion keywords. Those are all about search volume and intent. 

Beyond SEO tools, explore customer surveys, referring URLs, and PPC data for strategic insights. Aligning keywords with conversions is the key to driving revenue. 

Shift your focus, embrace strategic approaches and watch your business thrive.

Google is deprecating the crawl rate limiter legacy tool within Google Search Console on January 8, 2024. Google said this feature is no longer useful because Google said it has improved its “crawling logic and other tools available to publishers.”

What is the crawl rate limiter. It is a tool within the legacy version of Google Search Console that lets you communicate to Google crawl your site less than it currently does. Google has historically recommended against limiting the crawl rate unless you are seeing server load problems that are definitely caused by Googlebot hitting your server too hard.

You can access the tool over here, until it is removed. Here is what it looks like:

Why Google is removing it. Gary Illyes from Google said, “with the improvements we’ve made to our crawling logic and other tools available to publishers, its usefulness has dissipated.”

“Googlebot reacts to how the site–or more specifically the server handling the site– responds to Googlebot’s HTTP requests. For example, if the server persistently returns HTTP 500 status codes for a range of URLs, Googlebot will automatically, and almost immediately slow down crawling. Similarly, Googlebot slows down automatically if the response time for requests gets significantly longer. If you do experience unusually heavy crawling that your site can’t manage on its own, refer to this help article,” Gary Illyes added.

Illyes explained that the rate limiter tool within Google Search Console “had a much slower effect.” He said the tool often would “have taken over a day for the new limits to be applied on crawling.”

He added that the tool was “rarely” used by site owners and those who used it “in many cases set the crawling speed to the bare minimum.”

Crawl rate change. Google said with the deprecation of the crawl limiter tool, Google is setting the minimum crawling speed to a lower rate, comparable to the old crawl rate limits. So Google will “effectively continue honoring the settings that some site owners have set in the past if the Search interest is low, and our crawlers don’t waste the site’s bandwidth,” Illyes added.

What if you have issues. If you have issues with crawling, Google said you can read this help document and use the report form to let Google know.

Why we care. If you have been using this crawl rate tool, keep in mind it will be going away. So set a notification on your calendar to see what, if any, impact this has on your server when the feature is turned on.

It’s been nearly a year since Google last updated its Search Quality Rater guidelines. 

Unlike previous edits to the Search Quality Guidelines, which have introduced significant, new concepts (like the new E for Experience last year), the latest updates to the Search Quality Guidelines seem much more focused on user intent and needs met. 

Google is:

• Refining what it means to provide high-quality search results.
• Helping quality raters understand why certain results are more helpful than others. 

This level of nuance can help explain why we see certain volatility during core updates (as well as periods outside of announced algorithm updates).

If search quality raters have given Google ample evidence that its results are not meeting user expectations, this can lead to substantial intent shifts during core updates. 

Looking at search results for the same query, before and after major Google updates, makes a lot more sense when you understand the granularity with which Google approaches understanding the intent behind a query, and what it means to have high-quality, helpful content. 

Here are high-level insights into what has changed in the latest search quality rater guidelines update.

More guidance around rating page quality for forums & Q&A pages

Google added a new block of text to instruct raters on how to rate the quality of forum and Q&A pages, specifically in situations where the discussions are either brand new, or drifting into “combative,” “misleading” or “spammy content.” 

A forum page defaults to “medium” if it’s simply a new page that hasn’t had time to collect answers. But older posts without answers should be rated as low quality. 

Google mentions “decorum” a few times in this section, indicating that combative discussions that show a lack of respect should be rated as low quality. 

• This is a good reminder that the quality of comments on a given page can impact the overall quality of that page’s content, assuming Google can crawl and index the content in the comments. Often, comment sections are neglected or unmoderated, and if they become problematic, insulting or disrespectful, this can negatively impact an otherwise good-quality page.

Additionally, the new version of the Search Quality Guidelines includes a visual example of what a “medium” quality forum page looks like on Reddit. The question is only 9 hours old and has no answers, so it defaults to a score of “medium.”

It’s worth noting that Google implies there are no other low-quality characteristics on this page that otherwise could lean the score towards “low quality” for new discussions. 

Below is the example Reddit URL Google used in this case: 

A quick addition about the importance of a location to a query

Google added a short snippet about the importance of user location to understanding a query. For searches looking for nearby places, location is important, whereas generic questions like “how does gravity work” have the same answer, regardless of the user’s location. 

• Interestingly, Google felt this nuance was worth adding to the Rater Guidelines. While it seems self-explanatory, it’s true that the extent to which a search query has a built in “local intent” can have a major impact on the types of results that would best answer that query. 

Expanding on “Minor Interpretations”

Google added a deeper explanation about its definitions for “Minor interpretations.”

Minor interpretations describe a situation when a query can have multiple meanings, and the minor interpretations are the least likely to be the commonly expected meanings of the query.

Within minor interpretations, Google introduced:

• “Reasonable minor interpretations,” which help “fewer users” but are still helpful for search results. 
• “Unlikely minor interpretations,” which are theoretically possible but highly unlikely. 

“No chance interpretations” are interpretations of a query that are incredibly unlikely for the user to be looking for. Google provides the example of an “overheated pet” when the searcher types “hot dog” (although I feel that this interpretation is more plausible than a “no chance” rating!).

Google also added some new visual examples of how to interpret these definitions. For example, an “unlikely minor interpretation” of the search query “Apple” would be the U.S. city, Apple, Oklahoma.

Further defining ‘Know Simple’ and ‘Do’ queries

Google added several new examples of what types of queries are not “Know Simple” queries.

“Know Simple” queries are defined as queries that seek a very specific answer, like a fact or a diagram, that can be answered in a small amount of space, like one or two sentences. 

Google added three new examples of queries that are not Know Simple queries: when users want to browse or explore a topic, find inspiration related to a topic, or are seeking personal opinions and perspectives from real people.

What makes this addition interesting: The language used here is quite similar to the language Google uses when describing the value of SGE (Search Generative Experience). For example, the following language comes from the main SGE page:

• “Dive deeper on a topic in a conversational way.”
• “Access the high-quality results and perspectives that you expect from Google.”
• “I want to know what people think to help me make a decision”

Perhaps – and this is purely speculation – the feedback Google gets from quality raters about whether queries can be classified as “Know Simple” (or not) can help them understand when to trigger SGE.

Along the same lines, Google added two new examples of “Know Simple Queries” and “Know Queries” – the bottom two rows of the below table – to provide additional context about when a query is easily answered or when the answer is more open ended.

Google also added more examples of “Do” queries to the table below, starting with [shape of you video] and all queries below it. The three new examples represent queries that would be best answered with videos, images or how-to guides.

Further refining user intent

Google introduced new language around user intent by classifying what is an unlikely user intent for a set of keywords. 

In the table below, Google added a second column to explain unlikely intents for the keywords Harvard and Walmart. This limits the intent of these keywords to a more reasonable user intent, rather than a completely open-ended set of possible answers.

Users searching for “Harvard” could be looking for various details about the university, but are probably not looking for a specific course. 

Examples of Google’s SERP features that highly meet user needs

Google provides a table with various examples of search queries, the user’s location, and the user’s intent. It then shows the search result, and rates the extent to which the result met the expectations of the user (“Needs met”).

Google also offers an explanation about why these particular results are ranked as “highly meeting” the expectations of users. 

In this new version of the QRG, Google added and adjusted some of the examples in the “Highly Meets (HM)” results, the highest possible rating of meeting user needs “for most queries.” 

One example of a new example Google added to this list is one where the user is looking for “nearby coffee shops.” Google provides a screenshot of a Google Maps local pack with three coffee shops listed, and explains why this result highly, but not fully, meets the user’s expectation (it doesn’t list every possible coffee shop). 

Google even added a TikTok video as an example of a result that highly meets the needs of users looking an “around the world tutorial” for soccer. 

These are just some of the new examples, which seem to provide a more modern view of different results, both from external sites as well as Google’s own SERP features.

Dig deeper. An SEO guide to understanding E-E-A-T