In an ideal enterprise world, your CMO would plan a budget for SEO, leaving you to enjoy building your strategy without endlessly trying to prove ROI for each line item. 

Like I said, ideal. But not exactly realistic. 

Because CMOs are often hyperfocused on brand strategy or activation. Often, SEO is an entirely new marketing channel for enterprise companies. So, SEO professionals are left to their own devices. 

Anyone who’s ever had to plan an enterprise SEO budget knows that making this business use case can feel like you’re forced to eat grandma’s Jell-O salad. It’s a painful experience. 

So, how do you create an SEO budget for an enterprise company?

As an enterprise SEO Director for multiple companies, I’ve had the luxury of spending countless hours in spreadsheets and meetings to help the C-suite understand why SEO deserves a budget and how SEO will impact the bottom line. 

Below is my step-by-step process for building an SEO budget. 

1. Understand the business model

Why is this the first step to developing an SEO budget for an enterprise company?

The business model tells you what projects get money and don’t. Business models move your organization in a specific direction. 

By diving deep to grasp your company’s business model, you can strategically build your SEO budget to align with what the business wants to do. 

If you want to operate at an executive level in an enterprise company, you have to get below the surface. You can do this by finding the answers to these questions: 

• How do customers behave? How do they find us? What do they click on?
• What percentage of revenue comes from each business unit? 
• What revenue metrics are we reporting on (LTV, CAC, margin, etc.)?

The business model is the foundation you will build your SEO strategy on. Each different business model will dictate what your SEO tactics are. 

Let’s walk through some examples. 

Rover’s business model is built around lead generation. It’s got a beautiful map with dog walkers listed. Therefore, this page needs to be tailored to attract users to input their information.

Yelp’s business model is centered around sponsored listings and advertising dollars. The pages need to be structured to support the advertiser’s spend. 

2. Correlate SEO to revenue

When you get to an enterprise company as an SEO Director or above, you cannot speak to the C-suite executive team without mentioning revenue. 

​​Let’s pretend you’re presenting to a CEO. The CEO wants to see “I’d like to propose we do X, which will achieve $ZZZ and cost $Y.” How would you fill in the blank?

Here’s how Matt McKinley, SEO Manager for Revolt Media & TV, would fill in the blank. 

“X would represent one of the biggest projects from the SEO roadmap that’s needed to drive traffic/revenue. ZZZ is measured by the potential increase in traffic x average order value. Y would be the cost of resources needed for the project such as tools, other team’s time (developers, designers, editorial, etc.), additional SEO headcount, etc.” 

If you don’t mention revenue when pitching to the executive team, you’ll likely be more embarrassed than Adidas was for the Boston Marathon email debacle. 

Tough to recover from that, right? 

SEO professionals know how difficult it is to tie revenue to SEO, especially if your analytics is broken. 

I often get asked, “If we improve our page speed score by 10 points, how will that impact revenue?” Connecting a page speed to a revenue increase is a long-fetched idea. But we have to back into it somehow. 

Based on the below, you can say the number of good mobile URLs that are indexed impacts our organic traffic, therefore impacting new customers and revenue. 

Now, you can make a statement like:

“Each indexed Good Mobile URL is worth $10. If we improve the number of Good Mobile URLs by 50%, we can estimate a $5 million additional value.” 

Do you smell that? It smells like BS. 💩That’s because it is. 

Yet, this is how executives at enterprise companies operate. With these back-of-the-envelope calculations, you can frame the potential value of a project more strategically. 

Remember, it doesn’t have to be 100% accurate. Executives want to see that you’re looking beyond just the traffic. 

For example, let’s say you want to create a series of buying guides, but you need to expand the resources.

To sell this to the executive team, you would want to lead with something like, “We estimate that our guides are worth $20,000 in incremental revenue.”

Now to get this information to work up to the $20,000, you’ll want to look at historical data. Show a before/after scenario of an uptick in revenue after publishing a guide. 

You can look at categories. Using the example above, you could pull data from the Xbox category sales after launching the buying guide. 

Let’s walk through another example. I was working on a budget request for additional writers for a real estate company. 

Here is how I turned hypotheticals into ROI: 

Here is a link to my example document to give you something to work with. 

Again, it doesn’t have to be accurate. These are simply assumptions based on the data. 

3. Forecast SEO traffic

“Hey, we need a forecast for SEO in 2023.” This is a common ask I get from CEOs, CFOs, CMOs, etc. 

Before you can forecast traffic, you need to understand the business model. 

If you have a solid understanding of the business model, you can segment your forecasting based on the business structure instead of page type. 

For example, I worked on a project from Brother USA. Originally, I started to prioritize by page type.

But once I understood the business model and the different business units, I was able to deliver a better forecast. 

Once you know what areas of the website you are forecasting for, you can begin to build your SEO budget spreadsheet. 

I connected with Jori Ford, CMO at FoodBoss, to better understand how to forecast SEO. Ford shared:

“I outline existing traffic. Then, determine a projected lift based on SEO activities. After, I align that lift to the existing conversion metrics to show projected conversions. Next, I apply the overall value of those conversions as it relates to the business to do a rough range of potential financial gain whether that be revenue or just “value” vs. “cost.””

So you can see it’s all based on hypothetical scenarios. 

That’s why I recommend thinking about four different scenarios to give your executive a decision based on impact. 

• Scenario 1: Baseline (if we do nothing) 
• Scenario 2: Conservative
• Scenario 3: Average
• Scenario 4: Optimistic 

For example, if Best Buy wants to expand its buying guides, you can break down the investment into something like this. 

If you’re interested in learning more about financial models, I recommend Tom Critchlow’s SEO MBA course. Also, Tom Capper’s article on SEO Forecasting in Google Sheets is amazing. 

On the flip side, you can go a different route if you’re an agency. I connected with Will Critchlow, CEO at Searchpilot, and he shared insight into how his agency goes about this for enterprise companies. 

“We start with the trend from Google of on-site SEO being more unpredictable and less rules-driven than ever before, and the growth of their use of AI/ML continuing that trend. That, combined with the biggest problems large organizations face with SEO is in getting it done and proving the results lets us make the case that their on-site strategy should be built around SEO testing. From there, we build a budget case based on an ROI model that assumes a certain testing cadence, percentage of positive tests, and typical uplifts.”

SEO testing is underrated in enterprise companies. If you have the flexibility to A/B test your SEO tactics, you can make a better case for SEO ROI.

Before you present your enterprise SEO budget to your executives

Get eyes on the overall marketing budget

The golden rule of company budgets has always been that 10% of your company revenue goes toward your marketing budget. 

Then, that 10% is split between online and offline channels. 

In 2022, CMOs spend 56% of total marketing budget on online channels, according to Gartner’s 2022 Marketing Budget report. 

That means you’re fighting for a teeny tiny slice of the pie. 

Here’s a hypothetical range of enterprise SEO companies and how much they should spend on online channels. 

Before pitching your budget, you need to have eyes on the marketing budget before you ask for something above the overall marketing budget. 

Compare your SEO budget to your PPC budget

Again, hypothetically, you have 56% of the overall marketing budget going to online channels. Now you’re competing against other online channels for budget. 

In the same Gartner report, CMOs shared how the online channel marketing budget gets allocated: 

• 10.1% to social advertising
• 9.8% to search advertising
• 9.3% to digital display advertising 
• 8.8% to digital video advertising
• 8.1% to partners
• 8.1% to digital audio advertising
• 8.5% to SEO 
• 7.8% to content and messaging

It’s shocking. Out of the 56% of the overall marketing budget for online channels, CMOs are allocating 8.5% to SEO. 

Here is what that could look like for you: 

Then, with your SEO budget, you need to determine what percentage goes to technical, content and link building. 

Having this deep understanding of the marketing budget gives you a rough idea of the budget you could aim for. 

Embrace the suck of enterprise SEO budgets

Proving ROI in SEO may seem like a pointless, unuseful, tedious task made up of lies. 

But as SEO professionals, we must embrace the suck. 

Sure, it might seem like B.S. to you. But to your CMO or CEO, it might sound the alarm that you’ve been screaming. By speaking the business language, you’re speaking the language of your executives. 

The truth is, SEO professionals need to stop talking about traffic and pageviews and start talking about revenue.

Budget spreadsheets aren’t sexy. But assumptions are free.

Remember, it’s not about an accurate scientific method. It’s about giving you something to connect with the C-suite. 

I outline existing traffic, projected lift based on activities, and align that lift to the existing conversion metrics to show projected conversions. I then apply the overall value of those conversions as it relates to the business to do a rough range of potential financial gain, whether that be revenue or just “value” vs. “cost.”

I’d like to propose we increase monthly active users by 20%, which will achieve $ZZZ based on the LTV or first-time purchase value of a user and will cost us $Y to get there over the next ABC time frame.

Opinions expressed in this article are those of the guest author and not necessarily Search Engine Land. Staff authors are listed here.

About The Author

Anna Crowe is a Frank’s Hot Buffalo sauce advocate (leading with obvious priorities), writer, marketer and SEO nerd. Over the last 12+ years, Anna has successfully run SEO and content programs for brands like Dollar Thrifty Rental, Marriott, Hilton, Hearst Magazine, Mailboat Records, Philip Morris International, Bloomin’ Brands and Brother USA and many more. She enjoys burritos and puppies (in that order) when she’s not moonlighting as a soccer coach. https://annaleacrowe.com/

https://searchengineland.com/enterprise-seo-budget-plan-387117

Google has reenabled the validate fix feature in Google Search Console while launching the new “more simplified” classifications for the coverage reports. This was expected, earlier this month Google temporarily disabled the validate fix feature in order to launch these new classification labels.

Validate fix is back. Now when you go into Google Search Console and click on some of the coverage reports, the “validate fix” button should be back. Google disabled it while upgrading the classification system. There was a notice that read “Limited functionality: We are making some minor updates in the next few days. During this period you will not be able to issue new validation requests.” That notice has been removed and you now have the validate fix feature back.

Here is a screenshot I took:

New classifications. As mentioned above, Google launched more simplified classifications for these reports. Instead of the coverage report showing errors, valid with warnings and valid; now the report just shows invalid or valid information.

Google said the URLs or items in the Search Console report are no longer grouped at the top level by three or more status categories, i.e. Valid, Warning, and Error. Now they are grouped or classified into two more broad statuses that reflect whether those URLs or items are invalid or not. Google said invalid means that there is a report-specific critical issue in the page or item, and not invalid means that the item might still contain warnings, but has no critical issues. The implications and exact terms for the valid and invalid states varies by report type, Google added.

Google explained “grouping the top-level item (a rich result for the rich result reports, a page or URL for the other reports) into two groups: pages or items with critical issues are labeled something like invalid; pages or items without critical issues are labeled something like valid. We think this new grouping will make it easier to see quickly which issues affect your site’s appearance on Google, in order to help you prioritize your fixes.”

New Tour. When you login and go to the coverage overview section, you can now click on “full report” and Google will ask you if you want to see a tour of the changes, here is what the tour looks like:

We have a lot more detail on these changes in our earlier story over here.

Why we care. This should make it easier for you to understand errors in Search Console reports and thus which items to prioritize over others. This change is just a reporting change and does not change how Google crawls, indexes or ranks your content. But it is a change and you need to understand what changed and why.

Postscript. Google has now confirmed this is live with this tweet:

About The Author

Barry Schwartz a Contributing Editor to Search Engine Land and a member of the programming team for SMX events. He owns RustyBrick, a NY based web consulting firm. He also runs Search Engine Roundtable, a popular search blog on very advanced SEM topics. Barry can be followed on Twitter here.

https://searchengineland.com/google-reenables-the-validate-fix-feature-in-search-console-and-adds-new-classifications-387172

When Google announced they were improving featured snippets using the power of MUM, some of the wording around aligning with consensus reminded me of things that are described in the search quality evaluator guidelines (a.k.a., the Quality Rater Guidelines, or QRG). 

While the QRG are not an exact blueprint of Google’s algorithms, they give us many clues as to what it is that Google is attempting to accomplish. Google recommends studying the rater guidelines:

“If you understand how raters learn to assess good content, that might help you improve your own content. In turn, you might perhaps do better in Search.” 

Google, “What site owners should know about Google’s core updates“

We have had good success in helping sites improve by assessing them like a quality rater would. I highly recommend you study them!

This article will discuss:

• What it means to align with consensus.
• What the QRG says about aligning with consensus.
• What Google’s expanding content advisories could mean for sites writing on fringe or alternative topics.
• Google’s new content advisories for information gaps or low quality sites.
• Why understanding the concept of E-A-T is incredibly important.

What does it mean to align with consensus?

In Google’s blog post, they tell us they are introducing changes to what they call, “featured snippet callouts.” This is the part of the featured snippet that is bolded and in a larger font – essentially the answer to the searcher’s question.

In the example given, if a searcher asks. “how long does it take for light from the sun to reach earth”, the featured snippet callout would be “8 and 1/3 minutes.”

They tell us that these callouts will be checked “against other high-quality sources on the web, to see if there’s a general consensus for that callout, even if sources use different words or concepts to describe the same thing.”

At the time of writing this article, if you do this search, there is not yet a featured snippet with a callout, but you can clearly see that there is consensus from the top ranking sites on this answer.

Even though the results use slightly different wording – 499 seconds is the same as 8 1/3 minutes for example – there is a general consensus that this is a true fact. Google can likely feel comfortable that this answer is correct and therefore can feel confident in displaying it as a featured snippet callout.

Google says they’ve “found that this consensus-based technique has meaningfully improved the quality and helpfulness of featured snippets.”

With this change, aligning with general consensus (i.e. what the majority of high-quality sources say) is likely very important when it comes to winning featured snippet rankings, especially for “know-simple” queries where the searcher is looking for a specific concise answer.

If your content contradicts what the authoritative sites in your vertical say, Google will probably hesitate to show your answer as a featured snippet callout. 

Google is not saying that aligning with general consensus is a ranking factor at this point beyond being considered for featured snippet callouts. Given that there is a lot of information in the QRG to instruct the raters to assess whether content contradicts consensus, I think it’s reasonable to assume that for YMYL topics, aligning with consensus is important.

Whenever the topic of consensus comes up in SEO circles, it causes controversy.

Just because a bunch of people agree on something, does that make it factual?

It’s important to note it’s not just any site on the web that Google is looking at to determine whether there is consensus about an answer. They’re not looking for the most popular answer on the web. Rather, they say they are looking for consensus from “multiple high quality sources on the web”. (Google’s blog post on what site owners should know about core updates tells us a bit more about what they consider to be a high quality website.

Does this mean that any article that contradicts the viewpoint of highly ranking sites for a YMYL topic has no chance of ever being ranked by Google? I think Google possibly has a solution for this, which I will discuss shortly. 

What the QRG says about aligning with consensus

When the raters are taught to assess content, in several places of the QRG they are told to determine whether the content on the page aligns with expert consensus. This wording has been in the QRG for some time.

For YMYL topics, Google tells the raters it is a sign of high quality when content aligns with expert consensus and low quality if it does not.

In order for the raters to consider content high quality, it must be “factually accurate for the topic and must be supported by expert consensus where such consensus exists.”

In order for news articles or information pages on scientific topics to be considered high quality by a rater, they need to “represent established scientific consensus where such consensus exists.”

Again, for YMYL topics the raters are told to assess whether the page aligns with medical, scientific or even historical consensus.

In the most recent update to the QRG, Google stressed in several places that YMYL content should be assessed in terms of whether the topic, or misinformation on the topic has the potential to cause harm. “Health related advice that contradicts well-established expert consensus and could result in serious harm” is to be given the “lowest” rating by raters.

Aligning with expert consensus is important for sites that want to rank on Google with content covering YMYL (Your Money or Your Life) topics. 

Examples of sites that do not align with consensus

The QRG gives us several examples of content that raters should assess as low quality because they do not align with expert or scientific consensus. Here are a few.

1. A site promoting “proana” or “pro-anorexia” as a lifestyle choice

This site promotes anorexia as a lifestyle choice, which contradicts the recommendation of most doctors. Anorexia is viewed by doctors as an eating disorder and considered a mental illness.

If you read the content on the site, some of it is not bad. There is some relatively decent weight loss advice combined with the potentially harmful recommendation to drastically reduce calories eaten in a day. 

What do “high quality sites” have to say on this topic? No matter which search I did, the top ranking site exclaimed that proana was unsafe.

Google is unlikely to rank this page because it has advice that contradicts medical consensus and also has the potential to cause serious harm.

2. A page about the stomach flu

There are a few reasons why raters are told to assess this page as “lowest” quality. It is an article giving medical advice, but there is no evidence of medical E-A-T.

The raters are not told exactly which parts of this article contradict expert consensus and to be honest, most of the advice in this article does seem to be in line with what experts recommend. The only fault I could find is that the content recommends not eating where the Mayo Clinic recommends a patient with the flu does try to eat certain easy to digest foods.

I think the main concern with this content is that it is giving medical advice despite lacking medical E-A-T. Still, it is interesting to see that the raters are told it contradicts scientific consensus.

3. The Flat Earth society

This is an interesting one. The site has content that contradicts the general scientific consensus that the earth is round.

Google is confident here:

While you and I likely agree that the idea of the earth being flat is just silly, there are many people who truly believe that it is. But Google does not want to show searchers this information as it clearly contradicts scientific consensus.

Should they though? 

What if people are clearly searching for information that is contrary to consensus?

There is a line in the QRG that says that for medical or scientific pages to be rated as meeting a searcher’s needs, the content “must represent well-established scientific/medical consensus unless the user is clearly seeking an alternative viewpoint.” (Bolding added by me.)

What bothers me is that this is not yet the case for many queries in Google search.

Based on reading the example above, I did some searches for [is the earth flat]. I was curious to read the viewpoint of people who hold this belief – I wanted to understand why they believe this and to hear it from their perspective.

I tried several searches – [explanation of why the earth is flat], [why the earth is flat – flat earther’s viewpoint], or [evidence supporting the earth being flat]. These articles clearly exist. People like to write about their theories!

But even though I was clearly seeking an alternative viewpoint, Google only surfaced articles that were telling me why the earth was not flat and how everyone who says so is wrong. My intent as a searcher actually was to read information that contradicts scientific consensus. 

Who is Google to decide that I cannot search for and find alternative viewpoints on the web? Is Google acting like an overprotective mom deciding which content is safe for me to read?

I think it is possible that Google’s “expanding content advisories for information gaps” described in their announcement could be the first step in addressing this issue.

Content advisories for information gaps

Google already shows content advisories where news about a breaking story is unfolding quickly on the web. They may show searchers a message saying, “It looks like these results are changing quickly. If this topic is new, it can sometimes take some time for results to be added by reliable sources.” 

In Google’s recent blog post they tell us they are expanding this advisory beyond breaking news stories.

In the example Google gives, the search was [how to get in touch with the Illuminati]. I did this search and did indeed get one of these warnings:

While there was a historical group called the illuminati, today when they are mentioned general consensus is that much of the information discussed falls under the category of unsubstantiated conspiracy theory.

Again, though, is it Google’s responsibility to protect me from misleading or potentially harmful information on the web? What if I legitimately was doing research and wanted to read information supporting this conspiracy theory? 

Following the warning that Google may not have reliable information on this topic, they actually do display some sites I could read to explore this topic further.

I believe, although it remains to be seen, that this advisory warning is Google’s answer when it comes to people who are clearly wanting to see results from a viewpoint that contradicts consensus or is potentially unsafe. They can now present searchers with sites that present an alternative viewpoint, even if there is concern that the content could be misleading or harmful.

I could see this actually being good for many alternative medical websites! These content advisories may allow Google to display sites discussing medical treatments that are contrary to scientific consensus if it is clear that this is what the searcher is looking for.

Understanding E-A-T is incredibly important

I was thrilled to see Google emphasize in this blog post the importance of understanding the concept of E-A-T (expertise, authoritativeness and trustworthiness). 

Google wants to show results from authentic sources.

They tell us in their recent blog post that they are expanding the “About this result” feature to help searchers understand more about the company whose website they are viewing.

This all sounds very much like E-A-T as described in the QRG.

The QRG is absolutely filled with information to help us understand what the concept of E-A-T means. Each time I read them I find more clues that can help us improve E-A-T.

This recently published Google document called Search Quality Rater Guidelines: An Overview gives a really good summary of how raters assess E-AT:

E-A-T is so much more than simply adding an author bio. For some content, an author bio may not even be necessary or even helpful! 

E-A-T includes having content that takes time, effort, expertise or talent/skill to create.

If you’re writing on topics where a searcher would expect to see an answer written by an expert, then yes, demonstrating the article was written by someone with expertise, or perhaps being known as a company with expertise in this area is important.

Having a reputation for knowing your topic is a big part of E-A-T.

Consensus: What SEOs can learn from Google

We spoke a lot in this article about the importance of aligning with consensus, especially if you write on YMYL topics. I’d like to reiterate that Google’s blog post was talking about using consensus amongst high quality sites as a component to help them produce better featured snippet callouts. 

They were not confirming that consensus is a ranking factor when it comes to ranking in organic search.

However, given there is huge emphasis on aligning with consensus in the QRG, I think it is reasonable to assume that if we want to have our YMYL content rank on Google, aligning with consensus is something we should strive for.

This is likely a component of trustworthiness, the T in E-A-T.

My advice:

• If most of your site aligns with general/expert consensus but some content is controversial, it may be helpful to have the controversial content separated into its own subdomain or folder. It might also be helpful to present both sides of the story in your writing and make it clear that the content on this page does not line up with what many experts believe. Or, a better option may be to remove the content that is controversial.
• If the majority of your site contains content that contradicts the consensus of high quality sites in your vertical, it is unlikely Google will rank your content unless it is preceded by the “content information gap” warning mentioned above. You may want to look at marketing more on social media, email marketing or some channel other than Google organic search.
• If you have content that is borderline — actually good science, but not yet widely accepted by all experts in your field as valid, I would recommend doing all you can to get the experts in your field talking about this subject. Good PR could help here. Also, ensure everything you write is backed by authoritative references and written by someone who has extensive expertise. As your topic becomes more mainstream, and experts start to align with your position, you may find you are able to rank better.
• If you’re not sure whether your viewpoint could be seen as contradicting consensus, do some searches to see what the top ranking sites say. For example, if you wrote on a controversial topic in the financial world, you could search for something like, “is [keyword] beneficial site:bloomberg.com” or “the harm of [keyword] site:wsj.com”. If your viewpoint differs, you need to really consider whether you want to publish this article. 
• Do all you can to demonstrate your E-A-T. Read the QRG, especially the examples, and thoroughly study Google’s questions they say to ask yourself in regards to content.

Opinions expressed in this article are those of the guest author and not necessarily Search Engine Land. Staff authors are listed here.

About The Author

Dr. Marie Haynes is completely obsessed with trying to understand how Google assesses quality and then helping businesses to improve their websites. Her growing company, Marie Haynes Consulting Inc. specializes in site quality reviews and link audits. You can contact her team or sign up to her newsletter that keeps the SEO world up to date on the latest changes and tips in search.

https://searchengineland.com/google-consensus-qrg-seo-387142

Understanding how your users engage with your brand is vital to delivering a great customer experience.  

This is why you shouldn’t overlook the customer journey. It allows you to shape your website’s user experience (UX) and content strategies based on the people who matter – your customers.

Let’s explore how the customer journey can help guide your SEO program.

Customer journey and SEO

A well-mapped customer journey can greatly benefit SEO. 

Because SEO is a broad discipline, it is easy to scatter your efforts. Some people get too immersed in perfecting one aspect – either technical, on-page or external. 

Putting a premium on the customer journey helps align your SEO program and focus your efforts on what’s valuable. 

Whether you look at the customer journey as a linear or multi-touch path, four-level funnel, or a flywheel, you must implement an overall strategy. 

Within that strategy and even tactical activities and plans benefits from aligning with the customer journey.

Here are a few areas where the customer journey can guide your SEO efforts.

1. Mapping keywords and topics to the journey

Are you struggling with an extensive keyword list? Do you want to optimize for all the terms that you know are relevant but find it challenging?

Using your customer journey maps, you can get your content right overall and leverage the clarity for SEO. 

For instance, the best time to educate prospects is when they’re still in the research and exploratory phase. 

When someone is just gaining awareness of your brand, you should aim to become their trusted resource. Help them understand their problem and identify possible solutions. Avoid the urge to sell.

This strategy often works for B2B, but it can play out even in ecommerce and B2C. 

Let’s say you’re selling groomsmen gifts. You can plan on creating a comprehensive guide for everything an engaged couple needs to know when planning for their wedding. 

Doing so will help you get found as a great resource – even before they realize they need specific groomsmen gifts.

Building awareness and brand affinity early in the journey will help customers keep engaging with you until they’re ready to buy.

At that initial awareness stage, take the terms that you know are not likely to convert but are important. Then, strategically plan your content and optimization so you can be found at this stage in the customer journey.

The same is true for terms where you know the intent is to convert. Terms that show purchase intent – like a search for a specific product’s part number or action-oriented lead generation terms (e.g., “dentist near me”) – are further in the customer journey. 

Mapping the right keywords to this stage is critical too. At this point, we want to get them close to a purchase, contact form or another way to engage with us.

Don’t bury the call to action. Avoid hitting them with thought leadership or general awareness content.

Based on your customer journey mapping, you will also find a lot of in-between queries and topics to tie your content to. Use your understanding of the journey to categorize keywords and topics and map them to the right content on your site.

Hopefully, by categorizing your keywords this way and looking at content needs through this lens, you can gain some refreshing focus and clarity for your content plans and investment.

2. Conversion rate optimization goals and objectives

So you’ve invested a lot of time in mapping keywords and topics to respective stages of the customer journey and developed the content to fill any gaps. Unfortunately, there’s no guarantee that everything will work perfectly as intended.

Regardless of how objective and specific you are, there will always be something to test, optimize, and build upon.

In some cases, you might find out new things about what the customer journey looks like.

In others, you’ll get the impressions and traffic you want but won’t see the user move to the next step you expect in the journey.

This is where conversion rate optimization (CRO) comes in. 

Go back to your historical data and your projected data. 

• What percentage of users typically do what next? 
• What do you expect them to do?
• Do they return to the site three days later on a more specific page? 
• Do they usually visit certain pages or take specific actions?

Dig deeper. See what other keywords you could be optimizing for. 

Understand when they bounce back to Google and how they might refine searches. Leverage data from other channels.

Getting granular with Google Analytics, heat mapping and CRO tools, and your customer relationship management (CRM) platform will help. 

Leverage them to learn where the journey is accurate, whereas the UX or content needs improvement. Find areas where you can optimize the assumed or planned journey.

SEO elements must be incorporated here to include technical factors (site speed, Core Web Vitals, indexing), on-page, content strategy and calls to action.

3. Measurement and attribution

I could have listed this first as it is woven through my look at how the customer journey and SEO converge. 

I assume you have a well-defined journey. However, as I noted above, using CRO and other ways to optimize the journey and website overall, you will need to measure what is happening to make informed decisions fully.

Are the keywords you’re focused on at each stage driving traffic to your intended pages? 

Are the users doing what you want them to do and going deeper in the journey?

You’ll find weaknesses in your analytics setup and how you look at SEO and overall web analytics when you put it to the test. That’s a good thing.

Find new ways to view and measure the customer journey. Also, align your measurement of SEO factors and performance against that journey. 

This will help you focus your efforts and not broadly bucket global stats like rankings, impressions, traffic, and conversions.

Getting this deep, you can see what you’re leaving on the table. Perhaps you’re focusing too heavily on awareness keywords (and not getting conversions). Or you might be fixated on just action/”convert now” terms without fostering “unaware” users through the journey.

Conclusion

Focus and objective guidance is essential for SEO. Any prioritization and alignment with web and broader strategies can help, whether you have extensive resources to invest or can only adopt a lean approach.

Through leveraging a mapped-out customer journey, you can define needs, strategies, tactics, and goals for content. At the same time, you can manage expectations and investments.

SEO should not be done on an island. It’s best implemented as part of the whole. Customer journey thinking brings you closer to the overall marketing plan and strategy. Leverage it as an opportunity.

Opinions expressed in this article are those of the guest author and not necessarily Search Engine Land. Staff authors are listed here.

About The Author

Corey Morris is a skilled marketing professional with 15+ years of experience developing award-winning, ROI-generating digital strategies for local and national brands. He was recently honored as the recipient of the KCDMA 2019 Marketer of the Year award. Corey serves as the chief strategy officer at Voltage – a marketing firm based in Kansas City, MO. Previously, he founded the KC Search Marketing Conference to help build a local community for search marketers for career growth. He was recognized for his involvement in the conference and invited to join the global board of SEMPO (now part of DAA) as the VP of Cities.

https://searchengineland.com/customer-journey-guide-seo-387107

7-percent of all the new URLs clicked on URLs in Bing Search in the past month were sourced from the IndexNow protocol, Fabrice Canel from Microsoft told Search Engine Land. In fact, IndexNow has grown a lot, with more than 16 million websites that are using IndexNow, they are publishing over 1.2 billion URLs per day to the IndexNow API.

IndexNow’s growth. In February 2022 IndexNow has grown to support millions of sites by adding integration with Duda, All in One WordPress SEO plugin and Rank Math SEO plugin. Microsoft released a simple-to-use WordPress plugin for IndexNow and Google announced it would be testing this protocol. Then in February 2022, Microsoft said 80,000 websites are now using IndexNow for URL submission.In March 2022 SEOPress added integration building the platform by another 200,000 sites. In June, Yoast SEO added support providing easy access for a lot of WordPress sites to turn on the feature. With those providers integrating IndexNow, it gained millions of sites adopting IndexNow, just several months after IndexNow was initially rolled out. Now, we are at over 16 million websites that are using IndexNow.

Earlier this year, the Czech search engine Seznam.cz adopted IndexNow.

What is IndexNow. IndexNow provides a method for website owners to instantly inform search engines about the latest content changes on their website. IndexNow is a simple ping protocol so that search engines know that a URL and its content has been added, updated, or deleted, allowing search engines to quickly reflect this change in their search results. The co-sharing of IndexNow went fully live in January 2022.

Integrations. Why Microsoft built its own plugin for WordPress sites to use IndexNow, Yoast decided to build it in to its premium plugin. Why did Yoast add it now? “The latest change to the protocol is the second significant development that made us change our minds about IndexNow. The IndexNow team changed the endpoint of the API from pinging each individual-supported search engine to a single API managed by the protocol. This means that sites only have to ping a single site, making the process much more efficient. All of this fits into our view of a sustainable crawling and indexing solution, so now is the time to implement IndexNow with a new integration in Yoast SEO,” the company explained.

Google support? Nope, not yet. Google last year said it was testing IndexNow but since then has not said much about IndexNow. There have been some rumors that Google would not adopt this protocol but as we have seen in the search space, things change fast and often.

Why we care. Like we said before, instant indexing is an SEO’s dream when it comes to giving search engines the most updated content on a site. The protocol is very simple and it requires very little developer effort to add this to your site, so it makes sense to implement this if you care about speedy indexing.

Plus Microsoft said the company is “working to leverage IndexNow signals to further optimize our crawl machine learning stack.” By the Fall Microsoft said it expects to “crawl websites that have adopted IndexNow with a clear content change signal even more efficiently, resulting in faster updates and less crawl.”

About The Author

Barry Schwartz a Contributing Editor to Search Engine Land and a member of the programming team for SMX events. He owns RustyBrick, a NY based web consulting firm. He also runs Search Engine Roundtable, a popular search blog on very advanced SEM topics. Barry can be followed on Twitter here.

https://searchengineland.com/indexnow-attributed-for-7-of-all-new-urls-clicked-on-in-bing-in-the-past-month-387115

Third-party cookie tracking is going away, and the SEO industry is ready and diligently preparing ambivalent and posting memes on Twitter.

SEO has been dealing with the lack of cookie tracking since its existence.

So does the cookie’s demise actually matter?

Well, my friends, I’m here to tell you two things.

The good news: this change means something and there’s an opportunity. 

The bad news: it’s not going to be easy.

It takes some smart chops to get done, plus considerable resources you’re not likely to get sitting over there in the SEO corner of the office (remote, of course). 

What’s the situation?

Legislation, such as the EU General Data Protection Regulation (GDPR) and the California Consumer Privacy Act (CCPA), are tightening up how advertisers can use to track users. 

I think the claim behind this initiative – that consumers want more control of their data – is partly valid but not tremendous. 

Most users don’t care and don’t think about who’s tracking them, except in passing. It’s not a concern that causes them to change their online behavior unless they’re trying to hide something. 

Most of us would prefer to have clarity and reasonable limits on what an advertiser can track and how they target us. But in general, I’ve found we leave it at that. 

The average user doesn’t think much about it, especially since it gets technical and specialized quickly, and more so by the year.

But these privacy restrictions are coming and they’re a good thing. 

Have you noticed, for example, the rise of auditory targeted ads and content you’re being delivered by Google? 

Try an experiment some time in your home. 

Start talking about a random but specific topic and repeat the keyword(s) a few times. 

You’re likely to find it in your news feed, in ads, in search results, and sprinkled around in the most unusual “recommended” places. 

Freaky? Yeah, kinda…

It’s probably good we’ve got legislation setting some limits, however, limited they are at this early stage.

It’s not new, anyway. The focus on cookies has been happening for years. 

For example, Firefox began blocking third-party cookies as early as 2019. Safari followed suit in 2020. 

As the move to a cookieless future gains force and creates greater restrictions in digital advertising, SEO needs to keep pace.

We need to get a seat at the table, especially with regard to the measurement of channel effectiveness, attribution, and yes, incrementality (I said it!). 

The latter is a big word – and a difficult thing to do in SEO.

Historical measurement models

Traditional measurement models that leverage cookies, such as multi-touch attribution (MTA), will be increasingly phased out of analytics toolkits. 

The two primary models marketers have historically used are media mix modeling (MMM) and MTA. 

MMM is a top-down approach that typically covers multiple years of data, while MTA is a bottom-up approach, more granular, and reliant upon cookies to track sessions and users. 

The problems with cookies are of some significance, too. They fail to measure cross-device, and more recently, they’re opt-in only. 

But marketers still need to measure performance. Cookies have been handy for that.

Next-level ideas for tracking SEO

When considering how a cookieless future impacts SEO, follow the model already set forth by other measurement channels: build a clean room. 

The reality is that a clean room probably will not be built specifically for SEO. It doesn’t need to be – since SEO doesn’t have first-party data, anyway. 

This is where the hard reality of SEO relative to other channels becomes apparent. Measuring it will not lead to the investment of resources across an organization. Not by itself, anyway. 

But you can leverage the work others have done in paid media, for example, to get some interesting measurement applications for SEO.

Aggregated attribution

Rather than using individual data, this approach takes a high-frequency metric (i.e., organic search sessions) and examines how other media (i.e., TV spots) impacts the channel. 

This type of analysis provides insight into how SEO captures the demand created by a TV ad, offline campaign, or display campaign.

Modified media mix modeling 

Trying to force organic clicks into media mix modeling (MMM) is a misapplication of the metric due to the fact you’ll be changing the results already reported to the organization.

The paid media team would disagree, and the organization would be distracted and potentially stuck in arguments over attribution. 

Instead, we can take the MMM and set aside all the sales driven by media. Then, we can run SEO clicks against the base sales to attempt to tease out the signal of SEO that is hiding in the base. 

Additionally, we can consider running a model of paid media impressions against SEO clicks to understand media interaction. 

This is similar to the aggregated attribution approach but more granular.

Additional considerations

We have to balance the reality of how much teams are willing to invest in tracking the effectiveness of SEO, relative to other channels. 

There is a lot of money pouring into media, obviously, and this drives heavy innovation into media mix modeling and attribution for these channels. 

The same cannot be said for SEO. But we need to find ways to measure SEO’s effectiveness, and it needs to be sophisticated in line with other channels’ approaches today. 

Gone are the days of relying on some third-party Semrush charts, except in cases perhaps where we’re looking at competitive insights.

It may very well be that existing MMM solutions already have adequate insights available to them that include owned and earned observations without risking what analytics teams call, “collinearity,” the phenomenon of insights being skewed from data sets that are dependently correlated (i.e., linear) when sliced and diced.

Another consideration is that teams may simply not need, or have the budget for, complex modeling such as MMM. In these cases, perhaps Google Analytics 4 and Adobe do everything that’s needed at a basic level, which can be augmented with some SEO testing.

Conclusion

The answer to all this is simple but hard to accomplish. 

SEO as a channel famously plays second fiddle to media – be it paid search, display or paid social. 

Yes, companies invest in SEO and care about SEO. 

When everything else is accounted for, however, media dollars will always take precedence in any measurement conversations. 

Resources follow the money, and SEO is on the short end of the stick when it comes to resources from the analytics and data science teams.

But it doesn’t have to be.

Getting SEO data sets into the clean rooms and aligning them to other data sources is key for gaining insights into the channel. 

As digital marketers move toward using clean rooms such as Google Ads Data Hub (ADH) and others, SEO teams need to get site analytics data into these environments. 

By bringing the data together, SEOs can look at customer journeys across paid media impressions, clicks and site activity (including a tag for source as organic search). 

In this new environment with SEO analytics data added to a clean room, marketers can also get toward an attribution use case to measure the contribution and even the incrementality of the SEO channel and its relationship with the other channels.

But there’s a difficult catch here. The reason it’s hard is that generating buy-in from others is essential. 

There’s already enough focus and resources centered on this transition away from cookies toward clean rooms and more trackable solutions. 

This means resources aren’t sitting around (usually) waiting to accommodate SEO. And most marketers won’t have the interest to rank SEO priorities above media, especially when it comes to things like attribution and channel performance measurement. 

But that’s exactly what we need as SEOs more than ever: good performance tracking. 

And we especially need SEO’s contribution, and yes, incremental addition to the cross-channel picture. 

Doing this successfully is part of the SEO’s world: navigating resources and teams along with generating buy-in from the right groups to prioritize this work. 

If you can do that, the entire organization will benefit from greater clarity of SEO’s contribution and its value to the business.

Opinions expressed in this article are those of the guest author and not necessarily Search Engine Land. Staff authors are listed here.

About The Author

Adam Audette is SVP SEO and Data Science at Blend360. Adam started his career in the early 2000s by co-founding the SEO agency AudetteMedia. He was the global head of SEO for Merkle / Dentsu for nearly eight years. He now works at Blend360 combining the worlds of data science and SEO. Adam and his teams have worked with many of the world’s leading brands, including Google, Facebook, Amazon, Zappos, Walmart, Target, Experian, Samsung, Walgreens, and many others.

https://searchengineland.com/seo-future-world-without-cookies-387079

Everyone is talking about creating and marketing videos these days. The marketing benefits are clear: Videos spread well, generate engagements on social media and are able to still attract attention in our world of information overload.

But can videos also enhance your SEO strategy? 

Well, we don’t know everything about Google’s ranking signals, so we cannot be sure if there’s a direct factor there. Whether Google “likes” pages with videos on them more than pages without videos remains a secret.

But there are more indirect SEO benefits that are contributing here:

1. SERP real estate

Videos are taking a lot of real estate within Google’s search result pages, both on mobile and desktop. Google uses so-called “video carousels” to display related videos for many search queries.

In fact, on a mobile device, a video carousel takes the whole screen, and for many search queries it takes quite some time to scroll past:

Image source: Screenshot by the author

According to Mozcast, video carousels appear for the third of search queries. That’s a lot of SEO opportunities to dominate your target SERPs!

If you brand your video well and optimize your video Youtube page, one of these videos in a carousel will remind your target customer of your brand and increase your chances of conversion.

2. Rich snippets

A video-rich snippet is an enhanced search snippet that displays a video thumbnail next to your link. It is a good way to make your search snippet stand out.

Image source: Screenshot by the author

Video rich snippets are also surprisingly easy to get.

Unlike video carousels, video rich snippets send traffic to your site, versus the YouTube video page. To make this easier:

The best-case scenario is when you manage to grab both: A rich video snippet on page one of organic results and your video inside the video carousel on the same page:

Image source: Screenshot by the author

This will help you generate more brand awareness through a video thumbnail and generate both video views and clicks.

To increase your odds of getting a video rich snippet, you need to:

• Embed a video on your page. This doesn’t really have to be your video but mind that Google will display the channel name below your snippet om desktop, so it makes sense to use your brand’s channel
• Use video schema. Here’s a handy video schema generator for you to do that. There are also quite a few plugins that make this step easier.

Image source: Screenshot by the author

For desktop video rich snippets, mind that they also pull the date when you uploaded the video, so make sure your video is fresh to avoid your snippet looking outdated.

When embedding a video on your page, don’t forget to double-check your page load time. Surprisingly, Youtube videos do negatively impact page performance when you embed them, even though Youtube belongs to Google (so publishers usually assume it is optimized for Google by default).

To demonstrate the impact, here’s my page score without a YouTube video embedded on it:

Image source: Screenshot by the author

And here’s the same page with a Youtube video added:

Image source: Screenshot by the author

You can cope with this by using the trick described here which worked wonders in my case above. 

One of the unexpected benefits of using that method was the related videos Youtube shows at the end of each video when you embed it. Using the steps in that tutorial, I found those related videos disappear and my own video showing up instead.

Finally, don’t forget to use Google’s Search Console to find out how your video rich snippets are behaving.

In your “Performance” section, create a new filter “Search Appearance: Videos” to see which queries are triggering rich snippets for your site and monitor your CTR for those queries.

3. On-page engagement

There’s no official confirmation that Google uses on-page engagement as a direct ranking signal. In fact, I believe they denied using such a signal.

And yet, getting people to take a pause and do something on a page has a lot of marketing benefits that are normally part of an SEO strategy. These include:

• Higher chances of a conversion
• Lower chances of them bouncing or returning to search results (which, again, may or may not be a direct ranking signal)

Videos make great on-page engagement boosters, so if you feel like your page is failing to trigger some sort of action, try embedding a video on it. Again, mind your core web vitals when you do.

4. Referral traffic

Just like on-page engagements, referral traffic has never been confirmed to be a direct SEO signal. We know that Google likes when a link is clicked but we don’t know whether a page with 0 referral traffic from external sources  has lower authority in Google’s eyes than a page with high-quality referral traffic.

And yet, referral traffic increases a page’s chances to get backlinks and shares. And well-converting traffic is always good, whether it is coming from organic search or another site.

Youtube doesn’t make it easy to build referral traffic from your videos. You can have your clickable link in the video description which is rarely read or even seen (especially on mobile).

You can also link build direct traffic using Youtube by giving your domain name in the video intro and outro. Here’s how to create an engaging Youtube outro.

But then again, one should really find your video relevant and/or awesome to type your domain name in the address bar after seeing it in the outro.

That being said, Youtube does send traffic to a site but only if you are doing a great job creating outstanding videos and then using verbal CTAs inside the video to encourage clicks from the video description or direct type-ins.

A great thing about Youtube traffic is that, while it is hard to get, it’s highly relevant, so you will see highly engaged clicks that become your subscribers and customers:

Image source: Screenshot by the author

Conclusion

Using videos is a solid way to help your SEO strategy. While there may be no confirmed direct signals involved here, there are a few indirect SEO benefits of investing into video creation and marketing.

https://www.searchenginewatch.com/2022/08/11/how-do-videos-help-your-seo-strategy/

Google had a data logging error that impacted the Discover performance report in Google Search Console. This occurred on July 26, 2022 and may result in a drop in the number of clicks and impressions that Google logged.

This did not impact your rankings or visibility, it was just a reporting glitch.

The issue. Google wrote that on July 26 “due to a logging error, site owners might see a drop in their Discover clicks and impressions during this period. This is only a logging error, it is not a drop in actual views or clicks.”

Reporting issue only. Again, this did not impact your visibility in Google Discover, it is just an issue with reporting and data logging. So it may appear like you saw a drop in visibility but in reality, it was just an issue with Google’s reporting engine.

Here is a screenshot showing the annotation Google put on the report:

Fix coming? There does not appear to be a fix coming to replace the data, it seems like the data is gone and you will just have to annotate the date and move on.

Discover is hard. As a reminder, Google Discover traffic is super volatile, and thus, seeing massive spikes or massive declines in your Discover clicks and impressions is normal. So the loss of this data will be hard to backward engineer for any marketer or SEO.

Why we care. This is important for you to know and communicate to your clients, boss, stakeholders, etc when providing these reports to your client. So make sure to annotate the chart, which Google did for you, and then communicate these issues when presenting them to others.

About The Author

Barry Schwartz a Contributing Editor to Search Engine Land and a member of the programming team for SMX events. He owns RustyBrick, a NY based web consulting firm. He also runs Search Engine Roundtable, a popular search blog on very advanced SEM topics. Barry can be followed on Twitter here.

https://searchengineland.com/google-reporting-issue-with-search-console-discover-report-387105

“What’s the difference between crawling, rendering, indexing and ranking?”

Lily Ray recently shared that she asks this question to prospective employees when hiring for the Amsive Digital SEO team. Google’s Danny Sullivan thinks it’s an excellent one.

As foundational as it may seem, it isn’t uncommon for some practitioners to confuse the basic stages of search and conflate the process entirely.

In this article, we’ll get a refresher on how search engines work and go over each stage of the process.   

Why knowing the difference matters

I recently worked as an expert witness on a trademark infringement case where the opposing witness got the stages of search wrong.

Two small companies declared they each had the right to use similar brand names.

The opposition party’s “expert” erroneously concluded that my client conducted improper or hostile SEO to outrank the plaintiff’s website. 

He also made several critical mistakes in describing Google’s processes in his expert report, where he asserted that:

• Indexing was web crawling.
• The search bots would instruct the search engine how to rank pages in search results. 
• The search bots could also be “trained” to index pages for certain keywords.

An essential defense in litigation is to attempt to exclude a testifying expert’s findings – which can happen if one can demonstrate to the court that they lack the basic qualifications necessary to be taken seriously.

As their expert was clearly not qualified to testify on SEO matters whatsoever, I presented his erroneous descriptions of Google’s process as evidence supporting the contention that he lacked proper qualifications. 

This might sound harsh, but this unqualified expert made many elementary and apparent mistakes in presenting information to the court. He falsely presented my client as somehow conducting unfair trade practices via SEO, while ignoring questionable behavior on the part of the plaintiff (who was blatantly using black hat SEO, whereas my client was not).

The opposing expert in my legal case is not alone in this misapprehension of the stages of search used by the leading search engines. 

There are prominent search marketers who have likewise conflated the stages of search engine processes leading to incorrect diagnoses of underperformance in the SERPs. 

I have heard some state, “I think Google has penalized us, so we can’t be in search results!” – when in fact they had missed a key setting on their web servers that made their site content inaccessible to Google. 

Automated penalizations might have been categorized as part of the ranking stage. In reality, these websites had issues in the crawling and rendering stages that made indexing and ranking problematic. 

When there are no notifications in the Google Search Console of a manual action, one should first focus on common issues in each of the four stages that determine how search works.

It’s not just semantics

Not everyone agreed with Ray and Sullivan’s emphasis on the importance of understanding the differences between crawling, rendering, indexing and ranking.

I noticed some practitioners consider such concerns to be mere semantics or unnecessary “gatekeeping” by elitist SEOs. 

To a degree, some SEO veterans may indeed have very loosely conflated the meanings of these terms. This can happen in all disciplines when those steeped in the knowledge are bandying jargon around with a shared understanding of what they are referring to. There is nothing inherently wrong with that. 

We also tend to anthropomorphize search engines and their processes because interpreting things by describing them as having familiar characteristics makes comprehension easier. There is nothing wrong with that either. 

But, this imprecision when talking about technical processes can be confusing and makes it more challenging for those trying to learn about the discipline of SEO. 

One can use the terms casually and imprecisely only to a degree or as shorthand in conversation. That said, it is always best to know and understand the precise definitions of the stages of search engine technology.

The 4 stages of search

Many different processes are involved in bringing the web’s content into your search results. In some ways, it can be a gross oversimplification to say there are only a handful of discrete stages to make it happen. 

Each of the four stages I cover here has several subprocesses that can occur within them. 

Even beyond that, there are significant processes that can be asynchronous to these, such as:

• Types of spam policing.
• Incorporation of elements into the Knowledge Graph and updating of knowledge panels with the information.
• Processing of optical character recognition in images.
• Audio-to-text processing in audio and video files.
• Assessing and application of PageSpeed data.
• And more.

What follows are the primary stages of search required for getting webpages to appear in the search results. 

Crawling

Crawling occurs when a search engine requests webpages from websites’ servers.

Imagine that Google and Microsoft Bing are sitting at a computer, typing in or clicking on a link to a webpage in their browser window. 

Thus, the search engines’ machines visit webpages similar to how you do. Each time the search engine visits a webpage, it collects a copy of that page and notes all the links found on that page. After the search engine collects that webpage, it will visit the next link in its list of links yet to be visited.

This is referred to as “crawling” or “spidering” which is apt since the web is metaphorically a giant, virtual web of interconnected links. 

The data-gathering programs used by search engines are called “spiders,” “bots” or “crawlers.” 

Google’s primary crawling program is “Googlebot” is, while Microsoft Bing has “Bingbot.” Each has other specialized bots for visiting ads (i.e., GoogleAdsBot and AdIdxBot), mobile pages and more. 

This stage of the search engines’ processing of webpages seems straightforward, but there is a lot of complexity in what goes on, just in this stage alone. 

Think about how many web server systems there can be, running different operating systems of different versions, along with varying content management systems (i.e., WordPress, Wix, Squarespace), and then each website’s unique customizations. 

Many issues can keep search engines’ crawlers from crawling pages, which is an excellent reason to study the details involved in this stage. 

First, the search engine must find a link to the page at some point before it can request the page and visit it. (Under certain configurations, the search engines have been known to suspect there could be other, undisclosed links, such as one step up in the link hierarchy at a subdirectory level or via some limited website internal search forms.) 

Search engines can discover webpages’ links through the following methods:

• When a website operator submits the link directly or discloses a sitemap to the search engine.
• When other websites link to the page. 
• Through links to the page from within its own website, assuming the website already has some pages indexed. 
• Social media posts.
• Links found in documents.
• URLs found in written text and not hyperlinked.
• Via the metadata of various kinds of files.
• And more.

In some instances, a website will instruct the search engines not to crawl one or more webpages through its robots.txt file, which is located at the base level of the domain and web server. 

Robots.txt files can contain multiple directives within them, instructing search engines that the website disallows crawling of specific pages, subdirectories or the entire website. 

Instructing search engines not to crawl a page or section of a website does not mean that those pages cannot appear in search results. Keeping them from being crawled in this way can severely impact their ability to rank well for their keywords.

In yet other cases, search engines can struggle to crawl a website if the site automatically blocks the bots. This can happen when the website’s systems have detected that:

• The bot is requesting more pages within a time period than a human could.
• The bot requests multiple pages simultaneously.
• A bot’s server IP address is geolocated within a zone that the website has been configured to exclude. 
• The bot’s requests and/or other users’ requests for pages overwhelm the server’s resources, causing the serving of pages to slow down or error out. 

However, search engine bots are programmed to automatically change delay rates between requests when they detect that the server is struggling to keep up with demand.

For larger websites and websites with frequently changing content on their pages, “crawl budget” can become a factor in whether search bots will get around to crawling all of the pages. 

Essentially, the web is something of an infinite space of webpages with varying update frequency. The search engines might not get around to visiting every single page out there, so they prioritize the pages they will crawl. 

Websites with huge numbers of pages, or that are slower responding might use up their available crawl budget before having all of their pages crawled if they have relatively lower ranking weight compared with other websites.

It is useful to mention that search engines also request all the files that go into composing the webpage as well, such as images, CSS and JavaScript. 

Just as with the webpage itself, if the additional resources that contribute to composing the webpage are inaccessible to the search engine, it can affect how the search engine interprets the webpage.

Rendering

When the search engine crawls a webpage, it will then “render” the page. This involves taking the HTML, JavaScript and cascading stylesheet (CSS) information to generate how the page will appear to desktop and/or mobile users. 

This is important in order for the search engine to be able to understand how the webpage content is displayed in context. Processing the JavaScript helps ensure they may have all the content that a human user would see when visiting the page. 

The search engines categorize the rendering step as a subprocess within the crawling stage. I listed it here as a separate step in the process because fetching a webpage and then parsing the content in order to understand how it would appear composed in a browser are two distinct processes. 

Google uses the same rendering engine used by the Google Chrome browser, called “Rendertron” which is built off the open-source Chromium browser system. 

Bingbot uses Microsoft Edge as its engine to run JavaScript and render webpages. It’s also now built upon the Chromium-based browser, so it essentially renders webpages very equivalently to the way that Googlebot does. 

Google stores copies of the pages in their repository in a compressed format. It seems likely that Microsoft Bing does so as well (but I have not found documentation confirming this). Some search engines may store a shorthand version of webpages in terms of just the visible text, stripped of all the formatting.

Rendering mostly becomes an issue in SEO for pages that have key portions of content dependent upon JavaScript/AJAX. 

Both Google and Microsoft Bing will execute JavaScript in order to see all the content on the page, and more complex JavaScript constructs can be challenging for the search engines to operate. 

I have seen JavaScript-constructed webpages that were essentially invisible to the search engines, resulting in severely nonoptimal webpages that would not be able to rank for their search terms. 

I have also seen instances where infinite-scrolling category pages on ecommerce websites did not perform well on search engines because the search engine could not see as many of the products’ links.

Other conditions can also interfere with rendering. For instance, when there is one or more JaveScript or CSS files inaccessible to the search engine bots due to being in subdirectories disallowed by robots.txt, it will be impossible to fully process the page. 

Googlebot and Bingbot largely will not index pages that require cookies. Pages that conditionally deliver some key elements based on cookies might also not get rendered fully or properly. 

Indexing

Once a page has been crawled and rendered, the search engines further process the page to determine if it will be stored in the index or not, and to understand what the page is about. 

The search engine index is functionally similar to an index of words found at the end of a book. 

A book’s index will list all the important words and topics found in the book, listing each word alphabetically, along with a list of the page numbers where the words/topics will be found. 

A search engine index contains many keywords and keyword sequences, associated with a list of all the webpages where the keywords are found. 

The index bears some conceptual resemblance to a database lookup table, which may have originally been the structure used for search engines. But the major search engines likely now use something a couple of generations more sophisticated to accomplish the purpose of looking up a keyword and returning all the URLs relevant to the word. 

The use of functionality to lookup all pages associated with a keyword is a time-saving architecture, as it would require excessively unworkable amounts of time to search all webpages for a keyword in real-time, each time someone searches for it. 

Not all crawled pages will be kept in the search index, for various reasons. For instance, if a page includes a robots meta tag with a “noindex” directive, it instructs the search engine to not include the page in the index.

Similarly, a webpage may include an X-Robots-Tag in its HTTP header that instructs the search engines not to index the page.

In yet other instances, a webpage’s canonical tag may instruct a search engine that a different page from the present one is to be considered the main version of the page, resulting in other, non-canonical versions of the page to be dropped from the index. 

Google has also stated that webpages may not be kept in the index if they are of low quality (duplicate content pages, thin content pages, and pages containing all or too much irrelevant content). 

There has also been a long history that suggests that websites with insufficient collective PageRank may not have all of their webpages indexed – suggesting that larger websites with insufficient external links may not get indexed thoroughly. 

Insufficient crawl budget may also result in a website not having all of its pages indexed.

A major component of SEO is diagnosing and correcting when pages do not get indexed. Because of this, it is a good idea to thoroughly study all the various issues that can impair the indexing of webpages.

Ranking

Ranking of webpages is the stage of search engine processing that is probably the most focused upon. 

Once a search engine has a list of all the webpages associated with a particular keyword or keyword phrase, it then must determine how it will order those pages when a search is conducted for the keyword. 

If you work in the SEO industry, you likely will already be pretty familiar with some of what the ranking process involves. The search engine’s ranking process is also referred to as an “algorithm”. 

The complexity involved with the ranking stage of search is so huge that it alone merits multiple articles and books to describe. 

There are a great many criteria that can affect a webpage’s rank in the search results. Google has said there are more than 200 ranking factors used by its algorithm.

Within many of those factors, there can also be up to 50 “vectors” – things that can influence a single ranking signal’s impact on rankings. 

PageRank is Google’s earliest version of its ranking algorithm invented in 1996. It was built off a concept that links to a webpage – and the relative importance of the sources of the links pointing to that webpage – could be calculated to determine the page’s ranking strength relative to all other pages. 

A metaphor for this is that links are somewhat treated as votes, and pages with the most votes will win out in ranking higher than other pages with fewer links/votes. 

Fast forward to 2022 and a lot of the old PageRank algorithm’s DNA is still embedded in Google’s ranking algorithm. That link analysis algorithm also influenced many other search engines that developed similar types of methods. 

The old Google algorithm method had to process over the links of the web iteratively, passing the PageRank value around among pages dozens of times before the ranking process was complete. This iterative calculation sequence across many millions of pages could take nearly a month to complete. 

Nowadays, new page links are introduced every day, and Google calculates rankings in a sort of drip method – allowing for pages and changes to be factored in much more rapidly without necessitating a month-long link calculation process.

Additionally, links are assessed in a sophisticated manner – revoking or reducing the ranking power of paid links, traded links, spammed links, non-editorially endorsed links and more. 

Broad categories of factors beyond links influence the rankings as well, including: 

Conclusion

Understanding the key stages of search is a table-stakes item for becoming a professional in the SEO industry. 

Some personalities in social media think that not hiring a candidate just because they don’t know the differences between crawling, rendering, indexing and ranking was “going too far” or “gate-keeping”. 

It’s a good idea to know the distinctions between these processes. However, I would not consider having a blurry understanding of such terms to be a deal-breaker.

SEO professionals come from a variety of backgrounds and experience levels. What’s important is that they are trainable enough to learn and reach a foundational level of understanding.

Opinions expressed in this article are those of the guest author and not necessarily Search Engine Land. Staff authors are listed here.

About The Author

https://searchengineland.com/search-stages-crawling-rendering-indexing-ranking-387057

Google has announced several new enhancements to Google Search today that focus on improving the overall quality of the search results, while at the same time helping searchers evaluate the search quality of the results presented to them.

Google has made improvements to featured snippets, its content advisories, and the about this result.

Featured Snippets callouts now uses MUM and general consensus

Featured snippets in Google Search will now use MUM to help understand if there is a general consensus for the information Google shows as callouts in these featured snippets. Google said that its “systems can now understand the notion of consensus” by using MUM, Multitask Unified Model.

MUM has not been used to date in too many applications within Google Search, limited to COVID vaccine names, Google Lens features and some other applications – more on that here, including now featured snippets.

Now, with the help of MUM, Google can understand if there is a consensus across the web to help highlight the callout portions of the featured snippets. Consensus-based techniques, according to Pandu Nayak, Vice President of Search and Fellow, Google, have meaningfully improved the quality of the featured snippet callouts. It is important to note that this does not come to mean that featured snippets will show facts, it does not necessarily do that but it does help improve the overall quality of featured snippets callouts.

Here is an example where Google’s featured snippet callouts are improved. In the screenshot below, Google will now highlight this callout, the word or words called out above the featured snippet in a larger font, to provide a better answer for the searcher.

Here is what this looks like without this feature:

Can consensus be spammed? Pandu Nayak explained these featured snippets are generally taken from the top-ranked results, so he is hopeful that those top-ranked results in Google Search are not spammy. It is important to note that consensus is not being used as a ranking factor but rather being used for callouts for featured snippets.

False premise queries reduced by 40%

Another advancement with featured snippets is around what Google calls “false premise” queries. Queries that may be inaccurate or factually incorrect but are nevertheless used by some searchers in Google Search. Google said it has improved what featured snippets it shows for queries that contain information for things that did not happen.

Google will now show you information that is accurate and remove the false part.  Google said it will show fewer featured snippets that may show false or inaccurate information. Google said it reduced these occurrences for triggering featured snippets by about 40% in Google Search.

An example Google provided was for a search on [when did snoopy assassinate Abraham Lincoln]. Now, instead of showing information about snoopy, who obviously did not assassinate Abraham Lincoln, Google will ignore the snoopy part and show you the consensus on the web around this answer.

Pandu Nayak added that this also helps with the people also ask, since those are powered by featured snippets.

Here is how it might look:

Content advisories expanded to low confidence results

In April 2020, Google launched content advisories in Google Search with the aim of communicating to searchers that the search results are not 100% reliable either because they are new or Google does not have enough information about the topic yet.

Google said it is now expanding content advisories to searches where its systems don’t have high confidence in the overall quality of the results available for the search. Google said this “does not mean that no helpful information is available, or that a particular result is low-quality.” These notices provide context about the whole set of results on the page, and you can always see the results for your query, even when the advisory is present.

About This Result expanding as well

In February 2021, Google launched the about this result to communicate to searchers, before they click on the result, more information about that search result snippet they are looking at. Google has expanded the feature in terms of showing more details in more areas, as well as why the result is ranking for the query. Google now said this feature has been used over 2.4 billion times since it launched.

Later this year, Google is expanding in Pouguese (PT), French (FR), Italian (IT), German (DE), Dutch (NL), Spanish (ES), Japanese (JP), and Indonesian (ID) languages. Google also added the about this result to the Google app.

Google is also expanding what information is shown in the about this result, including how widely a source is circulated, online reviews about a source or company, whether a company is owned by another entity, or even when Google Search can’t find much information about a source.

About The Author

Barry Schwartz a Contributing Editor to Search Engine Land and a member of the programming team for SMX events. He owns RustyBrick, a NY based web consulting firm. He also runs Search Engine Roundtable, a popular search blog on very advanced SEM topics. Barry can be followed on Twitter here.

https://searchengineland.com/google-steps-up-featured-snippets-with-mum-reducing-false-premise-results-by-40-387085