{"id":10430,"date":"2022-06-01T09:45:24","date_gmt":"2022-06-01T07:45:24","guid":{"rendered":"https:\/\/wajari.com\/blog\/indexing-in-google-and-the-mother-that-bore-them\/"},"modified":"2025-02-28T11:17:15","modified_gmt":"2025-02-28T10:17:15","slug":"indexing-in-google","status":"publish","type":"post","link":"https:\/\/wajari.com\/en\/blog\/indexing-in-google\/","title":{"rendered":"Indexing in Google and the mother that bore them"},"content":{"rendered":"<div id=\"ez-toc-container\" class=\"ez-toc-v2_0_82_2 counter-hierarchy ez-toc-counter ez-toc-grey ez-toc-container-direction\">\n<div class=\"ez-toc-title-container\">\n<p class=\"ez-toc-title\" style=\"cursor:inherit\">Tabla de contenidos<\/p>\n<span class=\"ez-toc-title-toggle\"><a href=\"#\" class=\"ez-toc-pull-right ez-toc-btn ez-toc-btn-xs ez-toc-btn-default ez-toc-toggle\" aria-label=\"Toggle Table of Content\"><span class=\"ez-toc-js-icon-con\"><span class=\"\"><span class=\"eztoc-hide\" style=\"display:none;\">Toggle<\/span><span class=\"ez-toc-icon-toggle-span\"><svg style=\"fill: #999;color:#999\" xmlns=\"http:\/\/www.w3.org\/2000\/svg\" class=\"list-377408\" width=\"20px\" height=\"20px\" viewBox=\"0 0 24 24\" fill=\"none\"><path d=\"M6 6H4v2h2V6zm14 0H8v2h12V6zM4 11h2v2H4v-2zm16 0H8v2h12v-2zM4 16h2v2H4v-2zm16 0H8v2h12v-2z\" fill=\"currentColor\"><\/path><\/svg><svg style=\"fill: #999;color:#999\" class=\"arrow-unsorted-368013\" xmlns=\"http:\/\/www.w3.org\/2000\/svg\" width=\"10px\" height=\"10px\" viewBox=\"0 0 24 24\" version=\"1.2\" baseProfile=\"tiny\"><path d=\"M18.2 9.3l-6.2-6.3-6.2 6.3c-.2.2-.3.4-.3.7s.1.5.3.7c.2.2.4.3.7.3h11c.3 0 .5-.1.7-.3.2-.2.3-.5.3-.7s-.1-.5-.3-.7zM5.8 14.7l6.2 6.3 6.2-6.3c.2-.2.3-.5.3-.7s-.1-.5-.3-.7c-.2-.2-.4-.3-.7-.3h-11c-.3 0-.5.1-.7.3-.2.2-.3.5-.3.7s.1.5.3.7z\"\/><\/svg><\/span><\/span><\/span><\/a><\/span><\/div>\n<nav><ul class='ez-toc-list ez-toc-list-level-1 ' ><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-1\" href=\"https:\/\/wajari.com\/en\/blog\/indexing-in-google\/#What_is_Google_indexing\" >What is Google indexing?<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-2\" href=\"https:\/\/wajari.com\/en\/blog\/indexing-in-google\/#How_is_indexing_controlled\" >How is indexing controlled?<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-3\" href=\"https:\/\/wajari.com\/en\/blog\/indexing-in-google\/#Meta_robots_tag_syntax\" >Meta robots tag syntax<\/a><ul class='ez-toc-list-level-3' ><li class='ez-toc-heading-level-3'><a class=\"ez-toc-link ez-toc-heading-4\" href=\"https:\/\/wajari.com\/en\/blog\/indexing-in-google\/#noindex_follow\" >noindex, follow<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-3'><a class=\"ez-toc-link ez-toc-heading-5\" href=\"https:\/\/wajari.com\/en\/blog\/indexing-in-google\/#index_nofollow\" >index, nofollow<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-3'><a class=\"ez-toc-link ez-toc-heading-6\" href=\"https:\/\/wajari.com\/en\/blog\/indexing-in-google\/#noindex_nofollow\" >noindex, nofollow<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-3'><a class=\"ez-toc-link ez-toc-heading-7\" href=\"https:\/\/wajari.com\/en\/blog\/indexing-in-google\/#Index_follow\" >Index, follow<\/a><\/li><\/ul><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-8\" href=\"https:\/\/wajari.com\/en\/blog\/indexing-in-google\/#Is_there_a_difference_between_robotstxt_and_meta_robots_at_the_crawling_level\" >Is there a difference between robots.txt and meta robots at the crawling level?<\/a><ul class='ez-toc-list-level-3' ><li class='ez-toc-heading-level-3'><a class=\"ez-toc-link ez-toc-heading-9\" href=\"https:\/\/wajari.com\/en\/blog\/indexing-in-google\/#Other_directives_for_meta_robots\" >Other directives for meta robots<\/a><\/li><\/ul><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-10\" href=\"https:\/\/wajari.com\/en\/blog\/indexing-in-google\/#Why_are_there_currently_problems_with_indexing\" >Why are there currently problems with indexing?<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-11\" href=\"https:\/\/wajari.com\/en\/blog\/indexing-in-google\/#Search_Console_Coverage_Report\" >Search Console Coverage Report<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-12\" href=\"https:\/\/wajari.com\/en\/blog\/indexing-in-google\/#Quick_solution_to_indexing_failure\" >(Quick) solution to indexing failure<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-13\" href=\"https:\/\/wajari.com\/en\/blog\/indexing-in-google\/#Slow_solution_for_indexing_your_content\" >(Slow) solution for indexing your content<\/a><ul class='ez-toc-list-level-3' ><li class='ez-toc-heading-level-3'><a class=\"ez-toc-link ez-toc-heading-14\" href=\"https:\/\/wajari.com\/en\/blog\/indexing-in-google\/#1_Analyze_your_sitemapxml\" >1. Analyze your sitemap.xml<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-3'><a class=\"ez-toc-link ez-toc-heading-15\" href=\"https:\/\/wajari.com\/en\/blog\/indexing-in-google\/#2_Check_your_robotstxt\" >2. Check your robots.txt<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-3'><a class=\"ez-toc-link ez-toc-heading-16\" href=\"https:\/\/wajari.com\/en\/blog\/indexing-in-google\/#3_Analyze_the_URLs_not_indexed_in_Search_Console_using_the_inspector\" >3. Analyze the URLs not indexed in Search Console using the inspector.<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-3'><a class=\"ez-toc-link ez-toc-heading-17\" href=\"https:\/\/wajari.com\/en\/blog\/indexing-in-google\/#4_Analyze_your_internal_links_and_correct_errors\" >4. Analyze your internal links and correct errors<\/a><\/li><\/ul><\/li><\/ul><\/nav><\/div>\n\n<p>Currently discovered but not indexed. A Search Console status that takes us head over heels.  <\/p>\n\n<p>What should we do if <strong>your website is not being indexed<\/strong>?  <\/p>\n\n<p>It is an error that is being generated in many pages and I will discuss several methods to <strong>improve your coverage rate<\/strong> and not die trying.  <\/p>\n\n<p>For years in my <a href=\"https:\/\/wajari.com\/en\/about\/\">talks <\/a>I used to use the analogy that <strong>Google <\/strong>is like a bitch-swallower.  <\/p>\n\n<p><strong>It indexes <\/strong>everything very easily without you noticing it and therefore it is important to be careful with your indexing strategy.  <\/p>\n\n<p>That slide I deleted a long time ago, because things have changed.  <\/p>\n\n<p>For some time now, I have been detecting on my own websites, my clients&#8217; websites and the number of queries that I usually come across (in forums, support, Twitter, communities):  <\/p>\n\n<ul class=\"wp-block-list\"><li>What am I doing wrong?<\/li><li>Why doesn&#8217;t <strong>Google <\/strong>index me?<\/li><li>I get a notice in the Search Console coverage report of: &#8220;Discovered, but not indexed&#8221; in the excluded section.<\/li><li>Did I break something? Was it in a plugin update, from WordPress? (I saw this on <a href=\"https:\/\/es.wordpress.org\/support\/\" rel=\"noopener\">WP support<\/a>).<\/li><li>My website was hacked and since then Google has not indexed anything for me, etc.<\/li><\/ul>\n\n<p>And the truth is that it is normal that people far from the SEO world have these doubts. I&#8217;ve seen it in developers who have been in the digital world for years.  <\/p>\n\n<p>They&#8217;re not in the business, they&#8217;ve always had their content indexed without much trouble, and suddenly, a disturbance in the force, and the mother who bore them!  <\/p>\n\n<p>I spoiler: &#8220;You are NOT doing anything wrong&#8221;. A priori let&#8217;s go. But it needs to be analyzed.  <\/p>\n\n<p>From my point of view, <strong>Google does not communicate these problems in a clear way<\/strong>, this causes theories to be generated, when in reality, we are only witnesses of a possible &#8220;bug&#8221; or technical failure.  <\/p>\n\n<p>In many cases, these will be updates or <strong>changes to your indexing policy<\/strong>.  <\/p>\n\n<p>So I thought it would be useful to explain the process, the cases I see and possible solutions for this nuisance, to which the almighty search engine subjects us.  <\/p>\n\n<p>But first let&#8217;s go to the origin.  <\/p>\n\n<div style=\"height:30px\" aria-hidden=\"true\" class=\"wp-block-spacer\"><\/div>\n<div class=\"wp-block-image\">\n<figure class=\"aligncenter size-full\"><img loading=\"lazy\" decoding=\"async\" width=\"567\" height=\"367\" src=\"https:\/\/wajari.com\/wp-content\/uploads\/2022\/05\/spider.png\" alt=\"Google Spider according to Wajari\" class=\"wp-image-4979\" srcset=\"https:\/\/wajari.com\/wp-content\/uploads\/2022\/05\/spider.png 567w, https:\/\/wajari.com\/wp-content\/uploads\/2022\/05\/spider-300x194.png 300w\" sizes=\"auto, (max-width: 567px) 100vw, 567px\" \/><\/figure>\n<\/div>\n<div style=\"height:30px\" aria-hidden=\"true\" class=\"wp-block-spacer\"><\/div>\n\n<h2 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"What_is_Google_indexing\"><\/span>What is Google indexing?<span class=\"ez-toc-section-end\"><\/span><\/h2>\n\n<p>Search engines such as Google are composed of <strong>3<\/strong> essential <strong>components<\/strong>:  <\/p>\n\n<ol class=\"wp-block-list\"><li>A <strong>crawler <\/strong>that crawls our website. In the case of Google: <em>Googlebot<\/em>.<\/li><li>A <strong>database<\/strong>. This is what we can call indexing, when a web page arrives, Googlebot crawls it and incorporates it into its database to make it available for human searches. Best analogy: As if I were a librarian. It registers the book (web) and its content (pages) and enters it in its database.<\/li><li><strong>Algorithms<\/strong>. They organize information based on relevance and authority when a person does a search.<\/li><\/ol>\n\n<p>It is a theoretically simple process and for years you had to be very careful with the content you had on your website, because very often things were indexed that you didn&#8217;t want, for example:  <\/p>\n\n<ul class=\"wp-block-list\"><li>Cookie notices<\/li><li>Acknowledgements pages<\/li><li>Contents<em> lorem ipsum<\/em><\/li><li>Development versions of your website that you put in subdomains, etc.<\/li><\/ul>\n\n<div style=\"height:30px\" aria-hidden=\"true\" class=\"wp-block-spacer\"><\/div>\n\n<h2 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"How_is_indexing_controlled\"><\/span>How is indexing controlled?  <span class=\"ez-toc-section-end\"><\/span><\/h2>\n\n<p>With <strong>meta robots<\/strong>. As I told it in audio (and in writing) in my abandoned podcast<a href=\"https:\/\/seoparawp.com\/podcast\/episodio-11-meta-robots\/\" target=\"_blank\" rel=\"noreferrer noopener\">(SEO for WP: Meta robots<\/a>) and that I allow myself to repeat part of that content in this post:  <\/p>\n\n<p>Meta robots are a <strong>tag in HTML<\/strong> that gives an instruction to search engines.<\/p>\n\n<p>Like <strong>robots.txt<\/strong>, we can block search engines, but in the case of robots.txt, some guidelines can be ignored, especially if a URL receives an external link and is detected.  <\/p>\n\n<p>Header tags are usually the <strong>best way to control <\/strong>the behavior of each URL.  <\/p>\n\n<p>As <a href=\"https:\/\/www.humanlevel.com\/diccionario-marketing-online\/meta-robots\" target=\"_blank\" rel=\"noreferrer noopener\">Fernando Maci\u00e1<\/a> points out in his digital marketing dictionary:<\/p>\n\n<blockquote class=\"wp-block-quote is-layout-flow wp-block-quote-is-layout-flow\"><p>&#8220;Meta robots allows you to control how a page should be indexed and how it is displayed to users on the search results page.&#8221;<\/p><cite>Fernando Macia<\/cite><\/blockquote>\n\n<p>It doesn&#8217;t get any clearer than that.<\/p>\n\n<p>Also, in robots.txt we block a URL completely, while with meta robots we can have a URL that is still passing link juice or popularity, but we decide not to have it appear in Google&#8217;s indexes.<\/p>\n\n<div style=\"height:30px\" aria-hidden=\"true\" class=\"wp-block-spacer\"><\/div>\n<div class=\"wp-block-image\">\n<figure class=\"aligncenter size-full\"><img loading=\"lazy\" decoding=\"async\" width=\"640\" height=\"359\" src=\"https:\/\/wajari.com\/wp-content\/uploads\/2022\/05\/meta-robots.jpg\" alt=\"Meta robots\" class=\"wp-image-4985\" srcset=\"https:\/\/wajari.com\/wp-content\/uploads\/2022\/05\/meta-robots.jpg 640w, https:\/\/wajari.com\/wp-content\/uploads\/2022\/05\/meta-robots-300x168.jpg 300w\" sizes=\"auto, (max-width: 640px) 100vw, 640px\" \/><\/figure>\n<\/div>\n<h2 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"Meta_robots_tag_syntax\"><\/span>Meta robots tag syntax<span class=\"ez-toc-section-end\"><\/span><\/h2>\n\n<p>Very simple: <meta name=\"robots\" content=\"the value we want\"\/> and these are the options we can define:<\/p>\n\n<div style=\"height:30px\" aria-hidden=\"true\" class=\"wp-block-spacer\"><\/div>\n\n<h3 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"noindex_follow\"><\/span>noindex, follow<span class=\"ez-toc-section-end\"><\/span><\/h3>\n\n<pre class=\"wp-block-code\"><code>&lt;meta name=\"robots\" content=\"noindex, follow\"\/&gt;<\/code><\/pre>\n\n<p>In this case with the <em>noindex <\/em>we tell search engines NOT to index this content but you can<em>follow<\/em> the links.<\/p>\n\n<p>By following the <strong>links <\/strong>we maintain the link transfer and associated popularity juice.<\/p>\n\n<p>This is the most typical solution when you want to avoid indexing a URL that may be considered as thin content or duplicate content from other sections of your website.<\/p>\n\n<p>Very common in search results, which generates a change in the URL with the search term. In label files, author, etc.<\/p>\n\n<p>If you have <strong>Yoast<\/strong>, <strong>RankMath <\/strong>or any other SEO plugin installed, do a test: perform a search in your WP and check the source code of the result. You will probably see this tag in the header.<\/p>\n\n<div style=\"height:30px\" aria-hidden=\"true\" class=\"wp-block-spacer\"><\/div>\n\n<h3 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"index_nofollow\"><\/span>index, nofollow<span class=\"ez-toc-section-end\"><\/span><\/h3>\n\n<pre class=\"wp-block-code\"><code>&lt;meta name=\"robots\" content=\"index, nofollow\"\/&gt;<\/code><\/pre>\n\n<p>In this case the opposite is true, we tell you that you can index this URL but do NOT follow the links, therefore, they will not usually transmit their value.<\/p>\n\n<p>As <strong>Tom\u00e1s de Teresa<\/strong> points out (in an article that no longer exists, so I can&#8217;t link to it), it&#8217;s the ideal combination when you don&#8217;t back links from a particular URL, imagine user-created pages, for example in a <strong>forum<\/strong>.<\/p>\n\n<div style=\"height:30px\" aria-hidden=\"true\" class=\"wp-block-spacer\"><\/div>\n\n<h3 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"noindex_nofollow\"><\/span>noindex, nofollow<span class=\"ez-toc-section-end\"><\/span><\/h3>\n\n<pre class=\"wp-block-code\"><code>&lt;meta name=\"robots\" content=\"noindex, nofollow\"\/&gt;<\/code><\/pre>\n\n<p>We avoid indexing and tracking links. It is a form of total blocking of that URL. Its use is not very common.<\/p>\n\n<div style=\"height:30px\" aria-hidden=\"true\" class=\"wp-block-spacer\"><\/div>\n\n<h3 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"Index_follow\"><\/span>Index, follow<span class=\"ez-toc-section-end\"><\/span><\/h3>\n\n<p>There is a fourth tag which is index, follow but this tag is not necessary to put it because it is the normal behavior, in which a URL is identified, the links are followed and the content is indexed in search engines.<\/p>\n\n<p>A clarification: You don&#8217;t need to know HTML, obviously in technologies like WordPress plugins make this task very easy, just check or uncheck options and that&#8217;s it.  <\/p>\n\n<div style=\"height:30px\" aria-hidden=\"true\" class=\"wp-block-spacer\"><\/div>\n\n<h2 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"Is_there_a_difference_between_robotstxt_and_meta_robots_at_the_crawling_level\"><\/span>Is there a difference between robots.txt and meta robots at the crawling level?<span class=\"ez-toc-section-end\"><\/span><\/h2>\n\n<p>As Fernando Maci\u00e1 tells us, yes of course, remember that robots.txt is usually one of the first files that search engines will check.<\/p>\n\n<p>If we mark a <strong>disallow<\/strong> to a directory within that file, in principle Google will not waste time crawling that directory, while if it reaches a URL with the noindex tag, it does a crawl.<\/p>\n\n<p>In addition, with the robots.txt we can define patterns (imagine blocking directories or subsets of information) while the robots meta tag goes in each URL.<\/p>\n\n<p>What should we take into account in these two forms?<\/p>\n\n<p>These two methods are very necessary to control crawling and indexing.<\/p>\n\n<p>For this reason it is important to leave in the <strong>meta robots<\/strong> the directives that we want, in a way to control the final indexing that Google makes of our web.  <\/p>\n\n<div style=\"height:30px\" aria-hidden=\"true\" class=\"wp-block-spacer\"><\/div>\n\n<h3 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"Other_directives_for_meta_robots\"><\/span>Other directives for meta robots<span class=\"ez-toc-section-end\"><\/span><\/h3>\n\n<p>We can use more elements, some examples:<\/p>\n\n<ul class=\"wp-block-list\"><li><strong>archive \/ noarchive<\/strong>: whether or not to store the web content in the internal cache.<\/li><li><strong>noimageindex<\/strong>: not to index the images of the page.<\/li><\/ul>\n\n<p>And some other examples, but with less frequent uses that Google makes available to us in its <a href=\"https:\/\/developers.google.com\/search\/reference\/robots_meta_tag?hl=es-419\" target=\"_blank\" rel=\"noreferrer noopener\">help page for developers<\/a>.<\/p>\n<div class=\"wp-block-image\">\n<figure class=\"aligncenter\"><img decoding=\"async\" src=\"https:\/\/seoparawp.com\/wp-content\/uploads\/2019\/04\/X-Robots-Tag-Google-Developers.png\" alt=\"Meta robot directives\" class=\"wp-image-1759\"\/><\/figure>\n<\/div>\n<div style=\"height:30px\" aria-hidden=\"true\" class=\"wp-block-spacer\"><\/div>\n\n<h2 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"Why_are_there_currently_problems_with_indexing\"><\/span>Why are there currently problems with indexing?<span class=\"ez-toc-section-end\"><\/span><\/h2>\n\n<p>I would venture to say that in the last year, we have begun to see changes in this regard.  <\/p>\n\n<p>Everything was no longer indexed as easily as before.  <\/p>\n\n<p>But it didn&#8217;t happen every time. I have personally detected the following most frequent situations:  <\/p>\n\n<ol class=\"wp-block-list\"><li><strong>New pages<\/strong> with little history.<\/li><li>Newly registered <strong>domains <\/strong>(and with few external links).<\/li><li>Pages with &#8220;not very relevant&#8221; content in the eyes of the search engine, the mother who bore them!<\/li><li><strong>Websites that have been hacked<\/strong> recently, even if you don&#8217;t get the security warning. Typical case in which a lot of &#8220;Russian or Chinese tricks&#8221; have been indexed and even if you manage to clean up the site, it still affects your coverage rate.<\/li><li><strong>Slow web sites<\/strong> with WPO<em>(web performance optimization<\/em>) problems<\/li><li>Rare cases of websites that <strong>did not allow crawling <\/strong>correctly.<\/li><\/ol>\n\n<p>According to <a href=\"https:\/\/support.google.com\/webmasters\/answer\/7440203?hl=es\" target=\"_blank\" rel=\"noreferrer noopener\">Search Console&#8217;<\/a> s own <a href=\"https:\/\/support.google.com\/webmasters\/answer\/7440203?hl=es\" target=\"_blank\" rel=\"noreferrer noopener\">documentation<\/a>, they point out that:  <\/p>\n\n<blockquote class=\"wp-block-quote is-layout-flow wp-block-quote-is-layout-flow\"><p><strong>Discovered: currently unindexed.<\/strong>  Google has found the page, but has not yet crawled it; probably because it has determined that, if it did, the website would be overloaded. Therefore, he has had to postpone the tracking.<\/p><cite>Search Console Documentation<\/cite><\/blockquote>\n\n<p>This explanation makes many people link it directly to web overloading due to WPO issues.  <\/p>\n\n<p>They explain it well in the <a href=\"https:\/\/www.contentkingapp.com\/academy\/index-coverage\/faq\/discovered-not-indexed\/\" target=\"_blank\" rel=\"noreferrer noopener\">Content King<\/a> post as possible causes:  <\/p>\n\n<ol class=\"wp-block-list\"><li><strong>Overloaded server<\/strong>, which means that Google cannot crawl correctly.<\/li><li><strong>Content overload<\/strong>. Your website has more content than the spider can crawl at that moment. This is undoubtedly an exceptional case and I believe it is reserved for excessively large websites.<\/li><li>Poor <strong>internal link<\/strong> structure.<\/li><li>Low quality content, which does not add value to the user.<\/li><\/ol>\n\n<p>I don&#8217;t doubt that there are cases like that, but most of the ones I&#8217;ve come across were not due to that cause (WPO and\/or crawl budget), but to cases of <strong>inefficient internal linking<\/strong> and that <strong>&#8220;directive&#8221; of content quality<\/strong>, that according to them, your content doesn&#8217;t have a specific value.  <\/p>\n\n<div style=\"height:30px\" aria-hidden=\"true\" class=\"wp-block-spacer\"><\/div>\n\n<h2 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"Search_Console_Coverage_Report\"><\/span>Search Console Coverage Report<span class=\"ez-toc-section-end\"><\/span><\/h2>\n\n<p>To detect if we have URLs in this situation, we have to go to the coverage report and mark <strong>excluded<\/strong>.  <\/p>\n<div class=\"wp-block-image\">\n<figure class=\"aligncenter size-full\"><img loading=\"lazy\" decoding=\"async\" width=\"949\" height=\"468\" src=\"https:\/\/wajari.com\/wp-content\/uploads\/2022\/05\/Informe-Cobertura-Search-Console.png\" alt=\"Search Console exclusion index\" class=\"wp-image-4974\" srcset=\"https:\/\/wajari.com\/wp-content\/uploads\/2022\/05\/Informe-Cobertura-Search-Console.png 949w, https:\/\/wajari.com\/wp-content\/uploads\/2022\/05\/Informe-Cobertura-Search-Console-300x148.png 300w, https:\/\/wajari.com\/wp-content\/uploads\/2022\/05\/Informe-Cobertura-Search-Console-768x379.png 768w\" sizes=\"auto, (max-width: 949px) 100vw, 949px\" \/><\/figure>\n<\/div>\n<p>This report will show us all cases of URLs that are NOT indexed. Most common cases:  <\/p>\n\n<ul class=\"wp-block-list\"><li>Excluded by <em>noindex<\/em> tag<\/li><li>Errors<\/li><li>Redirections<\/li><li>And a long etc. that are not relevant<\/li><\/ul>\n\n<p>But the ones that occupy us in this article:<\/p>\n\n<ol class=\"wp-block-list\"><li><strong>Tracked: currently not indexed<\/strong>. In this case they are usually indexed later without much problem and without any action on our part. We also find in this section many things that do not make sense to be <strong>indexed<\/strong> as the <strong>feed<\/strong>, sorting <strong>filters <\/strong>that are not well configured and have meta robots as index, etc..<\/li><li><strong>Discovered: currently unindexed<\/strong>. The case we seek to solve in this post.<\/li><\/ol>\n\n<div style=\"height:30px\" aria-hidden=\"true\" class=\"wp-block-spacer\"><\/div>\n\n<h2 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"Quick_solution_to_indexing_failure\"><\/span>(Quick) solution to indexing failure<span class=\"ez-toc-section-end\"><\/span><\/h2>\n\n<p><strong>Emilio Garcia<\/strong> on his excellent YouTube channel and podcast: <a href=\"https:\/\/campamentoweb.com\/\" target=\"_blank\" rel=\"noreferrer noopener\">Campamento Web<\/a> posted this video that is great.  <\/p>\n\n<p>It explains in a clear and simple way how to index your content using RankMath&#8217;s indexing API:  <\/p>\n\n<div style=\"height:30px\" aria-hidden=\"true\" class=\"wp-block-spacer\"><\/div>\n\n<figure class=\"wp-block-embed is-type-video is-provider-youtube wp-block-embed-youtube wp-embed-aspect-16-9 wp-has-aspect-ratio\"><div class=\"wp-block-embed__wrapper\">\n<iframe loading=\"lazy\" title=\"\u26fa C\u00f3mo INDEXAR R\u00c1PIDO tus URLs en Google (FUNCIONA 2022)\" width=\"1200\" height=\"675\" src=\"https:\/\/www.youtube.com\/embed\/SgvG-0evDnE?feature=oembed\" frameborder=\"0\" allow=\"accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share\" allowfullscreen><\/iframe>\n<\/div><\/figure>\n\n<div style=\"height:30px\" aria-hidden=\"true\" class=\"wp-block-spacer\"><\/div>\n\n<p>Emilio uses the methodology explained in this RankMath post: <a href=\"https:\/\/rankmath.com\/blog\/google-indexing-api\/\" target=\"_blank\" rel=\"noreferrer noopener\">Google indexing API<\/a>.  <\/p>\n\n<p>A word of caution. As we can read in RankMath&#8217;s post, and they are very clear about it, this <strong>Google indexing API<\/strong>, is specifically designed for:  <\/p>\n\n<blockquote class=\"wp-block-quote is-layout-flow wp-block-quote-is-layout-flow\"><p>Google recommends using the indexing API ONLY for JobPosting or BroadcastEvent embedded in VideoObject websites [tipos de datos estructurados]. During our tests, we found that it worked on any type of website with great results and we created this plugin to test it.<\/p><cite>RankMath<\/cite><\/blockquote>\n\n<p>Therefore, they clarify that this methodology is NOT for everyone. But it certainly works. Do you want a quick and good solution? This is your method.  <\/p>\n\n<div style=\"height:30px\" aria-hidden=\"true\" class=\"wp-block-spacer\"><\/div>\n\n<h2 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"Slow_solution_for_indexing_your_content\"><\/span>(Slow) solution for indexing your content<span class=\"ez-toc-section-end\"><\/span><\/h2>\n\n<p>As <a href=\"https:\/\/wajari.com\/en\/\">SEO consultants<\/a> we often come across situations like this, that on client websites, there are certain things that make us uneasy.  <\/p>\n\n<p>This solution is slower, but in general I have seen very good results: Patience and the mother who bore them!  <\/p>\n\n<p>Everything goes through:  <\/p>\n<div class=\"wp-block-image\">\n<figure class=\"aligncenter size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"1024\" height=\"232\" src=\"https:\/\/wajari.com\/wp-content\/uploads\/2022\/05\/Screaming-Modo-Lista-1024x232.png\" alt=\"Sitemap tracking with Screaming Frog\" class=\"wp-image-4975\" srcset=\"https:\/\/wajari.com\/wp-content\/uploads\/2022\/05\/Screaming-Modo-Lista-1024x232.png 1024w, https:\/\/wajari.com\/wp-content\/uploads\/2022\/05\/Screaming-Modo-Lista-300x68.png 300w, https:\/\/wajari.com\/wp-content\/uploads\/2022\/05\/Screaming-Modo-Lista-768x174.png 768w, https:\/\/wajari.com\/wp-content\/uploads\/2022\/05\/Screaming-Modo-Lista.png 1174w\" sizes=\"auto, (max-width: 1024px) 100vw, 1024px\" \/><\/figure>\n<\/div>\n<h3 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"1_Analyze_your_sitemapxml\"><\/span>1. Analyze your sitemap.xml<span class=\"ez-toc-section-end\"><\/span><\/h3>\n\n<ul class=\"wp-block-list\"><li>Analyze the <a href=\"https:\/\/wajari.com\/sitemap_index.xml\">sitemap.xml<\/a> of your website. Copy the URL.<\/li><li>Using any <strong>crawler <\/strong>like <a href=\"https:\/\/www.screamingfrog.co.uk\/\" target=\"_blank\" rel=\"noreferrer noopener\">Screaming Frog<\/a> in mode: List &gt; Import &gt; Download sitemap.xml and paste the sitemap address.<\/li><li>This will download and analyze only the sitemap (not the entire site).<\/li><\/ul>\n<div class=\"wp-block-image\">\n<figure class=\"aligncenter size-full\"><img loading=\"lazy\" decoding=\"async\" width=\"483\" height=\"701\" src=\"https:\/\/wajari.com\/wp-content\/uploads\/2022\/05\/Codigo-200-sitemap.png\" alt=\"Response codes with Screaming Frog\" class=\"wp-image-4976\" srcset=\"https:\/\/wajari.com\/wp-content\/uploads\/2022\/05\/Codigo-200-sitemap.png 483w, https:\/\/wajari.com\/wp-content\/uploads\/2022\/05\/Codigo-200-sitemap-207x300.png 207w\" sizes=\"auto, (max-width: 483px) 100vw, 483px\" \/><\/figure>\n<\/div>\n<ul class=\"wp-block-list\"><li>RULE: 100% of the response codes must be <strong>code 200<\/strong>.<\/li><li>There should be NO redirects (3xx), and NO errors (4xx). If you have mistakes: fix the house first.<\/li><li>If everything is perfect, you can resubmit to Google through Search Console:<\/li><\/ul>\n<div class=\"wp-block-image\">\n<figure class=\"aligncenter size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"1024\" height=\"466\" src=\"https:\/\/wajari.com\/wp-content\/uploads\/2022\/05\/Sitemaps-1024x466.png\" alt=\"Add Sitemap to Search Console\" class=\"wp-image-4977\" srcset=\"https:\/\/wajari.com\/wp-content\/uploads\/2022\/05\/Sitemaps-1024x466.png 1024w, https:\/\/wajari.com\/wp-content\/uploads\/2022\/05\/Sitemaps-300x137.png 300w, https:\/\/wajari.com\/wp-content\/uploads\/2022\/05\/Sitemaps-768x350.png 768w, https:\/\/wajari.com\/wp-content\/uploads\/2022\/05\/Sitemaps.png 1114w\" sizes=\"auto, (max-width: 1024px) 100vw, 1024px\" \/><\/figure>\n<\/div>\n<div style=\"height:30px\" aria-hidden=\"true\" class=\"wp-block-spacer\"><\/div>\n\n<h3 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"2_Check_your_robotstxt\"><\/span>2. Check your robots.txt<span class=\"ez-toc-section-end\"><\/span><\/h3>\n\n<p>For obvious reasons you should check that you are not blocking any directory or have <strong>defects in the syntax<\/strong> of this file.  <\/p>\n\n<p>You can use the web validation tool: <a href=\"https:\/\/technicalseo.com\/tools\/robots-txt\/\" target=\"_blank\" rel=\"noreferrer noopener\">Technical SEO Tools<\/a>.<\/p>\n\n<p>Is your sitemap linked? <a href=\"https:\/\/es.wordpress.org\/plugins\/seo-by-rank-math\/\" target=\"_blank\" rel=\"noreferrer noopener\">RankMath <\/a>does it by default. Other SEO plugins do not and you would have to add it manually.  <\/p>\n\n<p>The syntax is very simple, it is only advisable to put it at the end and <strong>leave a space between the<\/strong> <em>user-agent<\/em> <strong>directive<\/strong> and the sitemap. Being minimalist with this file is good advice. Example:  <\/p>\n\n<pre class=\"wp-block-code\"><code>User-agent: *\nDisallow: \/wp-admin\/\nAllow: \/wp-admin\/admin-ajax.php\n\nSitemap: https:\/\/wajari.com\/sitemap_index.xml<\/code><\/pre>\n\n<div style=\"height:30px\" aria-hidden=\"true\" class=\"wp-block-spacer\"><\/div>\n\n<h3 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"3_Analyze_the_URLs_not_indexed_in_Search_Console_using_the_inspector\"><\/span>3. Analyze the URLs not indexed in Search Console using the inspector.<span class=\"ez-toc-section-end\"><\/span><\/h3>\n\n<p>Google can give you clues as to what might be going on by using the inspector.<\/p>\n<div class=\"wp-block-image\">\n<figure class=\"aligncenter size-full\"><img loading=\"lazy\" decoding=\"async\" width=\"330\" height=\"351\" src=\"https:\/\/wajari.com\/wp-content\/uploads\/2022\/05\/URL-Inspection.png\" alt=\"URL inspection in Search Console\" class=\"wp-image-4978\" srcset=\"https:\/\/wajari.com\/wp-content\/uploads\/2022\/05\/URL-Inspection.png 330w, https:\/\/wajari.com\/wp-content\/uploads\/2022\/05\/URL-Inspection-282x300.png 282w\" sizes=\"auto, (max-width: 330px) 100vw, 330px\" \/><\/figure>\n<\/div>\n<p>In this link you have all the official documentation about the <a href=\"https:\/\/support.google.com\/webmasters\/answer\/9012289#url_not_on_google\" target=\"_blank\" rel=\"noreferrer noopener\">URL Inspection Tool<\/a>.  <\/p>\n\n<p>If there are no apparent errors, and it simply does not appear indexed, as you well know, you can check <strong>request indexing<\/strong>.  <\/p>\n\n<p>This usually works very well. Of course, if we are talking about a few URLs there is no problem to do it this way manually, but if we are talking about hundreds or thousands of URLs, you have to look for other options.  <\/p>\n\n<div style=\"height:30px\" aria-hidden=\"true\" class=\"wp-block-spacer\"><\/div>\n\n<h3 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"4_Analyze_your_internal_links_and_correct_errors\"><\/span>4. Analyze your internal links and correct errors<span class=\"ez-toc-section-end\"><\/span><\/h3>\n\n<p>Crawlers such as <strong>Screaming Frog<\/strong> allow us to analyze internal links. It would take a whole post to explain this point, but remember that links are essential.  <\/p>\n\n<p>If we do not have our contents well linked, it can be a negative factor for Google to discover the sections and add them to its database.  <\/p>\n\n<p>Keep Search Console errors and warnings online.<\/p>\n\n<p>Try to improve all the aspects recommended by the tool, from <strong>structured data<\/strong> to <strong>core web vitals<\/strong> that are a ranking factor and can affect both crawling and positioning.  <\/p>\n\n<p>And last but not least: Patience.  <\/p>\n\n<p>Google, like any company, makes mistakes.  <\/p>\n\n<p>In my experience with my clients, we have solved most situations by simply following these steps.  <\/p>\n\n<p>We must be aware that I do not even want to imagine the size of what is involved in indexing all the websites on the Internet. I understand that it represents a technological challenge for the Californian giant.  <\/p>\n\n<p>In some exceptional cases (media) I solved it using the <a href=\"https:\/\/developers.google.com\/search\/docs\/advanced\/sitemaps\/news-sitemap?hl=es\" target=\"_blank\" rel=\"noreferrer noopener\">news sitemaps<\/a>, which obviously do not apply to all websites; but it allowed to quickly recognize the contents that were created and indexed at the time.  <\/p>\n\n<p>Using plugins like <strong>RankMath <\/strong>it is quite convenient to track because if you have it connected with <strong>Search Console<\/strong>, you can see in the statistics tab the status of the index.  <\/p>\n\n<p>In that tab: &#8220;Index status&#8221; with a list of your URLs, showing you if it shows rich results, if it is indexed or not, etc.  <\/p>\n<div class=\"wp-block-image\">\n<figure class=\"aligncenter size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"1024\" height=\"770\" src=\"https:\/\/wajari.com\/wp-content\/uploads\/2022\/05\/Analiticas-Indexado-RankMath-1024x770.png\" alt=\"RankMath index status\" class=\"wp-image-4973\" srcset=\"https:\/\/wajari.com\/wp-content\/uploads\/2022\/05\/Analiticas-Indexado-RankMath-1024x770.png 1024w, https:\/\/wajari.com\/wp-content\/uploads\/2022\/05\/Analiticas-Indexado-RankMath-300x226.png 300w, https:\/\/wajari.com\/wp-content\/uploads\/2022\/05\/Analiticas-Indexado-RankMath-768x578.png 768w, https:\/\/wajari.com\/wp-content\/uploads\/2022\/05\/Analiticas-Indexado-RankMath.png 1226w\" sizes=\"auto, (max-width: 1024px) 100vw, 1024px\" \/><\/figure>\n<\/div>\n<div style=\"height:30px\" aria-hidden=\"true\" class=\"wp-block-spacer\"><\/div>\n\n<p>It also has an <a href=\"https:\/\/rankmath.com\/kb\/how-to-use-indexnow\/\" target=\"_blank\" rel=\"noreferrer noopener\">Instant Indexing<\/a> module, although it only works with <strong>Yandex <\/strong>and <strong>Bing<\/strong>. It does this automatically with changes to your posts or pages, or you can even do it manually. A good invention indeed.  <\/p>\n\n<div style=\"height:30px\" aria-hidden=\"true\" class=\"wp-block-spacer\"><\/div>\n\n<p style=\"font-size:25px\">Final words<\/p>\n\n<p>As usual in SEO: It depends. Your case may have multiple causes. I just recommend you to be patient and look for the best solution for your website.<\/p>\n\n<p>It may piss us off, it may look bad to us, and I get your point. I empathize with you. But that anger will not allow you to solve the mistake.  <\/p>\n\n<p>As a content creator or business, you want your website to show up in Google, everyone wants that.  <\/p>\n\n<p>I understand that it is something that will be improved and in any case, is a possibility to improve our website, both in terms of <strong>crawling<\/strong>, <strong>linking<\/strong>, <strong>authority<\/strong>, <strong>content<\/strong>, <strong>speed<\/strong>, etc..  <\/p>\n\n<p>I just hope this post helps you to see it from the <strong>calmness<\/strong>, and not from the uneasiness of: I did something wrong.  <\/p>\n\n<p>You are not alone in this world of Discovered: currently unindexed!<\/p>\n\n<p>Has something similar happened to you? I will be happy to hear your case in the comments.  <\/p>\n\n<p>Live long and prosper!  <\/p>\n\n<div style=\"height:51px\" aria-hidden=\"true\" class=\"wp-block-spacer\"><\/div>\n","protected":false},"excerpt":{"rendered":"<p>I explain what to do in case Google is not indexing your website and many pages appear in the excluded from indexing status in Search Console. It is an error that is being generated in many websites and I will discuss several methods to improve your coverage rate and not die trying.  <\/p>\n","protected":false},"author":1,"featured_media":10020,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"inline_featured_image":false,"_uag_custom_page_level_css":"","_jetpack_memberships_contains_paid_content":false,"footnotes":""},"categories":[70],"tags":[71,72],"class_list":["post-10430","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-seo-en","tag-google-en","tag-search-console-en"],"featured_image_urls_v2":{"full":["https:\/\/wajari.com\/wp-content\/uploads\/2022\/05\/2022-06-Index-Google.png",640,426,false],"thumbnail":["https:\/\/wajari.com\/wp-content\/uploads\/2022\/05\/2022-06-Index-Google-150x150.png",150,150,true],"medium":["https:\/\/wajari.com\/wp-content\/uploads\/2022\/05\/2022-06-Index-Google-300x200.png",300,200,true],"medium_large":["https:\/\/wajari.com\/wp-content\/uploads\/2022\/05\/2022-06-Index-Google.png",640,426,false],"large":["https:\/\/wajari.com\/wp-content\/uploads\/2022\/05\/2022-06-Index-Google.png",640,426,false],"1536x1536":["https:\/\/wajari.com\/wp-content\/uploads\/2022\/05\/2022-06-Index-Google.png",640,426,false],"2048x2048":["https:\/\/wajari.com\/wp-content\/uploads\/2022\/05\/2022-06-Index-Google.png",640,426,false]},"post_excerpt_stackable_v2":"<p>I explain what to do in case Google is not indexing your website and many pages appear in the excluded from indexing status in Search Console. It is an error that is being generated in many websites and I will discuss several methods to improve your coverage rate and not die trying.  <\/p>\n","category_list_v2":"<a href=\"https:\/\/wajari.com\/en\/categoria\/seo-en\/\" rel=\"category tag\">SEO<\/a>","author_info_v2":{"name":"Wajari Vel\u00e1squez","url":"https:\/\/wajari.com\/en\/author\/wajari\/"},"comments_num_v2":"0 comments","jetpack_featured_media_url":"https:\/\/wajari.com\/wp-content\/uploads\/2022\/05\/2022-06-Index-Google.png","uagb_featured_image_src":{"full":["https:\/\/wajari.com\/wp-content\/uploads\/2022\/05\/2022-06-Index-Google.png",640,426,false],"thumbnail":["https:\/\/wajari.com\/wp-content\/uploads\/2022\/05\/2022-06-Index-Google-150x150.png",150,150,true],"medium":["https:\/\/wajari.com\/wp-content\/uploads\/2022\/05\/2022-06-Index-Google-300x200.png",300,200,true],"medium_large":["https:\/\/wajari.com\/wp-content\/uploads\/2022\/05\/2022-06-Index-Google.png",640,426,false],"large":["https:\/\/wajari.com\/wp-content\/uploads\/2022\/05\/2022-06-Index-Google.png",640,426,false],"1536x1536":["https:\/\/wajari.com\/wp-content\/uploads\/2022\/05\/2022-06-Index-Google.png",640,426,false],"2048x2048":["https:\/\/wajari.com\/wp-content\/uploads\/2022\/05\/2022-06-Index-Google.png",640,426,false]},"uagb_author_info":{"display_name":"Wajari Vel\u00e1squez","author_link":"https:\/\/wajari.com\/en\/author\/wajari\/"},"uagb_comment_info":0,"uagb_excerpt":"I explain what to do in case Google is not indexing your website and many pages appear in the excluded from indexing status in Search Console. It is an error that is being generated in many websites and I will discuss several methods to improve your coverage rate and not die trying.","jetpack_sharing_enabled":true,"_links":{"self":[{"href":"https:\/\/wajari.com\/en\/wp-json\/wp\/v2\/posts\/10430","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/wajari.com\/en\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/wajari.com\/en\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/wajari.com\/en\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/wajari.com\/en\/wp-json\/wp\/v2\/comments?post=10430"}],"version-history":[{"count":5,"href":"https:\/\/wajari.com\/en\/wp-json\/wp\/v2\/posts\/10430\/revisions"}],"predecessor-version":[{"id":11330,"href":"https:\/\/wajari.com\/en\/wp-json\/wp\/v2\/posts\/10430\/revisions\/11330"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/wajari.com\/en\/wp-json\/wp\/v2\/media\/10020"}],"wp:attachment":[{"href":"https:\/\/wajari.com\/en\/wp-json\/wp\/v2\/media?parent=10430"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/wajari.com\/en\/wp-json\/wp\/v2\/categories?post=10430"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/wajari.com\/en\/wp-json\/wp\/v2\/tags?post=10430"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}