.Google's John Mueller answered a question concerning why Google.com indexes pages that are actually forbidden from creeping by robots.txt and also why the it is actually safe to dismiss the relevant Browse Console records about those creeps.Robot Traffic To Query Parameter URLs.The person talking to the question chronicled that crawlers were actually generating hyperlinks to non-existent question criterion Links (? q= xyz) to web pages along with noindex meta tags that are additionally blocked in robots.txt. What caused the inquiry is that Google.com is actually creeping the links to those webpages, getting shut out through robots.txt (without envisioning a noindex robotics meta tag) then acquiring shown up in Google.com Look Console as "Indexed, though obstructed by robots.txt.".The individual inquired the following question:." However right here's the significant inquiry: why would Google.com mark pages when they can not also view the web content? What's the advantage during that?".Google's John Mueller confirmed that if they can not creep the web page they can't observe the noindex meta tag. He also helps make an exciting reference of the site: hunt driver, recommending to ignore the outcomes considering that the "average" users will not view those outcomes.He composed:." Yes, you are actually right: if we can not crawl the webpage, our company can't view the noindex. That stated, if our experts can't creep the webpages, after that there is actually not a great deal for our team to mark. Therefore while you might see a few of those pages with a targeted internet site:- question, the normal consumer won't observe them, so I definitely would not bother it. Noindex is actually additionally great (without robots.txt disallow), it only indicates the Links are going to wind up being actually crept (and end up in the Search Console report for crawled/not listed-- neither of these conditions induce problems to the remainder of the internet site). The vital part is that you do not create all of them crawlable + indexable.".Takeaways:.1. Mueller's solution affirms the restrictions in using the Site: hunt progressed hunt operator for diagnostic causes. Among those explanations is actually due to the fact that it's not hooked up to the routine hunt mark, it's a different factor altogether.Google's John Mueller talked about the website hunt driver in 2021:." The short solution is actually that a web site: inquiry is actually not suggested to become full, nor used for diagnostics reasons.A website question is a details type of hunt that limits the outcomes to a certain website. It is actually generally merely the word website, a bowel, and afterwards the site's domain name.This inquiry confines the end results to a details web site. It is actually not implied to become an extensive collection of all the web pages from that site.".2. Noindex tag without using a robots.txt is actually alright for these type of situations where a bot is connecting to non-existent pages that are acquiring found out through Googlebot.3. URLs with the noindex tag will definitely create a "crawled/not recorded" item in Explore Console which those will not have an adverse impact on the remainder of the website.Read the question and respond to on LinkedIn:.Why would certainly Google index pages when they can't even see the material?Featured Photo through Shutterstock/Krakenimages. com.