Robots.txt for SEO: what to block, what to allow and how to test it
What robots.txt can and cannot do for SEO, the syntax Google follows, what to block and never block, how errors are handled, and ready-to-copy files for a clini...
Free tools for SEO, AI search, Google Ads and local businesses. No sign-up. See all tools
Founder, DigitalSnot
Mohsin Beg is the founder of DigitalSnot, a digital marketing agency that helps businesses get found on Google and in AI answers. Mohsin writes the DigitalSnot guides on technical SEO, AI search (GEO), keyword research, on-page SEO and local SEO, with every claim linked to its source so readers can check it.
What robots.txt can and cannot do for SEO, the syntax Google follows, what to block and never block, how errors are handled, and ready-to-copy files for a clini...
How to find the searches nearby customers make, from "near me" and city terms to Hindi and Hinglish queries, and decide which deserve a page, a profile field or...
How to optimise product pages for Google Search, Shopping and AI shopping answers: titles, descriptions, merchant listing markup, shipping, returns, feeds and a...
A beginner's method for server logs: get the file, find Googlebot and AI crawlers, verify them, and turn what they fetch into fixes, with tested commands.
What Google has actually said about topical authority, plus a cluster worksheet, a "worth writing" test and linking rules that avoid thin pages and scaled conte...
Which topics deserve a video, and how to write titles, chapters and transcripts, embed videos on your site and measure citations in AI Overviews and ChatGPT.
Mentions now matter more than links for AI answers. Find the sources ChatGPT and AI Overviews cite in your niche, earn a place on them honestly and skip the tac...
Told to "improve E-E-A-T" after a ranking drop? What Google actually says, plus a page-by-page audit to prove real experience, expertise and trust on a small bu...
A step-by-step entity SEO guide: define your brand on one page, mark it up with sameAs, make every listing match, and correct what Google and AI assistants get...
Which business listings still matter in 2026, from Google, Apple Business and Bing Places to Justdial and IndiaMART, how to fix wrong details, and when to stop...
How to decide which towns and branches deserve a page, what to put on each one, and how to stay clear of Google's doorway and scaled content policies.
A free, repeatable method to collect the questions customers ask ChatGPT, AI Mode and Perplexity, sort them by intent and stage, and decide what to build first.
A simple internal linking system for 20–500 page sites: Google's link rules, a hub-and-service page model, a free orphan-page audit, a spreadsheet template and...
A Search Console method to tell real keyword cannibalisation from harmless overlap, with thresholds you set and a merge, differentiate, canonical or leave decis...
A free-first workflow to see who outranks you on Google, who AI answers cite instead of you, and which gaps are worth closing.
A before, during and after checklist for moving domain, platform or URLs without losing rankings, with a redirect map template, Change of Address rules and a 30...
How Google's AI Mode splits one question into many searches, what Google says you can ignore, and a free method to map sub-questions and decide between one page...
What Google will and won't remove, how to report and appeal a fake review, what to do about review bombing and extortion, and how US, UK and Indian review laws...
Most Google searches now end without a click. Score keywords by click potential, intent and visibility value, then decide what to build for each one.
A plain-English checklist for AI agents that book, buy and fill in forms: what is live in October 2026, what is still experimental, and what to fix first.
Why ChatGPT, Claude and Perplexity may miss JavaScript content that Google indexes, how to see what each bot gets in five minutes, and which rendering fix to pi...
What Google's 2026 spam policies and AI-content guidance actually say, with a human-in-the-loop workflow for drafting with ChatGPT, Claude or Gemini safely.
Set up GA4 to show visits and leads from ChatGPT, Gemini, Perplexity, Copilot and Claude with a tested regex, and learn which AI clicks GA4 cannot see.
A free-data method to find pages losing clicks or AI citations, decide whether to refresh, merge, redirect or remove each one, and make updates that count.
What every Search Console Page indexing status means, which ones to fix first, how to fix them and how long to wait, and why indexing now decides AI visibility.
A free, statistically honest way to measure whether ChatGPT, AI Mode and Copilot mention you: prompt sets, repeat runs, mention rates and the free data Google a...
Which schema types still earn Google features after FAQ rich results ended in May 2026, what the evidence says about schema and AI answers, and a keep, fix or r...
What Search Console's generative AI report counts, what it hides, how to estimate the clicks AI answers cost you, and when the AI opt-out control makes sense.
AI Mode, Ask Maps and ChatGPT name far fewer local businesses than the map pack. What 2026 data shows they rely on, and a 30-day plan to get your business named...
Which AI bots to allow or block, bot by bot: training vs search vs user-triggered crawlers, three ready-made robots.txt policies and the Cloudflare setting that...
Lost Google traffic in 2026? A step-by-step way to prove whether a core or spam update caused it, measure it the way Google recommends, and fix the right pages.
How to set up, optimise and maintain a Google Business Profile within Google's rules, mapped to the relevance, distance and prominence factors that drive local...
A practical guide to generative engine optimisation: how ChatGPT, Perplexity, Copilot and Google AI Overviews choose sources, and what you can do to be one of t...
How to do keyword research that leads to pages people actually want: seed terms, free sources, search intent, metrics, clustering and a scoring model for priori...
A step-by-step on-page SEO checklist built on Google’s own documentation: titles, meta descriptions, headings, content, links, images, schema and freshness.
A practical technical SEO audit in the order Google processes your site: crawl, index, architecture, Core Web Vitals, rendering, then a scoring model to priorit...
YouTube's thumbnail sizes and file limits, how small thumbnails really appear, how to make text readable on a phone, and how to keep thumbnails within YouTube's...
The two formulas for adding GST to a price and taking it out, worked examples at 5%, 18% and 40%, CGST and SGST or IGST, rounding on bills, MRP and business quo...
The main image rules Amazon.in, Flipkart, Meesho and Google Merchant Center publish, the photo faults that break them, and a shoot checklist for a small seller.
See what Google says decides the order on Maps, which parts you can change, what never helps, and how to compare your profile with a competitor fairly.
How broad, phrase and exact match work in Google Ads today, why close variants apply to all three, when broad match needs Smart Bidding and where a small accoun...
Check who holds Admin access, billing, GST invoices and tracking in your Google Ads account, and move them to your business in the right order.
What the DPDP Act and the DPDP Rules, 2025 mean for a small business website: notice, consent, cookies, security, breaches, children's data and what to do this...
The YouTube Partner Program thresholds in India, the higher bar from 1 February 2027, what else YouTube checks, and how to work out your channel's gap.
The 13 dark patterns in India's 2023 guidelines, what each looks like in a small online shop, the honest alternative, and how to audit your own checkout.
The three post types Google offers, the rules a post must pass, a four-week rota with Diwali 2026 dates for a sample shop, and what to measure each month.
How to sign in to Stanford email at webmail.stanford.edu with your SUNet ID and two-step, what the new Microsoft login changes, and where to get help.
Sign in to PTD Webmail at promail.ptd.net with the part of your address before @ptd.net. Settings for email apps, limits, password help and support numbers.
The DigitalSnot helper
We use essential cookies to run this site. With your permission we also measure how the site is used (analytics) and use marketing cookies for ads. Privacy policy