Methodology · V1 · Relevé du 11 mai 2026

The standard for AI readiness for your product pages.

We analyze both what a standard AI agent can read on your page, and what ChatGPT & Gemini actually manage to understand from your URL. These analyses feed a transparent score out of 100, explainable criterion by criterion.

Fieldwork

Relevé du 11 mai 2026

Readings

2 parallel modes (arithmetic mean)

Sub-scores

3 dimensions · 25% · 45% · 30%

Engine

Rule-based + AI URL retrieval strict

AI providers

Gemini url_context · OpenAI web_search

Targets

ChatGPT · Gemini · Perplexity

Scoring

/100 weighted

§ I · Dual reading

Two scores in parallel, one global you can defend.

Each audited page produces two independent scores, averaged to give the global one. The agent-agnostic mode measures what an external non-privileged crawler can read from your page. The Major AI mode measures what Gemini and OpenAI actually observe when they access the URL. The gap between the two tells a concrete story: a universally readable page, dependence on private agreements, or a hole in AI agents despite correct HTML.

What a generic non-privileged agent sees

Agent agnostic

Our external crawler reads the HTML, the Playwright rendering, the JSON-LD and the Schema.org signals. This is the « raw » measurement, the one that applies to any agent that has no private agreement or allowlist with the brand.

→Direct fetch + Playwright browser rendering
→Parsing JSON-LD Schema.org Product / Offer / Review
→robots.txt, sitemap, llms.txt, GPTBot / OAI-SearchBot / Googlebot / Google-Extended bot probes
→Bright Data Web Unlocker cascade if a WAF intercepts

What Gemini & OpenAI observe through their own channels

Major AI

Strict URL retrieval from each provider, with no prior knowledge or third-party source. We measure what they actually read from the URL (retrieval status, citation, fact extraction).

→Gemini: gemini-2.5-pro + url_context tool (only, google_search disabled), rejected if url_retrieval_status ≠ SUCCESS
→OpenAI: gpt-5.5 + web_search tool (tool_choice required), strict citation match of the target pathname
→Strict prompt: forbidden to use its own knowledge, to infer from the URL, or to consult a third-party source
→Confidence status: verified_by_both, single_provider, conflict or not_observable

Reading the delta between the two modes

Both high : universally agent-ready page, readable by all generic agents as well as Gemini / OpenAI.
Major AI high, agent-agnostic low : the page probably depends on private agreements or allowlists (Gemini / OpenAI). Risky for unlisted agents.
Major AI low, agent-agnostic high : correct HTML markup but something blocks Gemini / OpenAI (timeout, paywall, JS-only rendering).
Both low : a real readiness problem - to be fixed as a priority.

§ II · Can agents reach the page and read it?

Reach & read

Technical reachability and readability of the product page. Includes our crawler signals (agent-agnostic mode) AND the confirmation that Gemini / OpenAI actually read the URL (Major AI mode).

Weighting

25 %

Of the global score (weighting: 25% · 45% · 30%).

Evaluated criteria

01
Fetch HTTP(S) and canonical
200 status, controlled redirects, canonical aligned with the audited URL. If our cloud IP is blocked by a WAF, automatic escalation to Bright Data (residential). Otherwise the criterion is marked « Not measurable ».
02
robots.txt, meta robots, X-Robots-Tag
No unintentional blocking of useful crawlers (GPTBot, OAI-SearchBot, Googlebot, Google-Extended). Reading the intent declared by the site.
03
Empirical bot probes
When Bright Data Custom Headers is enabled: we request the page with User-Agent = GPTBot, OAI-SearchBot, Googlebot, Google-Extended from a residential proxy. Penalizing criterion. Without the feature: non-penalizing diagnostic probes (cloud IP).
04
Sitemap, llms.txt and TTFB
XML sitemap exposed, llms.txt file present (low-weight bonus), thresholded TTFB (≤ 800 ms = green).
05
Raw HTML vs browser rendering
Diagnostic: name, price, availability, description, JSON-LD in the initial HTML vs after hydration. Undetermined when the WAF blocks us.
06
Chunkability
Informative subheadings, specs in an extractable structure (table / dl / schema), short paragraphs, id anchors - for RAG and extraction by agents.
07
Gemini retrieved the URL
url_context returns URL_RETRIEVAL_STATUS_SUCCESS for the target page.
08
OpenAI cited the target URL
The OpenAI response references the target pathname (strict citation match).
09
Effective AI extraction
At least one of the two providers ended up outputting product facts; multi-provider confidence evaluated separately.

§ III · Can agents extract facts?

Product Data Completeness

Observable product facts. On the agent-agnostic side: Schema.org Product / Offer parsed from our crawler. On the Major AI side: facts actually extracted by Gemini/OpenAI from the URL.

Weighting

45 %

Of the global score (weighting: 25% · 45% · 30%).

Evaluated criteria

01
Schema.org Product (agent-agnostic)
Valid JSON-LD with name, description, brand, image, sku, identifiers (GTIN / MPN), Offer (price, currency, availability, shippingDetails, hasMerchantReturnPolicy), AggregateRating, Review, BreadcrumbList.
02
HTML / JSON-LD consistency (agent-agnostic)
The name, price and availability from the JSON-LD must also appear in the visible HTML. Reinforced weighting.
03
Product entity (agent-agnostic)
GTIN with GS1 checksum, brand aligned page / schema, name consistency (title / H1 / og:title / Product.name), a single main Product, URL / slug consistency.
04
AI-observed facts (Major AI)
Gemini and OpenAI return a strict JSON: name, brand, price, currency, availability, description, imageUrls, identifiers, specs, rating, reviewsCount, useCases, audience, differentiators, decisionSupport, evidence, sourceUrls.
05
Multi-provider confidence (Major AI)
verified_by_both = 100, single_provider = 70, conflict = 45, not_observable = 0. Conflicts detected on name / brand / price / currency / availability.
06
Textual evidence
Short literal quotes (≤ 200 chars) extracted from the read page. Anti-hallucination safeguard.

§ IV · Can agents decide?

Semantic Decisioning

Decision-oriented editorial density. On the agent-agnostic side: rule-based extraction + semantic embeddings. On the Major AI side: what Gemini/OpenAI output as use cases, audience, differentiators and evidence.

Weighting

30 %

Of the global score (weighting: 25% · 45% · 30%).

Evaluated criteria

01
Decision-oriented description
Description rich enough for an agent to explain the product (material, use, benefits, constraints).
02
Use cases
When to use the product, in what context. Schema, section, embeddings or AI extraction.
03
Audience
Product.audience.audienceType or dedicated section. On the AI side: audience made explicit by the providers.
04
Differentiation
Rich Product.additionalProperty, « Why us » section, innovations, exclusives. On the AI side: differentiators cited without hallucination.
05
FAQ and decision support
FAQPage Schema.org, size guide, comparisons, observable practical constraints.
06
Specifications
Product.additionalProperty (≥ 5 entries) or <dl>. On the AI side: specs extracted as name / value pairs.
07
Reviews
Structured AggregateRating + Product.review, or a recognized third-party widget (Trustpilot, Yotpo, Loox…). On the AI side: rating and review volume actually observed.
08
Hn hierarchy and metadata
A single H1, consistent H2/H3 order, title (25–70 chars) + meta description (80–220 chars) aligned.

§ V · Anti-WAF cascade

An audit that isn’t blocked by a WAF.

On a site protected by Cloudflare, Datadome, Akamai or Imperva, our outbound cloud IP can be intercepted. Rather than unfairly penalizing the Reach & read score, we use an explicit cascade with provenance traced in the report.

1 · Direct
HTTPS fetch from our servers with a reference Chrome UA and an X-Dataiads-Audit header signed with HMAC. Most audits stop here.
2 · Bright Data
If a WAF challenge is detected → escalation to Web Unlocker (residential proxy + JS rendering). The affected criteria carry a via BD badge in the audit report.
3 · Undetermined
If BD also fails (very rare), the affected criteria drop out of the calculation (effective weight 0). The sub-score stays honest rather than being unfairly zeroed out.

All the fetches of an audit (HTML, robots.txt, sitemap, llms.txt, bot probes) carry a short HMAC token. E‑commerce players can authorize Dataiads via a WAF rule based on this header - without having to allowlist our cloud IP (risky on the security side).

Configure the audit bypass on the WAF side →

§ VI · Transparency

See what the engine actually read.

On every completed audit report, a « Extracted data » side panel first exposes what Gemini/OpenAI observed (facts, sources, evidence, confidence), then the JSON-LD, the raw HTML and the Playwright-rendered HTML. Goal: distinguish the data actually seen by the agents from our technical diagnostics.

§ VII · Principles

Four principles that hold up the methodology.

Parallel dual reading

Each audited page produces two independent scores: agent-agnostic (our crawler) and Major AI (Gemini + OpenAI URL retrieval). The global score is the arithmetic mean of the two. The gap is itself a business signal.

Deterministic and explainable

Binary or thresholded criteria, weighted and documented. On the AI side: strict JSON with textual evidence, provider retrieval status and strict pathname matching. Every point of the score is explained criterion by criterion.

No AI hallucination

Gemini and OpenAI are constrained by a strict prompt that forbids prior knowledge, inference from the URL and third-party sources. Citations are rejected if they do not match the target pathname. When a provider does not read the URL, the score stays honest rather than made up.

Adapted to the market

Sector benchmark computed per market (FR by default, BE, CH, LU, CA, ES, IT, DE, UK, US). Sector score specific to the retailer's market, not a worldwide average.