Checklist Teknis Website untuk OpenAI Search dan Agent di 2026
Checklist teknis yang buruk dimulai dengan 50 checkbox.
Checklist teknis yang bagus dimulai dengan satu pertanyaan:
“Website ini ingin OpenAI melakukan apa?”
Mencari content?
Mengutip source?
Mengirim referral?
Memvalidasi ads?
Membantu user mengisi form?
Membantu shopping?
Masing-masing punya requirement berbeda.
Pada 2026, OpenAI ecosystem punya beberapa jalur ke web:
OAI-SearchBot.
GPTBot.
OAI-AdsBot.
ChatGPT-User.
ChatGPT Atlas agent.
Product feeds.
Karena itu checklist tidak boleh dimulai dari “allow AI bots.”
Ia harus dimulai dari business intent.
Berikut practical audit yang bisa dipakai brand, publisher, ecommerce, dan B2B website.
1. Tentukan crawler policy
Dokumentasikan:
OAI-SearchBot:
allow atau block?
GPTBot:
allow atau block?
OAI-AdsBot:
needed?
ChatGPT-User:
normal user-triggered access.
Jangan wildcard tanpa reason.
OpenAI current docs menyatakan OAI-SearchBot untuk Search dan GPTBot untuk potential training dengan settings independent.
If Search visibility desired but training opt-out:
allow SearchBot, block GPTBot.
If Ads active:
allow AdsBot on public landing pages.
Policy before config.
2. Pastikan robots.txt benar-benar bisa diakses
Fetch production:
`/robots.txt`
Check:
200.
plain text.
no challenge.
no login.
correct host.
No stale cache.
OpenAI says Search systems can take roughly 24 hours to adjust after robots change.
Record deployment.
Don't edit repeatedly.
3. Verifikasi published crawler IP ranges
OpenAI publishes crawler ranges.
Security team should use official source.
Don't copy random list.
Don't trust UA only.
If provider verified bot exists, use.
For AdsBot, OpenAI says Cloudflare recognizes it as verified.
Still custom rules may block.
Verification strategy:
UA + official range/provider verified signal.
4. Test actual HTTP access
For OAI-SearchBot:
homepage.
articles.
service.
research.
For AdsBot:
campaign landing pages.
Check final:
200.
real content.
No “Just a moment.”
No CAPTCHA.
No login.
No 403.
No 429.
Robots allow isn't enough.
HTTP proves.
5. Audit WAF and CDN
Cloudflare.
Akamai.
Sucuri.
hosting firewall.
Rules.
Bot protection.
Geo block.
Rate limit.
JavaScript challenge.
The crawler may be blocked before origin.
Check edge logs.
If intended crawler blocked, create narrow exception.
Do not disable security globally.
Least privilege.
6. Check redirect chains
Old URL -> new URL.
Host.
HTTPS.
Slash.
Language.
Campaign.
Minimize.
Final destination stable.
Ads and Search crawlers should not traverse unnecessary chains.
Track UTM without creating redirects that strip attribution before analytics.
7. Keep URL identity clean
One resource, one clear primary URL where possible.
HTTPS.
single host.
consistent slash.
Internal links.
sitemap.
canonical.
Although OpenAI webmaster docs do not expose a detailed canonical-selection algorithm, clean URL architecture reduces ambiguity and helps general web hygiene.
Don't call canonical a secret ChatGPT factor.
8. Audit noindex
Public Search pages should not accidentally noindex.
Pages intentionally excluded:
use correct strategy.
OpenAI notes noindex can prevent certain link/title surfacing, but crawler must access page to read meta.
Robots and noindex are different.
Don't stack blindly.
9. Make critical content available as text
Price.
service.
location.
feature.
policy.
product spec.
Don't hide only in image.
Agent can use visual context, but text and semantics are more robust.
Accessibility.
Search.
Agent.
All benefit.
10. Use semantic HTML
Buttons as buttons.
Links as links.
Labels.
Inputs.
Headings.
Landmarks.
OpenAI says Atlas uses ARIA labels, roles, and states to understand interactive elements.
Native HTML first.
ARIA to supplement.
Bad ARIA can mislead.
11. Label forms clearly
Audit:
contact.
demo.
checkout.
booking.
signup.
Field labels specific.
Required clear.
Format clear.
Errors clear.
Submit action specific.
“Nomor” is bad.
“Nomor telepon” better.
“Lanjut” weak.
“Konfirmasi booking” better.
This is agent readiness and accessibility.
12. Expose interactive state
Expanded.
selected.
disabled.
invalid.
loading.
success.
pending.
Use correct ARIA/state patterns.
Synchronize.
Visual and accessibility tree must agree.
Agent should not guess whether action happened.
13. Test keyboard flow
Tab.
Shift+Tab.
Enter.
Space.
Escape.
Arrow behavior where applicable.
If keyboard cannot operate, agent/accessibility may struggle.
Custom widgets especially.
Date picker.
dropdown.
modal.
tabs.
Use WAI-ARIA authoring practices.
14. Design clear success state
After submit:
what happened?
Reference number.
Date.
Price.
Next step.
Agent needs to report back.
Human needs confidence.
Don't end with empty modal.
15. Design specific errors
No generic:
“Something went wrong.”
Say:
slot unavailable.
email invalid.
payment declined.
session expired.
Preserve input.
Offer recovery.
Agent should not loop.
16. Protect high-impact actions
Purchase.
delete.
cancel.
financial.
account change.
Need:
auth.
step-up if necessary.
review.
confirmation.
audit.
Don't reduce security for agents.
OpenAI Atlas includes safety controls, but website remains final system of record.
Authorization is your responsibility.
17. Make actions idempotent where needed
Double submit can happen.
Human double click.
Agent retry.
Payment and booking should not duplicate.
Backend needs transaction safety.
Unique request IDs or server-side protections depending architecture.
This isn't AI-specific.
Agent speed makes it more important.
18. Audit CAPTCHA placement
CAPTCHA before public page:
bad for crawl.
CAPTCHA on high-risk action:
maybe appropriate.
Don't remove blindly.
Accessible alternative.
Human takeover.
Separate read access from write protection.
19. Validate authentication flow
Login should be clear.
MFA.
Session expiration.
Return path.
Agent may use logged-in sites in Atlas.
If auth expires mid-task, give recoverable state.
Don't lose cart/form.
20. Test human-agent handoff
Agent fills.
User takes over.
Changes option.
Agent resumes.
Does state persist?
This is new UX pattern.
Run test.
Don't assume.
21. Check content freshness
Technical readiness with wrong facts is failure.
Review:
leadership.
address.
price.
product.
availability.
policy.
platform capability.
High-visibility pages.
Current.
Timestamp where relevant.
Correction system.
Search can retrieve old page.
Knowledge governance.
22. Clean entity consistency
Brand name.
legal entity.
product names.
founder/current leadership.
locations.
Parent/subsidiary.
Official source.
Third-party profiles.
AI Search representation depends public information ecosystem.
Don't create fake entity links.
Fix actual sources.
23. Allow OAI-SearchBot if source visibility desired
OpenAI states public sites can appear in ChatGPT Search and recommends not blocking OAI-SearchBot if you want content found, summarized, cited, and linked.
Then verify infrastructure.
No guarantee top placement.
OpenAI also says there is no guaranteed top ranking.
Search inclusion is eligibility, not promise.
24. Keep GPTBot decision separate
Training policy.
Do not allow because someone says Search requires it.
OpenAI explicitly separates.
Legal decides.
Marketing doesn't need training permission for SearchBot visibility.
25. Prepare analytics for `utm_source=chatgpt.com`
OpenAI automatically attaches parameter to referral URLs.
Make sure:
redirect doesn't strip before capture.
GA/source parsing works.
canonical remains clean.
Create dashboard row.
Identifiable ChatGPT referral.
Not total AI impact.
26. Add self-reported attribution
Form:
How did you hear about us?
ChatGPT/AI assistant.
This catches no-click discovery.
Especially B2B.
CRM.
Sales notes.
Don't rely only UTM.
27. Build controlled Search observation panel
30-50 queries.
Core.
Repeat.
Track:
brand mention.
owned link.
third-party source.
accuracy.
recommendation.
Surface/date.
This is observational.
Not first-party complete.
Label.
28. Separate Search from Ads reporting
ChatGPT Ads are sponsored and separate from organic answers.
Do not mix.
Paid:
impressions.
clicks.
spend.
conversions.
Organic:
source presence.
referrals.
mentions.
Accuracy.
Separate.
29. If using Ads, allow OAI-AdsBot
OpenAI says required for landing validation.
Test WAF/CDN.
200.
actual content.
Offer matches ad.
Do not rely support bypass.
Fix root.
30. For ecommerce, audit product feed
Stable IDs.
title.
description.
price.
currency.
availability.
brand.
image.
URL.
variant.
identifiers.
OpenAI commerce feed has separate search/checkout/ads eligibility fields/configurations.
Feed accepted != product guaranteed visible.
Keep distinction.
31. Sync feed and storefront
Feed price matches page.
Stock.
Variant.
URL.
Image.
If product data updates faster than cache, fix.
AI shopping depends temporal truth.
32. Check Ads product eligibility intentionally
`is_ads_eligible`.
Don't default all.
Legal/commercial policy.
Market availability.
Inventory.
Margin.
Ads program/account availability by country.
For Indonesia, current official advertiser availability must be verified at launch.
Don't rely old article.
33. Monitor server logs by bot type
Timestamp.
UA.
IP.
URL.
status.
WAF.
response time.
Bot verification.
Expected policy.
SearchBot 403 = issue.
GPTBot 403 might be intentional.
AdsBot 403 = campaign issue.
Context.
34. Monitor 429
Rate limit.
Bulk crawl.
Ad review.
Search crawl.
If legitimate bot throttled, adjust narrowly.
Do not remove protections everywhere.
35. Store robots and WAF change history
Who changed?
When?
Why?
Hash/config version.
Then incident analysis possible.
Without history:
guessing.
36. Test stale cache
Robots.
landing page.
price.
product.
CDN may serve old version.
Purge mechanism.
Cache TTL.
Commerce especially.
37. Secure private data properly
Robots is not security.
Noindex is not security.
Private:
authentication.
authorization.
network control.
Do not expose confidential page and hope crawler obeys.
Agentic era does not change basic security.
38. Review user-generated content
Comments.
reviews.
forums.
Agent can encounter malicious prompt injection content.
OpenAI actively works on Atlas prompt-injection defenses.
Website should still moderate, sanitize, and minimize execution risk.
Don't embed hidden instructions for agents.
39. Use least privilege
If agent acts in account:
user permission.
Not admin by default.
Sensitive action confirm.
Backend authorization.
The agent only has what user/session has.
This is safest architecture.
40. Keep official documentation watch
OpenAI products move fast.
Crawler UA.
IP.
Ads availability.
Atlas.
Commerce.
Check monthly or quarterly.
Use official OpenAI sources.
Version SOP.
No fossilized 2024 rules.
41. Run five real agent tasks
Don't stop at validators.
Examples:
book demo.
find product and variant.
submit quote.
change setting safely.
find cancellation policy.
Observe.
Does it succeed?
Does it fail safely?
Task-level QA beats markup score.
42. Run five Search tests
Branded fact.
non-branded category.
comparison.
commercial.
current fact.
Check:
source.
accuracy.
link.
Referral potential.
Again, no guaranteed result.
But baseline.
43. Create incident severity
P0:
private data exposure, unsafe action.
P1:
wrong price, wrong identity, major crawl block.
P2:
source drift, single broken page.
P3:
minor markup.
Not every AI variation emergency.
Operational calm.
44. Assign owners
SearchBot:
SEO/Infra.
GPTBot:
Legal/Infra.
AdsBot:
Ads/Infra.
ARIA/forms:
Product/Accessibility.
Feeds:
Commerce/Data.
Brand accuracy:
Comms/Content.
No owner = no maintenance.
45. Record limitations
At end audit:
What we know.
What we observed.
What we cannot verify.
Don't present speculation as OpenAI requirement.
This is core GEO discipline.
A checklist should prevent overclaim as much as technical failure.
The final PASS is not “all boxes green”
Website might intentionally block GPTBot.
That's PASS if policy.
AdsBot not needed because no ads.
PASS.
Checkout agent blocked by human confirmation.
Could be PASS.
Readiness is objective-specific.
So final report should say:
Search readiness: PASS.
Training policy: OPT-OUT.
Ads readiness: NOT APPLICABLE.
Agent UX: PARTIAL.
Commerce feed: NOT USED.
Brand accuracy: PASS.
This is more meaningful than 92/100.
OpenAI Search and Agent readiness in 2026 is not one optimization project.
It is coordinated web operations.
Crawlability.
Accessibility.
Security.
Content.
Data.
Ads.
Commerce.
Analytics.
Governance.
The most mature website won't be the one that allows the most bots.
It will be the one that knows which capabilities it wants, exposes the right public information, protects the right private information, and can prove the system works from request to action.
That's what the checklist is for.
Not to make the website look AI-ready.
To make it actually ready.
Bacaan terkait di GEO.or.id
- Cluster: ChatGPT & OpenAI Search
- Kerangka terkait: ChatGPT Analysis
- Kerangka terkait: AI Crawler Access and Referral Measurement Protocol
- Artikel terkait: Cara Audit Crawlability untuk OAI-SearchBot tanpa Membuka Semua Bot
- Artikel terkait: Dari SearchBot ke AdsBot: OpenAI Mulai Memisahkan Fungsi Crawler Lebih Jelas
- Artikel terkait: ChatGPT Atlas sebagai Agent: Apa yang Berubah untuk UX Website
- Artikel terkait: Kenapa Landing Page Iklan ChatGPT Harus Bisa Dicrawl OAI-AdsBot
