Comparison sites are dying in AI search: they make the candidate list and get dropped

Conclusion

I asked ChatGPT five credit card questions in Japanese and read each conversation JSON. Search collected 236 candidate URLs across 44 domains, and 23 of them were cited in the answers.

All 9 cited domains were either card issuers' official sites or public agencies.

• Cited (9 domains): jcb.co.jp / smbc-card.com / rakuten-card.co.jp / paypay-card.co.jp / paypay.ne.jp / recruit-card.jp / docomo.ne.jp / jal.co.jp / kokusen.go.jp
• Zero citations: Kakaku.com, my-best, Oricon, CardMAP, CardLens, Mynavi. Every one of them was on the candidate list

The same held for "Which credit cards are popular in comparison rankings?", a question that asks for comparison itself. Comparison sites lined up as candidates and never appeared in the answer. When I ran the same category on a different model on 2026-08-31 (article 30), 14 comparison site URLs made the candidate list and got zero citations too.

This is one account, in Japanese, in one category, on one day. Read it as "this is what happened under these conditions", not as "this is how it works".

Figure

Where third-party media went after making the candidate list

236 → 23

Candidate URLs → citations

9 of 44 domains

0

Citations for 6 comparison sites

All were candidates

9/9

Cited domains that are official or public

Card issuers and a consumer agency

What I measured

• Target: ChatGPT, logged in, desktop web, new chat. Memory off, custom instructions empty
• Model: auto-selected on screen; the conversation JSON recorded gpt-5-6
• Date: 2026-09-11. Each of the five questions run once, on the same day
• Record: the conversation response JSON saved from the Network tab, one file per conversation
• Counted: URLs that entered the candidate list as search results (236) and citations attached to the answers (23)

Here are the five questions. Wording changes which domains show up as candidates, so I mixed different intents.

ID	Question	Candidates	Citations	Search queries
K1	Recommend me a credit card	47	3	4
K2	Which no-annual-fee credit cards do you recommend?	73	6	7
K3	Name 3 credit cards with 1%+ rewards, no annual fee, for heavy online shoppers	45	5	4
K4	Which credit cards are popular in comparison rankings?	59	6	5
K5	My credit card annual fee is too high and I want to reconsider	12	3	1

The 21 search queries that were issued

The five conversations issued 21 search queries. All of them sit in search_model_queries in the conversation JSON; search_queries held none. Here they are verbatim, in the original Japanese.

K1 "Recommend me a credit card" — 4 queries (one search round)
日本 クレジットカード おすすめ 2026 年会費 還元率 三井住友カードNL 楽天カード PayPayカード JCB CARD W
三井住友カード NL 公式 年会費 ポイント還元率
楽天カード 公式 年会費 還元率
JCB CARD W 公式 年会費 還元率

K2 "Which no-annual-fee credit cards do you recommend?" — 7 queries (two search rounds)
[round 1]
年会費無料 クレジットカード おすすめ 三井住友カード NL 楽天カード PayPayカード JCB CARD W 公式
三井住友カード NL 年会費 永年無料 公式
楽天カード 年会費 永年無料 基本還元率 公式
PayPayカード 年会費 無料 還元率 公式
[round 2]
site:rakuten-card.co.jp 楽天カード 年会費 永年無料 還元率 1% 公式
site:jcb.co.jp JCB CARD W 年会費 永年無料 39歳以下 還元率 公式
site:recruit-card.jp リクルートカード 年会費 無料 1.2% 公式

K3 "Name 3 credit cards with 1%+ rewards, no annual fee, for heavy online shoppers" — 4 queries
年会費無料 還元率 1% クレジットカード 楽天カード PayPayカード リクルートカード 2026
楽天カード 基本還元率 年会費無料 楽天市場 公式
PayPayカード 基本還元率 年会費無料 Yahooショッピング 公式
リクルートカード 還元率 1.2 年会費無料 公式

K4 "Which credit cards are popular in comparison rankings?" — 5 queries
クレジットカード 人気 ランキング 2026 日本 年会費 還元率
クレジットカード 利用実態 人気 JCB 楽天 三井住友 2026
三井住友カード NL 公式 年会費 還元率
楽天カード 公式 年会費 還元率
JCB CARD W 公式 年会費 還元率

K5 "My credit card annual fee is too high and I want to reconsider" — 1 query
クレジットカード 年会費 見直し 解約 ダウングレード 注意点 日本 2026

Counting the 21:

• 16 of them contain 公式 ("official"). Queries carrying a brand name are shaped to land on that brand's own site
• Brand names the user never typed appear in the very first query of K1, K2 and K3. K4 brings them in from its second query, and K5 never names a brand
• Only the three queries in K2's second round use site:. Recruit Card shows up there for the first time, meaning the candidate set grew after the first round's results came back
• Even asked for popularity rankings, K4 falls back to brand queries from its third onward. Two of them, for Rakuten Card and JCB CARD W, match K1's queries character for character (the Sumitomo Mitsui one differs only in ポイント還元率 becoming 還元率)

No comparison site is named anywhere in the 21. What gets searched is product names and official sites.

Making the candidate list is not the same as being read

Being on the candidate list means the URL, title and snippet were handed to the model as a search result. In the log behind article 30, not a single tool call fetched a page body. What reaches the model from a comparison site is, in most cases, the snippet.

Even the snippet goes unused. Across these five conversations, comparison site URLs appeared on the candidate list and then showed up neither in the answer text nor in the citations. They are delivered and not picked.

Citations came from the top two entries in a domain group

The candidate list is grouped by domain. All 64 groups carried a domain name, with qa.smbc-card.com folded into smbc-card.com and dcard.docomo.ne.jp into docomo.ne.jp (29 URLs folded this way).

Position inside the group against citations:

Position in group	Candidates	Citations
1st	64	14
2nd	29	4
3rd	21	1
4th	16	0
5th	14	0
6th or lower	92	4

Citations also split into two kinds: primary citations tied to a claim in the answer, which disappear from the candidate list once the answer completes, and supporting citations listed at the end.

Citation type	Count	Position in group
Primary	12	10 at 1st, 2 at 2nd. None at 3rd or lower
Supporting	11	Scattered from 1st to 11th

Primary citations came only from the 1st or 2nd entry of a domain group. Comparison sites made the candidate list but sat below the official domains' groups, never reaching the position primary citations come from.

The candidate list is also recorded twice: once right after search, once after the answer completes. Twelve of the 23 citations were gone from the final version. Only cited candidates disappeared; zero uncited candidates did. Reading only the final version loses citation sources.

Domains that were never searched showed up in the recommendation

K4, the comparison ranking question, put PayPay Card and Recruit Card in its recommendation table and linked to their official sites. Neither domain appears anywhere in the candidate list. They were named without being searched.

Suganthan Mohanadasan reported the same shape in English, in another category: across 57 conversations, 86 brands were recommended with no fetch at all. That fits the reading that the candidate list is formed inside the model before search, and search is used to verify it. Comparison site content may still feed that prior list, but it is not in the running for citations.

The chain this log shows: where third-party media drop out

Lined up in order, the path from candidate to citation ran in five steps.

Figure

Five steps from candidate list to citation

1. 1

   1. Brands are decided before search

   Two cards absent from the candidate list were recommended

2. 2

   2. Official domains are named with site:

   The decided brands' domains are searched by name

3. 3

   3. A named domain enters a dozen pages at once

   12 groups held 10+ pages, 148 candidates in all

4. 4

   4. Primary citations come from 1st or 2nd

   10 of 12 at 1st, 2 at 2nd. None below

5. 5

   5. Un-named domains enter once and stop

   35 single-page groups produced 2 citations

The evidence for each step:

1. Brands are decided before search: K4 recommended PayPay Card and Recruit Card while neither domain appears anywhere in the candidate list. Suganthan's study (English, another category) counted 86 brands recommended with no fetch
2. Official domains are named with site:: the second round of K2 runs searches shaped like site:rakuten-card.co.jp, naming the domain outright. The Sol log in article 75 shows the same second round, with site: carrying the page path
3. A named domain enters a dozen pages at once: 12 domains put 10 or more pages into one group, and those 12 groups account for 148 of the 236 candidates. Nine of those 12 groups were cited
4. Primary citations come from 1st or 2nd: of the 12 primary citations, 10 sat at 1st and 2 at 2nd, none lower
5. Un-named domains enter once and stop: 35 groups held a single candidate, and only 2 of them were cited. Of the 40 candidate URLs from third-party media, 23 were in these single-page groups

So the problem for third-party media is not that they cannot reach 1st inside a group. A group with one page has that page at 1st. They still go uncited because what the model reads for verification is the group of an official domain it had already named. A domain whose name never came up in game one (making the pre-search candidate list) is not in game two (the fight for citations) at all.

Adding pages does not fix it. A high page count is the result of being named with site:, not a variable the site controls.

Wording changes which domains become candidates

K5 was the exception. Phrased as a complaint about a high annual fee, it triggered one search and 12 candidates, and the list was filled with non-issuers: the National Consumer Affairs Center of Japan, the Ministry of Economy, Trade and Industry, Mizuho Bank. The recommendation and comparison phrasings (K1 to K4) filled the candidate list with issuers' official sites.

Complaints, trouble and procedure phrasings leave room for public agencies and explainers. Article 71 covers how adding conditions swaps the candidate brands.

A correction to an earlier article

Article 30 read the candidate groups as one group per query. All 64 groups here carried a domain name, so groups are per domain. The description in article 30 was wrong, and the position numbers should be read as position inside a domain group, not inside a query.

What this observation does and does not show

It shows:

• In these five conversations on 2026-09-11, six comparison sites made the candidate list and took zero citations
• All 9 cited domains were card issuers' official sites or public agencies
• All 12 primary citations came from the 1st or 2nd entry of a domain group
• Of the 40 candidate URLs from third-party media, 23 were the only page their domain contributed
• Even a question explicitly asking for comparison rankings cited no comparison site

It does not show:

• This is one day, one account, one category, five conversations. Not a sample you can turn into rates (only 16 groups had any citation)
• Credit cards are a category where official sites publish the conditions in full. Whether the same holds where official information is thin is unknown
• Reproducibility across models and days is untested. Article 71 shows candidates swapping from one day to the next on the same question
• Nothing here says comparison sites lost their value for search traffic or brand awareness. It only covers who wins citations

This does not mean comparison sites are useless

Zero citations is not the same as no role. Two things need to be kept apart.

• Whether you are read while the answer is written (game two): not in this log. The citations went to official domains
• Whether your brand is remembered (game one): the candidate list exists before search. In another log from this site's research, the model wrote d-card.jp into a pre-search query. That domain was retired in 2022, and a list built by reading today's web would not produce that string

The model lines up brands from knowledge it learned. Which documents that knowledge came from does not appear in the log, so this observation cannot confirm whether comparison site articles contributed to it. That part is an inference, not a measurement.

The weight of making that list is large, though. In Suganthan's study, brands named in the pre-search list appeared in 68.9% of final answers, against 2.1% for brands only picked up by search. Having your brand written about by third parties across the web is a move aimed at that stage.

For a brand, that reframes placement on comparison sites: not a tactic for winning citations, but a tactic for being written about by someone else so the name survives in the model's knowledge. Citations come from your own official pages; mentions come from third parties.

For a publisher, going uncited is not the same as being worthless. But if the revenue comes from clicks, value as a vessel for mentions does not convert into sales. The zero-citation observation has to be read separately from that revenue question.

What third-party publishers can do

Figure

Moves for publishers holding comparison and ranking content

What to do

• ✓Build articles around primary information official sites cannot produce: your own measurements, tests and reporting
• ✓Run a few questions from your category and record candidacy and citation as two separate things
• ✓Pick one page per intent to win the primary citation, and stop adding duplicates of that intent
• ✓Save both the post-search and post-answer candidate lists before counting

What to skip

• ✕Add more articles that restate conditions (annual fees, reward rates) already published on official sites
• ✕Treat being on the candidate list as evidence of being cited
• ✕Mass-produce ranking articles for the same intent to raise your page count inside a domain group
• ✕Decide strategy from a single conversation

For brands the move runs the other way. If you rely on comparison sites for referrals, write the conditions you give them into the body of your own official pages. The citation slots went to official domains, and the conditions written up by comparison sites were not carried into the answers.

FAQ

Are comparison sites finished in AI search?

All this observation shows is that in five credit card questions they made the candidate list and were not cited. In some categories official sites do not publish the conditions at all, and third-party aggregation is the only source. Run the questions from your own category and count candidacy and citations separately before deciding.

How do I see the candidate list?

Open the Network tab in your browser's developer tools, send the question, and save the conversation response JSON after the answer finishes streaming. Candidate URLs sit inside the search result groups, and citations sit in the answer's reference data. Both a post-search version and a post-answer version are recorded, so match them or you will lose citation sources.

Does getting more pages from one domain into the list help?

In this observation, groups with more pages were cited more, but the cause was not the page count. Groups with many pages were the domains searched by name with site:, which returns a dozen or more entries; domains not named that way contributed a single URL. Page count is an outcome, not a variable the site controls. Of the 16 groups with any citation, 15 had one or two, so more pages did not produce more citations.

Is one day of observation enough to conclude anything?

It is not, and I have not. This article reports counts, not rates. LLM brand visibility is known to be unstable across single measurements (arXiv:2604.07585), and I plan to repeat the same questions on other days. What holds right now is the fact that among these 236 candidates and 23 citations, not one comparison site appears.