How to Audit AI Engine Recommendations for Your Law Firm

Ask ChatGPT for a personal injury lawyer in Phoenix and you get names. The interesting question for anyone running a firm is where those names came from.

We ran a city-anchored collection across four practice areas and three sizes of market to find out what these answers are built from.

The short version: you are competing inside directory profiles you do not control, and the smaller your market, the more completely that is true.

One boundary before any number. This measures which sources appear alongside the answers. It does not measure whether a directory profile causes a firm to be named, and nothing here was designed to test that.

The Directory Layer Is Most Of The Answer

Across a city-anchored set of legal queries spanning four practice areas and three metro sizes, run on ChatGPT and Google AI Mode, August 2026, at least one named legal directory appears in 71.57% of cited answers.

Roughly seven in ten. Here is how the individual sources rank.

Super Lawyers at 42.15%, Justia at 38.17%, Avvo at 37.38%, Expertise at 31.81%. Reddit is the only source in the top six that is not a directory, at 28.63%.

No individual firm's own domain appears in that leading group.

In this category, the sources answering the question are the ones that list you, not the one you built.

Worth saying how we identified those directories, because we got burned on classification elsewhere in this research. We did not run a classifier. We read the source names individually and kept only unambiguous, well-known legal directories. That narrow, hand-checked approach is the only kind of categorisation that has survived independent checking in this project.

Reputation Questions Invert The Whole Picture

The collection carried three kinds of question, and treating them as one category would have hidden the most useful thing in it.

Ask the engines to find you a lawyer and a named directory appears in 81.40% of cited answers, with Reddit in 26.36%.

Ask whether a specific firm is any good and it flips completely. Reddit appears in 60.00% of those answers and a directory in 22.00%.

Same category, same collection, same engines. Two questions, two almost disjoint source layers.

Directories carry 81.40% of the discovery answers and 22.00% of the reputation ones. Reddit runs the other way, 26.36% against 60.00%.

That is a more actionable split than anything else in this collection, because most firms treat visibility as one problem with one budget. It is two problems. A firm with immaculate directory profiles and an ugly Reddit thread is highly visible in discovery answers and undermined in the reputation answers that follow.

And reputation questions are the ones asked later, by someone who already has your name and is checking it. That is the closest thing to a bottom-of-funnel moment this category has, and it runs on a surface no firm controls and most do not monitor.

The middle family, questions about how to choose a lawyer at all, sits between the two: directories in 51.52% of cited answers, Reddit in 18.18%. So the directory layer weakens steadily as the question moves from finding to evaluating.

It Gets Worse In Smaller Markets, Not Better

The result we did not expect sits in the market-size cut.

Small metros: 89.26% of cited answers contain a directory. Major metros: 72.13%. A 17.13-point spread, and it runs opposite to the intuition that a crowded market would be the one dominated by aggregators.

Our reading, which the collection cannot test: in a smaller market there is simply less independent coverage of local firms, so the directory profile is most of what exists to cite. Treat that as a hypothesis, not a result.

What it means operationally is concrete. A firm in a secondary city is competing almost entirely inside profiles on sites it does not own. A firm in a major metro has roughly a quarter of answers reaching past that layer to something else.

The uncomfortable implication for smaller firms is that the cheapest visibility lever available to them, their own website, is also the one with the least presence in the answers their prospective clients see. Publishing more of it does not appear to change which sources these answers draw on, though our collection cannot test whether sustained effort would.

Practice area moves the measure as well, with family and immigration questions leaning on directories more than business and personal injury ones. We are putting no percentages on that split. Three of its four arms fall below the minimum size we set for publishing a share before collection started, so it stays a direction and nothing more.

The Two Engines Are Reading Different Sources

Directory presence itself is similar across engines, and we are not going to dress up the small gap we measured as a finding: it did not clear the threshold we set for calling an engine difference real.

What does show up is at the level of individual sources. Martindale and Lawyers.com appear on the ChatGPT side and effectively not on Google AI Mode's. Google's own properties appear on the Google AI Mode side and not on ChatGPT's at all.

So the two engines are assembling legal answers from partly different source sets, even though the directory layer as a category is present in both.

The practical consequence: a pooled source report averages two different pictures together and gives you a number that describes neither. If you track one engine and act on it, you may be optimising for a source the other never touches.

On trigger behaviour, ChatGPT ran a search on 91.11% of its runs here and Google AI Mode on 99.26% of its own. An August 2026 observation about these query sets, not a fixed property of either engine, and no basis for preferring one.

What A Firm Can Do About A Layer It Does Not Own

Nothing above tells you that fixing a directory profile will get you named. It tells you where the answer is being assembled from, which is the prerequisite question.

Given that, five things are worth doing. They are the writer's judgment on the observed associations, not results, and the collection does not establish that any of them changes what gets cited.

• Audit your profiles on the named directories first. Super Lawyers, Justia, Avvo, Expertise, Lawyers.com, Martindale, Best Lawyers, FindLaw. These are the pages appearing in answers about your practice area and your city, whatever state they are in.

• Weight the effort by market size. In a smaller metro the directory layer covers close to nine in ten answers, so profile completeness carries more of your visibility there than it does in a major city.

• Split your query set by intent before anything else. Discovery questions and reputation questions returned almost disjoint source layers here. A single blended report describes neither, and the reputation half is the one most firms never measure.

• Track engines separately. Given that individual sources are close to engine-exclusive, one pooled report will hide which sources matter where.

• Log it over time. A single snapshot cannot distinguish a source that is structural in your market from one that appeared once. Only a repeated record separates them.

Qvery supplies the tracking layer for that. It tracks visibility, share of voice, and average rank across ChatGPT and Google AI Mode daily, in over 200 countries, and captures every citation tied to the query and engine that produced it. In Qvery Assistant you can ask about your own visibility, share of voice, citation, topic, and query data, add queries, trigger a run, or export a report.

The directory classification and the city and practice-area tagging stay yours to maintain alongside those records.

Start your free trial and audit your first month of legal citations by city and practice area.

Writer's Judgment, Not A Finding

Flagging this clearly, because the rest of the article stays inside what we measured.

Given that directories occupy roughly seven in ten of these answers and close to nine in ten in small markets, our judgment is that directory profile quality deserves a larger share of a firm's visibility budget than it usually gets. Content on the firm's own site looks unlikely to substitute for it in this category.

That is a view about where effort is likely to pay. The collection does not establish it.

A third judgment, from the query-family split: we would treat reputation visibility as a separate workstream with its own owner, because directory and Reddit presence differ sharply between the two pre-defined query families.

A second judgment: because the practice-area split fell below the size we needed to publish a number, the honest position is that we do not yet know how much practice area matters here. We would want a larger collection before acting on that dimension.

What the collection did establish is narrower and firmer. Named legal directories appear in 71.57% of cited answers, rising to 89.26% in small metros against 72.13% in major ones, and the leading individual sources are Super Lawyers at 42.15%, Justia at 38.17%, and Avvo at 37.38%. Those are the bounds. What you do inside them is a decision, not a finding.