Short answer
I use AI, mostly Claude, to work faster on the parts of CRO that involve a lot of text and repetition: sorting reviews and survey answers into themes, drafting hypothesis lists and copy variations, writing test plans and readouts, checking my sample size math and prototyping pages in code. It doesn’t replace research with real users or real test data. It can make things up, so I check everything it produces, and I don’t give it private customer data.
Key takeaways
- AI is most useful for sorting large amounts of text, drafting, and turning notes into structured documents.
- It speeds up the work around a test. The test itself, with real visitors, is still what decides.
- AI can state wrong things confidently. Every quote, number and claim gets checked against the source.
- Remove or anonymize personal data before anything goes into an AI tool, and follow the client’s rules.
Where does AI fit in a CRO workflow?
Around the test, not in place of it. A CRO project has a lot of reading, sorting and writing: research notes, hypotheses, copy, test plans, readouts. That is where AI saves me time. The parts that decide whether a change works, real visitors and real test data, stay the same.
I mostly use Claude. The table below sums up how I split the work.
| Task | How AI helps | What I still do myself |
|---|---|---|
| Research synthesis | Groups reviews, survey answers and support tickets into themes, with quotes for each | Read a sample of the raw text, check the quotes exist, decide which themes matter |
| Hypothesis lists | Turns research themes into many candidate hypotheses in a set format | Remove weak ones, tie each to evidence, rank them by impact and effort |
| Copy variations | Drafts headline, subhead and button options from the research language | Pick, rewrite in the brand voice, check every claim is true and allowed |
| Test plans and readouts | Turns my notes into a clear plan or a readable summary | Set the metric, sample size and decision rules; check every number against the testing tool |
| Analysis help | Checks sample size math, suggests segments to look at, explains a statistic | Run the real numbers, treat any segment “win” as a new hypothesis |
| Prototyping | Builds a working page or section in code from my design | Design in Figma first, review the code, QA on real devices before any test |
How do I use AI for research synthesis?
I give it a batch of reviews, survey answers or support tickets and ask it to group them into themes, with direct quotes for each theme. Reading hundreds of reviews by hand is slow, and it is easy to remember the dramatic ones and forget the common ones. Sorting them first, then reading by theme, is faster and more even.
Two rules make this work. First, I ask for word-for-word quotes, and I search the source to confirm each one exists. Anthropic’s own guidance on reducing hallucinations recommends grounding answers in direct quotes and having the model retract claims it can’t support with a quote. Second, I still read a random sample of the raw text myself. Themes can sound tidy and still miss what customers really care about.
What AI can’t do here is talk to customers. It can only sort what customers already said. If the reviews don’t cover a question, I need a survey, an interview or a user test, not a better prompt.
How do I use AI for hypotheses and copy?
As a fast first draft that I then cut down hard. From a set of research themes, Claude can write a long list of hypotheses in the format I use: the change, the expected effect on a named metric, and the reason. Most of the list is ordinary. A few are useful, and sometimes one points to something I hadn’t considered. I keep the ones tied to real evidence and rank them. See how to prioritize A/B test ideas for how I rank.
For copy, I ask for headline and button variations that reuse the customers’ own words from the research. Then I rewrite. AI copy tends to sound generic, and it will happily write a claim the product can’t back up. Every promise on a page has to be true and allowed for that business, and that check is mine.
How do I use AI for test plans and readouts?
I write the decisions, and AI helps turn them into a clear document. A test plan needs the hypothesis, the primary metric, guardrails, sample size, duration and what happens if the test wins, loses or comes out flat. I decide all of that. Claude helps me write it so a stakeholder can read it in two minutes, and points out anything missing from my template.
Readouts work the same way. I give it the numbers from the testing tool and my notes, and it drafts the summary. Then I check every number in the draft against the tool, line by line. A readout with one wrong number is worse than no readout. For how I turn a result into money, see how to calculate the revenue impact of an A/B test.
How do I use AI for analysis?
As a second pair of eyes on the math and a source of questions, not as the analyst of record. I use it to:
- Check sample size math. I work out the sample size, then ask Claude to redo it step by step. If the two answers differ, I find out why. The real number comes from a proper calculator. See A/B test sample size.
- Suggest segments. Device, traffic source, new versus returning, landing page. Useful for planning. A segment that “wins” inside a flat test is a new hypothesis, not a result.
- Explain a statistic. When I need to explain a confidence interval or sample ratio mismatch to someone, it helps me find plain words.
I don’t ask it to decide whether a test won. That comes from the testing tool and the rules I set before launch.
How do I use AI to prototype pages?
I design in Figma, then use Claude to turn the design into a working page or section in code. A clickable, real page on a phone shows problems a static mockup hides: tap targets, load time, how a sticky button feels, what happens when the copy is longer. It also gets a variant closer to something a developer or a testing tool can use.
The code still needs a review. I check it on real devices and browsers, make sure tracking fires the same way as on the control, and keep it light so it doesn’t slow the page. A prototype that looks right but breaks on one browser is a bad test.
What are the limits of AI in CRO?
It doesn’t replace users, data or judgment, and it can be confidently wrong.
- It isn’t research with real users. It can summarize what people wrote. It can’t watch someone struggle with a form, and its guesses about “what users want” are guesses.
- It isn’t test data. No prompt can tell you whether a variant will win. Only a test with real visitors can.
- It makes things up. Quotes, numbers, statistics and sources can be invented. Anthropic’s guidance notes that its techniques reduce hallucinations but don’t remove them, and that critical information should always be checked. I check every quote, number and claim before it goes anywhere.
- Private data needs protecting. I remove names, emails, phone numbers and other personal details before any customer text goes into an AI tool, I follow each client’s rules on what can be shared, and when the rules aren’t clear, I don’t share it.
- It writes average copy by default. Without specific research and a strong edit, the output sounds like every other page.
Used this way, AI gives me more time for the parts that need a person: talking to customers, deciding what to test and reading results carefully. If you want help with research, design and testing, see A/B testing or book a call.
FAQ
Can AI replace a CRO designer?
No. AI can speed up research synthesis, drafting and prototyping, but it can’t run tests on real visitors, judge which research matters, or take responsibility for a claim on a page. It is a tool inside the process, not the process.
Can AI predict which A/B test variant will win?
Not reliably. It can suggest ideas based on common patterns, but whether a change wins depends on your visitors, offer and traffic. Only an A/B test with real visitors answers that.
Is it safe to put customer data into AI tools?
Only with care. Remove personal details such as names, emails and phone numbers first, follow your company’s or client’s data rules, and check the data settings of the tool and plan you use.
Which AI tool do you use for CRO?
Mostly Claude, for research synthesis, drafting, test documentation, math checks and prototyping in code. I design in Figma and read results in GA4 and the A/B testing tool.
Sources
I’m George (Djordje) Peric, a CRO designer, CXL certified in Conversion Optimization and Ecommerce Marketing. I design ads, landing pages and funnels and A/B test them.