Code for the rules, AI for judgment
Banned-phrase matching runs as exact and pattern matches, with no AI involved. Each calculation is recomputed by a script. The AI judges claims and writes rewrites, inside hard constraints.
Case file 04 of 07
AI fact-checking and SEC Marketing Rule review
Evergreen Wealth and Evergreen.ai, about nine months
Claims checked. Rules applied. Before compliance sees it.
I built the AI fact-checker and SEC Marketing Rule checker that review each draft first, fix it in plain sight and log the run, so the Chief Compliance Officer approves instead of rewriting.
19%
of claims fixed or sent to a human before review, about 1 in 5
130
rules in a rulebook the Chief Compliance Officer controls, nearly 4x its starting size
75%
less manual review, at most, for low-risk content once sampling replaced 100% review
450+ hrs
of manual review saved, from the first logged pieces alone
Claims extractedMath recomputed in codePrimary sources foundLinks testedSEC Rule 206(4)-1Banned phrases scannedApproved disclaimers placedAudit log in each documentApproved in Themis
01
The situation
One compliance officer, a content engine, and each word an advertisement.
- Company
- Evergreen Wealth, an SEC-registered investment adviser, and Evergreen.ai, its consumer AI financial-advice app
- Covered
- Blogs, social posts, app components, in-app videos, podcast clips, press releases and study assets
Under the SEC Marketing Rule, nearly any piece Evergreen published counted as an advertisement, and the Chief Compliance Officer read each piece by hand. I was building an AI content engine that would multiply the volume.
Faster drafting alone would only lengthen the queue. So I put the review inside production: a fact-checker and a compliance checker that run before a person sees anything, and hand the reviewer a documented draft.
The constraint
One reviewer, each word regulated, and each approval on the record in Themis, the compliance record system.
02
My part
What I owned.
I owned
- Both checkers, end to end: architecture, scripts, prompts and tests
- The three-layer design: written procedures, AI orchestration and deterministic Python
- The rulebook design, plus the guide that let Compliance edit it without code
- Content classification, the banned-phrase scanner and each hardening pass
- The audit trail, run IDs and report-only mode, enforced in code
- The move from a local vector store to Pinecone so the whole team shared one memory of approved language
03
The calls I made
Six calls that made AI review something compliance could sign.
Hand the rulebook to the Chief Compliance Officer
The rules live in a five-tab Google Sheet the checker reads on each run, so a change needs no code release or restart. He later added AI-avatar disclosure rules himself.
Classify before checking
Each piece is sorted as an adviser ad, AI product content or education, and by format, so the right rules and disclaimer placement apply. When Compliance ruled that the Marketing Rule covers app content in full, I shipped the change the next day.
Show each edit, log each run
Fact fixes land in blue and compliance fixes in green, with an audit log at the bottom of the same document. Each run gets an ID, CC or FC plus a timestamp, and the checker reads the document back to confirm each edit landed.
Approved language only
Disclaimers go in verbatim from the approved library, placed by channel, with short versions for tight formats. The checker never writes its own disclosure text. Video and audio that already aired get report-only review, enforced in code.
Each miss becomes a rule
When a variant of "guarantee" slipped through, I hardened the scanner. Reviewer feedback is routed into the rulebook or the skill as a principle, not a one-off patch. I traced over-firing legal-review flags to one rule, and Compliance rewrote it.
04
How it works
Two gates between the draft and the approver.
Each pipeline ends the same way: the fact-checker, then the compliance checker, then human approval in Themis with the audit trail attached.
01 Draft
Any format
Blog, social post, app component, video script, press release or study asset, in a Google Doc.
02 Fact-checker
Is it true?
Each claim, number and link checked against primary sources. Fixes in blue.
03 Compliance checker
Is it allowed?
SEC Rule 206(4)-1, banned phrases and approved disclaimers by channel. Fixes in green.
04 Themis
Human approval
The Chief Compliance Officer approves on the record, with both audit logs attached.
05 Publish
Ship it
Website, social, the app and the press desk.
Sources by tierExa web search for primary sources, plus the study's stat ledger for internal numbers.
The Compliance Knowledge BaseProhibited rules, required practices, disclaimers, approved claims and content formats, read live.
Risk-tiered samplingHigh-risk claims reviewed in full; low-risk content sampled.
Gate 1, edits in blue
The fact-checker
- Extract each verifiable claim: statistics, dates, laws, tax rules, quotes
- Recompute each calculation in Python, not in the AI
- Test assumptions against benchmarks, such as equity returns above 12% or withdrawals above 6%
- Verify against primary sources found with Exa, and test each link
- Rule each claim verified, incorrect, unverifiable or misleading, with a source
- Apply fixes, re-read the whole piece for contradictions, then log the run
Gate 2, edits in green
The compliance checker
- Sync the live rulebook from Google Sheets on each run
- Classify by category and format
- Scan for banned phrases in code; each match is a mandatory rewrite
- Rewrite from previously approved language, found by vector search in Pinecone
- Place the right approved disclaimers, verbatim, by channel
- Apply and verify each edit, then log the run
05
What the reviewer sees
A documented draft, not a blank page.
Tax-Efficient Investing: How It Works
Whether tax-loss harvesting creates a meaningful net benefit depends on your personal situation. It requires having both losses available to harvest and gains to offset, and it is subject to the wash sale rule, which can disallow a loss if a substantially identical security is purchased within 30 days before or after the sale.
. . .
Taxable accounts are often the focus, since that is where many annual taxable events occur. But, tax-efficient planning may also involve tax-deferred accounts, Roth-style accounts, and the sequencing of withdrawals across all account types.
FACT-CHECK AUDIT LOG
Run: FC-175926 | 14 claims: 13 verified, 1 corrected
| # | Original text | Status | |
|---|---|---|---|
| 1 | Interest income is generally taxed as ordinary income. | VERIFIED | |
| Claims 2 to 10 verified | |||
| 11 | ...can disallow a loss if a substantially identical security is repurchased within 30 days. | INCORRECT | |
COMPLIANCE AUDIT LOG
Run: CC-180242 | Category: educational | Format: full_length
| Edit | Original | Replacement |
|---|---|---|
| rewrite | where most annual taxable events occur | where many annual taxable events occur |
Blue, the fact-checker
A tax rule, half right
The draft gave the wash-sale window as 30 days after a sale. IRS Publication 550 says 30 days before or after. Fixed, sourced and logged.
Green, the compliance checker
An unsupported "most"
"Most" became "many," with the rule it triggered named in the log. Even a denial gets caught: "is not a guarantee of higher returns" became "is not an assurance of higher returns."
Also caught
A QSBS holding period
Misstated as 4 years, corrected to 5 before anyone in Compliance opened the draft.
06
What it caught
About 1 in 5 claims needed work. The checkers found them first.
From my tally of the fact-check and compliance logs in finished documents over about five months: social copy, podcast clips and in-app video scripts.
1 in 5
claims were corrected, or flagged as unverifiable, misleading or needing a human, before a reviewer saw them.
Claim verdicts, share of claims checked
The rest of the 19% were unverifiable claims, one misleading statistic and claims flagged for human review.
Nearly 2
SEC Marketing Rule edits per document, made before the Chief Compliance Officer's review.
What the compliance checker changed
- Rewrites, citing the rule triggered51%
- Approved disclaimers added43%
- Removals6%
Share of Marketing Rule edits in the same documents.
The rulebook Compliance grew inside it
Prohibited-language rules in the Compliance Knowledge Base, against the seed rulebook. Required practices more than doubled.
The rulebook changed month after month, and none of those changes needed a code release.
2gates on each pipeline
5rulebook tabs Compliance controls
111deterministic tests
07
What changed for compliance
From reading each piece to auditing a sample.
With both gates in place, the CEO replaced 100% manual review of low-risk content with risk-tiered sampling and set firm turnaround targets. High-risk claims still get the Chief Compliance Officer's full review.
Share of content read in full by hand, by risk tier
Before the checkersEach piece
100%
Tier 1, high-risk claimsPre-approved by the Chief Compliance Officer
100%
Tier 2, low-risk contentSampled
50%
Tier 3, low-risk educationalSampled
25%
Sampling steps down 5 points each time a tier holds a 95% pass rate. For the low-risk tiers, that is 50 to 75% less manual review.
24hrs
The CEO's turnaround target for content cleared by the checkers
4hrs
His target for non-controversial items
Files a month in the factory's shared workspace
Source: the shared drive listing, against the month it shipped.
450+ hrs
About 11 work weeks of manual fact-checking and first-pass review, from the first pieces logged for compliance.
- Strategy and planningTime back for the plan, not the proofreading.
- More contentMore posts, videos and articles each month, through the same two gates.
- Higher-risk reviewThe Chief Compliance Officer's time moved to the claims that need a human.
- More AI toolsNew skills and tools built for other teams.
08
Results
What moved.
| Result | What it means |
|---|---|
| 1 in 5 | Claims fixed or sent to a human before review (19%) |
| 1 in 6 | Claims that were factual errors, caught and corrected before review (16%) |
| Nearly 2 | SEC Marketing Rule edits per document: 51% rewrites, 43% disclaimers added, 6% removals |
| Nearly 4x | Prohibited-language rules, in a rulebook the Chief Compliance Officer edits without code |
| Up to 75% | Less manual review for low-risk content, sampled at 25 to 50% instead of 100% |
| 24 hours | The CEO's turnaround target for checker-cleared content, with 4 hours for non-controversial items |
| 22x | Growth in files a month in the factory's shared workspace, in about four months |
| 450+ hours | Manual review saved, about 11 work weeks, from the first logged pieces |
09
How I would scale it
From a review gate to a compliance system that learns.
The Chief Compliance Officer asked what Evergreen could show a regulator who wanted a demo. Each checked document already carries its own audit trail. These are the next moves, most of them already designed.
Filing
Approved pieces file themselves
I pushed the compliance platform for an API, and designed the filing layer so only the uploader changes when it arrives. Cleared pieces and their logs then land in the record system on their own, with no manual upload.
Learning
Each reviewer edit trains the checker
I designed a pass that reads Compliance's comments and accepted edits and turns them into rules. Each override becomes a principle, so the same note never has to be written twice.
Evidence
Sampling that earns its way down
Track pass rates and draft-to-approval time by tier, and step sampling down only when the data holds. Review effort follows risk, on evidence a regulator can read.
Coverage
More formats
Check images, video and app components before they render, not after.
The principle
Give the rule owner the rules, run what must never vary as code, and let AI do only the judgment. Then the reviewer stops reading each piece in full and starts auditing a sample.
For your brand
A fact-check and compliance gate, built on your rules.
- 01Map what your reviewers flag most, and where content waits
- 02Move your rules into a rulebook your compliance team edits without code
- 03Run banned phrases, math and disclosures as code, and leave judgment to AI
- 04Add a fact-check gate with primary sources and recomputed math
- 05Put an audit trail in each document, ready for your record system
- 06Move low-risk content to sampling, and step it down on evidence