Documented experiments.
Not marketing claims.

Eighth session on an ongoing build diary: a month-end close application I'm building myself, session by session, with an AI assistant. The earlier sessions are further down this page. This was the first session in a while that produced actual code — a rebuild of the component that calculates close deadlines in business days, skipping weekends and holidays.

I found what looked like a real defect in that component, fixed it, and had the fix checked three separate times by cold-start reviews with no conversation history — just the filed documents. All three confirmed it. One listed it explicitly under "verified as correct." Then I asked the organization a single question about how it actually observes a holiday that falls on a weekend, and the finding evaporated. It does not shift an observance backwards into the previous calendar year. Nothing had been missing. The original data was right. My fix had put a wrong date into a view people use, and the test suite I had just written pinned that wrong date in place as a regression check.

Every review audited my reasoning — is the arithmetic right, does the rule land on that day. None audited my premise, because I never wrote the premise down as an assumption. I wrote it as a fact, so it got checked as a fact and it checked out. Reviewers verify what you claim; they cannot verify what you did not know you were assuming. The step that catches this is not another review — it is listing every place an assumption stood in for an answer, and then going and asking. The accounting version is the tie-out that proves the computation while nobody checks where the input rule came from.

Same session. Until this point the application had no automated tests at all. This was the first one — written for the component that counts business days for close deadlines.

Version one never actually ran the code. It read the holiday list out of the page as text and compared it to a list I had built separately, so when I sabotaged the underlying rules identically in both places it passed all ten checks against a calendar that would have shipped wrong. Version two ran the code, but proved the safety mechanism existed by searching the file for its name — a comment mentioning the function faked a pass. Version three stripped comments first and was still a text search: moving the call inside a function that nothing ever calls also passed. And all three shared a larger hole. Not one of them asserted a single calculated date. With the day-counting function deliberately broken so that a real deadline landed on a holiday, version three passed fourteen checks out of fourteen.

When you write a check, name the thing that would be wrong and ask whether this check would see it. "The reference data is complete" and "the answers it produces are correct" are different claims, and only the second one is what the work depends on. The same distinction separates a review step that confirms a schedule is attached from one that confirms the schedule foots. A check that looks for the presence of a control is not a test of the control.

Same session. This continues the entry from August 6 below — the one where re-testing a repair pass turned up seven new defects inside it. This is the sharper version of that lesson, and it took two more sessions to see it.

Four defects were fixed in one session, and every fix reproduced the class of defect it was fixing. Fixing a problem where things were identified by their position in a list, I renumbered a rule that six other documents cite by number — inside the document whose entire subject is that identity must not be positional. Fixing a place where two different facts had been recorded as one, my correction stated the wrong one of the two. Building a guard that stops the engine calculating outside the range it can vouch for, I left eight unguarded calls sitting in front of it. Writing a check that a decision had actually become code, I searched the file for the code instead of running it.

The defect you are repairing is the shape currently occupying your attention, which makes it the shape most likely to come back out of your hands. The countermeasure is narrow and cheap: after fixing a defect of class X, audit the fix itself for class X. Not the whole system, not "review carefully" — one question aimed at one thing. In practice that is the difference between a remediation that closes a finding and one that quietly reopens it somewhere else.

Seventh session, same build diary. Design work on a new module — a location and property register. The project keeps its rules, its decisions and its current state in one folder, and every instruction I have written says to read that folder first because it is the authority.

The folder was not mounted. The tool had been reading a cached copy — a mirror my own rules describe as not authoritative. So the context reads happened, four scope decisions got made and a full proposal got written, all against a copy, with nothing anywhere reporting that the real thing was unreachable. I had written a contingency for this. It was aimed at the wrong failure: it described a person becoming unavailable, and what actually happened was a device failing silently while I sat there working normally.

The fix was not a warning banner. It was an instruction to stop. A check that surfaces a problem and then lets the work continue reproduces the original defect more politely — the mechanism is still silent about its own limits. If a precondition matters, failing it has to halt the work. Worth asking of any process that reads from a synced folder or a shared drive: what happens when the definitive copy simply is not there, and would anyone find out?

Same session. A single date-calculation component drives every deadline in two modules of the close app. It works in business days, because close deadlines do.

The new module tracks contract notice periods, which are calendar-day obligations. "Not less than 90 days prior written notice" means 90 calendar days — a holiday does not extend it and neither does a weekend. Running that through a working-day engine produces an answer that is wrong and authoritative-looking: right field, right format, full confidence. So the new module is calendar-day by default, and the working-day engine is confined to internal preparation dates, marked explicitly wherever it is used.

Before reusing a shared calculation, ask whether the new domain runs on the same clock — not whether the component is available. This is the first time in this project a shared component has been deliberately excluded, and the precedent matters, because "shared component" had quietly started to mean "use everywhere." The same question is worth asking wherever one calculation serves two kinds of deadline.

Sixth session on an ongoing build diary: a month-end close application I'm building myself, session by session, with an AI assistant. The earlier sessions are further down this page. This time I asked a second, independent AI to review the project's filed documents cold — no conversation history, just what's on file.

The review came back with real findings, three of which would have caused genuine problems later. Fourteen items got fixed. The instinct was to report that and move on. Instead I re-tested the fixes themselves, narrowly, against the specific claims they made — and they had introduced seven new defects. Three places claimed a task was "done" while a to-do list in the same document still listed it outstanding. A document that had just declared itself "the single source of truth for counts" had a wrong count in its own text. Something edited an hour earlier was still described as unchanged. A finding count didn't match its own findings list. A numbering scheme meant to demonstrate that order isn't identity was itself out of sequence.

A repair pass is a mechanism, and it fails the same way every other mechanism fails: by asserting correctness instead of demonstrating it. If you have AI clean up a schedule, a mapping or a workpaper, the cleanup is not evidence. Test the specific claims the cleanup made. That was the fifth instance of this same pattern I found in a single session — in a numbering scheme, a document mirror, a certification rule, a piece of analysis, and finally in the act of fixing the first four.

Sixth session, same build diary. Opened intending to start the next module and got no further than loading the documents of record.

Reading both documents before writing anything — a rule set two sessions earlier — turned up something that stopped the session cold. Two of my own files each define "module 3," and they name different modules. Both use the same phrase, which is what made it dangerous rather than obviously wrong. Building the wrong one would have skipped a reconciliation step entirely while my own tracker recorded that work as done. The fix was not renumbering: a build order that had already changed once was not a stable enough thing to freeze. Every module now carries a permanent name describing what it is, independent of where it currently sits in any list, with the old numbers kept as history rather than deleted.

Underneath that was the larger finding. The status tracker had not merely renumbered modules over several sessions — it had silently redesigned parts of the application, splitting one module, merging two and dissolving a third into scattered pieces, with none of it recorded anywhere as a decision. All three re-cuts turned out to be defensible on review. The finding is not that they were wrong. It is that real design decisions got made without anyone deciding them on purpose, because a reorganized table does not announce that it has become a redesign. Any recurring schedule that gets tidied between cycles has the same exposure.

Same session, working from real account data for the first time in this build. I classified a couple of hundred reconciliations and flagged one figure as the session's most valuable output.

One sentence from the person whose data it actually was ended that: the underlying figures were not finished and should not have been used. Everything built on them went with them — a flagged discrepancy, several status counts, and the item I had personally labelled most urgent, all of it zero once complete data was used instead. The part worth sitting with is that I had already written down the evidence that something was off, several paragraphs earlier, in an unrelated observation about the same accounts — and filed it as a question rather than stopping to answer it before running the analysis. The right move was one clarifying question before two hundred rows of work, not a retraction after.

What survived is the more useful half. Everything that depended on what an account is — its category, its method, its risk level — held completely, because none of it depended on a specific balance. Everything that depended on a number from unfinished data collapsed. Structural judgment is robust to bad data in a way classification never is, which is an argument for doing the structural thinking first, before touching any numbers at all.

Fifth session, same build diary. This one built a key-dates calendar — meetings, statutory filings, and prep-start dates derived from them — from a single approved source document.

Building a calendar that mixes confirmed and inferred dates exposed a gap I hadn't planned for: a date confirmed from a filed document and a date derived from a rule render identically in a plain date field. Nothing distinguishes "known" from "probably true," and a reader trusts both equally, which is backwards. The fix was making provenance a required field rather than a documented best practice — every new entry must declare, before it saves, whether its date came from an approved document, was derived and is unverified, or was typed in by hand. That answer is written permanently alongside the date.

The accounting parallel is direct: "confirmed per statement" and "estimated pending statement" need to be distinguishable in the record, not just in someone's memory of how the number was produced. If the system can't tell the two apart, it will eventually be trusted for something it doesn't actually know. It had to be required rather than optional because a similar expectation already existed as unenforced guidance elsewhere in the same project — and had already been silently skipped once under time pressure. A rule that depends on someone remembering it under pressure isn't a control. A rule the entry form itself enforces is.

Fourth session on the close application. No organizational data of any kind was involved — which makes this one fully replicable by anyone reading it.

I opened the AI project that supposedly held four sessions of work and got back "no knowledge." The cause was a naming collision between the project (a knowledge container) and the connected folder (where everything actually lived). Nothing was lost. But it exposed that I had no written rule about which container was authoritative, and no mechanism to keep the answer true over time. So the session became a governance one: a one-page operating model declaring the folder the single source of truth, a context file at the folder root that the tool reads automatically at the start of each session, the previous session's documents finally filed, an open data-classification question resolved, and a fingerprint manifest that flags which documents have drifted since they were last synced.

Two things the AI got wrong along the way, both corrected only after I pushed back: it implied a connected folder would stay attached to every future session (it doesn't — folders are selected per session), and it told me a staging folder was safe to delete (it isn't — that folder is what the drift check compares against). Confidently wrong is the failure mode to plan for, not obviously wrong. If you are about to start multi-session AI work, decide in writing which artifact is authoritative before session two.

Fourth session, same build diary. Resolving a data-classification rule I had written earlier: no financial figures would live in the third-party database — statuses and flags only.

It was the cautious answer and it was wrong, because a reconciliation stripped of its balances cannot be reviewed. A reviewer looking at "marked complete" has not reviewed anything; they have read an assertion. The balance, the reconciling amount and the difference are the substance. The replacement is tiered. In: account-level balances, reconciling amounts, differences. Out: transaction detail, payroll and compensation, personal information, full account numbers. Staying in-tenant: the general ledger system of record and audit-grade working papers.

The transferable point is about who draws the line. A security-only instinct minimises what leaves and produces a control nobody can operate. A convenience-only instinct lets everything out. The workable line needs someone who knows what the work actually requires — which in practice is usually the Controller, not the person who owns the tool. Account-level figures are internal management information; transaction and compensation data are a different category and do not leave the tenant.

An inherited quarterly reporting pack in real accounting work — undocumented, unknown age, no tie-out controls, single preparer. Diagnosed, remediated, and then run live for a quarter. Anonymized throughout: no organization, no real figures, magnitude bands only.

I built roughly 88 live tie-out checks on a pack that had never had one — layer-to-layer amount ties for every month of every data block, mapping-exception lines that name what to fix, and an "amount at risk" column so exceptions get triaged by cost rather than counted. It ended at 86 green, with one documented accepted variance and one informational count. Then it missed something. One key existed as two rows in the data with the balance split between them; refreshing from source wrote the full figure into both, overstating a reporting line by a five-figure amount for the quarter. The reports and the checks read the same two inflated rows, so they agreed with each other perfectly.

What caught it was recomputing the headline figures directly from the raw source, in code, independently of the model — then comparing. After the fix, every line matched to the cent, quarter and year to date, with prior quarter plus current quarter equalling year to date on every line. The rule this produced: a control that reads the same data as the thing it controls has a blind spot by construction. Verifying a model against itself is not verification. With one preparer and no second reviewer, an independent recomputation isn't a nice-to-have — it's carrying part of the review.

Reviewing an inherited quarterly reporting pack before automating it. Undocumented, unknown age, no tie-out controls, a single preparer. This is the companion to the entry above, from the same engagement.

On one summary sheet, the quarter columns referenced the year-to-date sources and vice versa. In a first quarter those two windows are identical, so the error produced correct output and reconciled cleanly every time anyone checked it. It would have transposed every figure on the sheet the moment a second quarter loaded. It was not found by checking the numbers — the numbers were right. It was found by asking what the model does when the period changes, then flipping the period on data I already trusted and proving the windows moved.

Additivity is the check that catches this class: prior quarter plus current quarter must equal year-to-date, cross-checked against an independent sum of the raw month columns. More generally, a control that only ever runs under one set of conditions has not been tested — it has been observed under the conditions that hide the defect. The period logic now lives in two input cells, fiscal year and quarter, which also drive every heading on every sheet; before that it was hardcoded month ranges across roughly 750 rows and a dozen labels retyped by hand each cycle.

Third session on the close application, and the first one that built anything real. Across the two prior planning sessions I had fixed one rule: the system of record stays in our own environment — files we control — specifically so nothing gets trapped in a vendor's tool. No organizational data was involved; the app holds close-process metadata only.

I built the first component, hit a wall — the thing I'd built couldn't actually be shared with a team — and solved it by moving the backbone to a managed third-party database. A good fix. It also quietly overturned the "keep it in our own environment" rule, and I made that reversal without my own blueprint in front of me. At the end of the session I fed the two prior diaries back in and asked the AI to reconcile the new build against them and flag anything that superseded an earlier decision. It caught the contradiction immediately. I then made the reversal deliberately — written down as a formal supersession, with the tradeoff spelled out: I gain live team sharing and automation, I give up some of the "nothing trapped" custody.

Externalize the memory. Each AI session is amnesiac; you are the continuity. A written decision log plus an occasional "where does the new work contradict the old?" pass is cheap, and it does exactly what an auditor's workpapers or an engineer's changelog does — it catches drift. Changing your mind is fine. Changing it without noticing is the risk.

Second session on the close application, run as a deliberate comparison. Whether the "right" way to build a working tool with AI is to design it thoroughly up front or build it iteratively as you use it — tested by doing both, on two different tools. Tool one: a daily task-and-meeting tracker built on the fly with an AI coding assistant, no blueprint, roughly ten hours over nine days. Tool two: this month-end close application, designed across two planning sessions without a single line of it written yet.

Building on the fly was useful almost immediately and taught me what I actually wanted by making me live with it. Features earned their place by solving a real morning annoyance. Planning first produced a coherent scope, a written record of what I chose not to build so scope can't creep back in, and a security and data-handling decision made before any real number was at risk. Each cost something. On the fly, I redid one screen layout five times and had no upfront check on architecture or security. Planning first, after two sessions nothing existed yet.

Neither method is correct in the abstract. Match it to the stakes and the blast radius: a personal productivity tool, build on the fly; a system touching financial data, multiple entities and a year-end audit, plan it first. The only real mistake is using the low-stakes method on a high-stakes build. The same session showed why the domain expert has to be in the room — asked to cut features tied to funding types the organization doesn't have, the AI correctly removed them, but it had filed functional expense allocation (required on every nonprofit's Form 990 and audited statements) right next to those optional fund-accounting features. A careless cut would have deleted a filing requirement along with the extras. AI groups features by surface resemblance; a practitioner groups them by reason for being.

A separate build from the close app: a combined task, meeting, project and month-end-close tracker, built with an AI coding assistant. Roughly 10 hours of work spread across 9 days. I described what I wanted in plain English; the AI wrote every line of code. My side was requirements, visual judgment and testing. It runs as a private web app on free hosting tiers, secured to a single login.

It replaced a fragmented notebook-plus-spreadsheet setup with one screen — a three-day dashboard, a close checklist with live progress, a multi-week calendar, and a meeting tracker that turns a pasted transcript into a structured summary and lets me ask questions about any past meeting. Describing new features in plain English worked as advertised. The honest part is what took iteration: layout was harder than logic, with one dashboard view taking about five rounds in a single session to look right while the underlying data logic worked almost immediately. The AI summaries needed tuning — too brief, then too verbose, then right. And a bulk file edit I ran corrupted the text encoding across the whole app until the AI diagnosed it from a screenshot and rewrote the affected files.

The transferable skill is not coding, it is describing exactly what you want. "Make it better" gets you nothing. "Make the cells taller so five items show" gets you the thing you pictured. That is the same specificity a good review note requires, and most people already have it — they just have not tried pointing it at software.

First session on the close application. Whether an AI assistant could take a Controller's full requirements and produce a genuinely usable design for a month-end close application — architecture, security assessment, data pipeline spec — without any real financial data entering the tool. One working session, roughly two to three hours. Inputs were requirements, the ERP vendor's public documentation, and the organization's AI-use policy. No GL balances, no vendor or customer data, no dollar amounts.

The session produced a five-module blueprint, a two-page IT integration request, a security assessment mapped against the AI policy, and a detailed GL export/import specification. The architecture it landed on — your own cloud storage as the system of record, structured workbooks as the database, a compact query layer so large files aren't re-read — mirrors how commercial close platforms are built, on infrastructure a small team already owns. Then it missed something I knew from experience: its first-pass export design assumed the standard GL detail report carries vendor and customer names. It doesn't. I flagged it; the corrected design joins small master lists on the stable source numbers.

An AI is a fast architect, not a substitute for domain judgment. The more useful result was one I didn't ask for: pressed for an honest security assessment, the tool ranked five operational risks it judged larger than the training-data question everyone opens with — led by connector write permissions, a tool connection that can act rather than just read. The training-data question is the one everyone asks and rarely the biggest real risk. A tool that tells you your question was the wrong one is more trustworthy than one that only reassures.