Flawed methods and procedural irregularities — the third Demer factor — are the earliest bias evidence in a claim file, and most of it costs nothing to find. One rule governs: read the expert’s report cold, against these checks, before you read anything else in the file — your first read is the nearest you will come to a neutral gatekeeper’s view, and you are building an exhibit, not forming an impression. Here is the screen — the checks only. The case anchor behind each one, the discovery sequences that convert a check into proof, and the objection scripts are in the Implementing Kit on the companion Substack.
The facial screen: ten checks on the report itself
- Find the methodology statement. What did the expert do, in what order, with what tools, under what protocol? A report that cannot describe its method cannot be checked against one — note the absence and move on; you have your first exhibit.
- Ask “says who?” of every principle. List each general principle the report applies — damage of this kind indicates that cause; findings like these are inconsistent with disability — then look for its source: literature, a field standard, a guideline, data. In flawed reports the answer, over and over, is no one. An unsourced principle is untestable, its omission departs from the expert’s own field’s rigor, and you should presume it invented until the expert identifies its authority. This is the most commonly failed and most commonly overlooked check on the list.
- Check for the field’s own standard. Where the discipline publishes a governing standards document, a report that never mentions it is telling you whether it was followed.
- Look for tested hypotheses. A neutral investigative question, competing explanations enumerated and ruled out — or a conclusion that arrived fully formed.
- Look for qualified conclusions. Limitations, probabilities, what could not be determined. Experts qualify; advocates conclude.
- Read the assignment framing. “Whether the damage is consistent with the insured’s account” steers; cause-and-origin does not. Bias can enter in the referral question, before the expert writes a word.
- Screen for boilerplate. Pre-formatted conclusions and recycled text — and, across claims, near-identical reports with the names changed — are pattern evidence of the most direct kind.
- Audit the documentation. Do the photographs and excerpts record the scene, or support the conclusion? Note every contrary indicator the report leaves unmentioned.
- Match the conclusion to the cited evidence. Are the hallmark indicators of the attributed cause actually documented in the material the report itself cites?
- Check preservation. Material retained for independent review, photographs dated and authenticated, testing replicable? A process that forecloses checking was built not to be checked.
The process screen: ten checks around the report
- Flag the paper-only credibility rejection. A reviewer who discounted the claimant’s reported symptoms without ever conducting an examination — the irregularity at the center of Demer.
- Pull every draft and every referral question. The report sent back with progressively narrower questions until the desired answer emerged is a recognized irregularity — and it lives in the iterations, never the final document.
- Flag unexplained shifts. An assessment that flipped — restrictive to permissive, disabled to not — between drafts or reviewers, with no stated reason.
- Flag the cherry-pick. One office note credited against years of contrary treatment records.
- Flag disregarded findings. The SSDI award, the workers’-comp finding, the treating physicians — dismissed without engagement rather than addressed.
- Ask whether anyone called the treating physician. Where the dispute turned on the treaters’ findings, the failure even to contact them is itself the irregularity.
- Compare versions and signatures. The “corrected” report identical to yesterday’s but signed by a different physician; the vendor-drafted opinion adopted by the signer.
- Ask what file the reviewer received. A reviewer fed the insurer’s curated version of the record reviewed the insurer’s case, not the claimant’s.
- Read the retention letter. An engagement letter that announces the expected conclusion before the work began — in one seminal case, a letter from an in-house consultant who had never examined the claimant had already asserted there were no objective findings of disability — has pre-committed the opinion.
- Name the invented standard. A demand for “objective evidence” of a self-reported condition, imposed by no policy term and sourced to no authority — check #2, institutionalized. It was the signature device of the industry’s most fully documented biased-expert operation: the case study.
The four mistakes that waste the screen
- Reading the file before the report — the adjuster’s framing contaminates the cold read that a neutral gatekeeper — and a court — would give the document.
- Spending irregularities alone — a single claim’s defects, without relational metrics and pattern (or reasonable-measures) evidence, reads as a quality dispute, not bias. The structure that wins is multiplicative: metrics × irregularities × pattern.
- Arguing adjectives without a yardstick — “flawed” is rhetoric until it is deviation from a standard. Get the expert’s, the vendor’s, the insurer’s, and the field’s standards early; then every deviation is an exhibit.
- Attacking qualifications instead of methodology — the credentials fight is the fight insurers want. The screen is about what this expert did on this file, which no diploma answers.
Get the full Kit. The twenty checks above are the screen. The working version — the case anchor behind each check; the “sources demand” sequence that makes the expert identify the authority for every principle (and what each possible answer proves); the standards-and-practices discovery program at the expert, vendor, and insurer levels; the process-timeline reconstruction; and the scripts for the six comebacks insurers reliably make — is the Implementing Kit, published on the companion Substack.
Get the Implementing Kit → Read the overview → See the case study →
Related
See the factor this checklist serves in The Bias You Can Read in the Report Itself, and the invented standard at industrial scale in the UNUM case study. The record-building sequence across all factors is in Building a Bias Record.
Distilled from the project’s Factor 3 doctrinal synthesis, including the generalized facial checklist derived from testing a set of insurer-commissioned forensic reports against the field’s gold-standard investigation guide. Seminal authority — Demer v. IBM Corp. LTD Plan, 835 F.3d 893 (9th Cir. 2016); Sargon Enterprises, Inc. v. University of Southern California, 55 Cal.4th 747 (2012). The case anchors, discovery sequences, and objection scripts behind each check are reserved for the subscriber Implementing Kit. Educational and informational only; not legal advice.