# Gut Health Recall Tracker — method A standing quarterly reference. Every US food recall that names a pathogen or contaminant with gastrointestinal presentation, from both federal agencies that issue them, normalised so that year-over-year comparisons are possible. Everything below is reproducible by running `build-recall-tracker.py`. --- ## 1. Sources ### FDA — openFDA food enforcement - Endpoint: `https://api.fda.gov/food/enforcement.json` - Documentation: https://open.fda.gov/apis/food/enforcement/ - No key required. - Dataset `last_updated` at the time of this pull: **2026-09-16**. The API caps a page at 1,000 records and caps `skip` far below the size of an eight-year pull, so the window is sliced into one query per calendar year and each year is paginated to exhaustion. The exact query, for each year Y in 2019…2026: ``` GET https://api.fda.gov/food/enforcement.json ?search=report_date:[Y0101 TO Y1231] &limit=1000 &skip=0,1000,2000,… ``` The build asserts that the number of records retrieved equals the `total` the API reports for that window, so a truncated pull fails loudly rather than silently producing a short year. Records retrieved per year (report_date basis): | Year | Records | |---|---| | 2019 | 1,863 | | 2020 | 1,286 | | 2021 | 1,181 | | 2022 | 1,652 | | 2023 | 2,155 | | 2024 | 1,409 | | 2025 | 1,617 | | 2026 (to 2026-09-16) | 959 | | **Total** | **12,122** | ### USDA FSIS — recall and public health alert API - Endpoint: `https://www.fsis.usda.gov/fsis/api/recall/v/1` - Documentation: https://www.fsis.usda.gov/science-data/developer-resources/recall-api - No key required. The endpoint returns the **entire** dataset on every request; there is no pagination and no date filter to apply. **Access note, and it matters.** At the time of this pull the whole `fsis.usda.gov` host returned HTTP 403 to this network — to `curl`, to a server-side fetcher, with and without browser user-agent strings, over both HTTP/1.1 and HTTP/2, on the API path and on ordinary web pages alike. This is a CDN-level block on the requesting network, not an authentication requirement and not anything about the query. The build therefore falls back to the Internet Archive's capture of **the same official endpoint**, which is the agency's own response body, byte for byte: ``` https://web.archive.org/web/20260724064506id_/https://www.fsis.usda.gov/fsis/api/recall/v/1 ``` - Snapshot timestamp: **2026-07-24 06:45:06 UTC** - Latest FSIS notice contained: **2026-07-23** Consequences, stated plainly: - The USDA half of the tracker ends on **2026-07-23**, two months earlier than the FDA half (2026-09-16). Any 2026 figure that mixes the two agencies is reading two different windows. The summary flags this everywhere it applies. - A request for a fresh Internet Archive capture was attempted and failed (the archiver could not reach the host either), so this gap cannot be closed from here. It closes by itself the moment the build is run from a network the FSIS CDN does not block — `build-recall-tracker.py` tries the live endpoint first on every run and only falls back if that fails, recording which route it took in `recall-raw/fsis-provenance.json` and in `recall-summary.json`. Two earlier archived captures exist (2025-02-01, 2025-02-03). They were checked and **not** merged: the only records they hold that the July 2026 capture does not are 361 notices from 2010–2013, entirely outside this tracker's window. The July 2026 capture is complete for 2019 onward on its own. The FSIS payload ships an English and a Spanish record for many notices (`langcode`). Spanish duplicates are dropped; 2,012 raw records reduce to 1,223 notices, of which 548 fall in the 2019–2026 window. ### CDC NORS — not used The NORS dashboard (https://wwwn.cdc.gov/norsdashboard/) was considered as an illness denominator and **skipped**. It is an interactive dashboard whose data comes out through a manual export, there is no clean programmatic pull, and its outbreak-year basis does not align with a recall-report-date basis. Attaching illness counts to recall counts without a shared unit of analysis would have produced a comparison that looks rigorous and is not. The tracker therefore describes recalls and only recalls. ### Pull timestamp - openFDA retrieved: **2026-09-22** (dataset `last_updated` 2026-09-16) - FSIS retrieved: **2026-09-22**, from the 2026-07-24 archived capture - Build run: see `generated_utc` in `recall-summary.json` --- ## 2. Unit of analysis — the single most important thing on this page **The two agencies do not count the same way, and neither does anyone who republishes their numbers.** - The FDA publishes **one enforcement record per recalled product line**. One recall of a salad manufacturer covering many SKUs appears as many records. The largest single event in this window, Fresh Ideation Food Group in March 2023, is **409 records** and one recall. - FSIS publishes **one notice per recall**. So a naive row count is not a recall count, and an FDA-versus-USDA comparison built on row counts is meaningless — it compares SKUs against recalls. In this window the FDA side averages 3.29 records per event while the FSIS side is 1.00 by construction. The tracker keeps both: - `recall-tracker.csv` is **one row per source record** — nothing is collapsed, nothing is lost. - `event_id` groups records into events. For FDA rows it is openFDA's own `event_id`; for FSIS rows it is the recall number. - **Every figure in `recall-summary.md` is an event count unless the line says *records*.** In the 2019–2026 window: 12,670 records, 4,231 distinct events. --- ## 3. Date axis The year axis is: - FDA: `report_date`, the date the recall appeared in the FDA Enforcement Report. - FSIS: `field_recall_date`, the date of the notice. `report_date` is used rather than `recall_initiation_date` because the pull window itself is defined on `report_date`, which makes each year complete and internally consistent. `recall_initiation_date` is carried in the CSV as `date_initiated` and is used for the month-of-year analysis, where the event date matters more than the publication date. **There is a lag between the two, and it is often large.** The Midwest Poultry shell-egg recalls were initiated 2026-07-22 and published 2026-08-19. A recall covered in the news in one month can land in the Enforcement Report in the next. Records initiated in late 2018 can appear in the 2019 report year, and recalls initiated in the weeks before the pull date may not have been published yet — which means **the most recent two to three months of 2026 are under-counted and will fill in on the next quarterly rebuild.** Seasonality is computed on `recall_initiation_date` and restricted to the complete years 2019–2025, so the partial 2026 does not tilt any month. Each event is placed in the month of its earliest record, so no event is counted twice. --- ## 4. Normalisation rules All classification is rule-based and ordered; the first rule that matches wins. There is no model and no judgement call in the pipeline — the rules are in `build-recall-tracker.py` and can be read, argued with and changed. ### 4.1 Pathogen, from free text Matched against the recall reason (FDA: `reason_for_recall`; FSIS: `field_recall_reason` + `field_title` + `field_summary`), case-insensitive, in this order: | Output value | Matches on | |---|---| | Cyclospora | `cyclospor` | | Hepatitis A | `hepatitis a`, `HAV` | | Norovirus | `norovirus`, `noro virus`, `norwalk` | | Cronobacter | `cronobacter`, `sakazakii` | | Clostridium | `clostridium`, `botulinum`, `botulism`, `perfringens` | | Listeria | `listeria`, `monocytogenes`, `L. mono` | | E. coli STEC | `E. coli`, `escherichia`, `STEC`, `shiga toxin`, `O157`, `O26`, `O103`, `O111`, `O121`, `O145` | | Salmonella | `salmonell` | | Other pathogen | `staphylococc`, `bacillus cereus`, `shigella`, `vibrio`, `campylobacter`, `yersinia`, `giardia`, `toxoplasm`, `cryptosporid`, `coliform`, `enterotoxin`, bare `pathogen`, `microbial contamination` | Three things to know about this list: 1. **Order is specificity, not frequency.** The specific organisms are tested before the catch-all, and `Other pathogen` is discarded whenever a specific organism also matched. 2. **Multi-organism recalls are not lost.** A reason naming both Salmonella and E. coli gets a single `pathogen` (the more specific of the two, by the order above) so that pathogen columns sum to the total without double counting, and the full list is preserved in `pathogens_all`. 154 records in this window name more than one organism. 3. **`E. coli STEC` is broader than its name.** FDA reason text is frequently just "E. coli" with no serotype, and a small number of recalls concern generic or non-toxigenic E. coli as a hygiene indicator rather than a Shiga toxin-producing strain. These are folded into the same bucket because the source text does not reliably distinguish them. Read the bucket as "E. coli named in the reason", and check `reason_text` before writing a sentence that turns on the serotype. `gi_pathogen_recall` is 1 when any pathogen rule matched. That flag, not the whole file, defines the tracker's headline population. ### 4.2 Undeclared allergens — held separately, on purpose **Undeclared allergens are not gut-pathogen recalls.** They are the single largest category of food recall in the United States and they will swamp any count that does not separate them. They are kept in their own boolean column, `undeclared_allergen`, with the specific allergens in `allergens_named`, and they are excluded from every pathogen figure in the summary. To exclude them from your own cut, filter `undeclared_allergen = 0`. A record is flagged only when a labelling-failure phrase (`undeclared`, `does not declare`, `not listed on the label`, `misbrand`, `mislabel`, and similar) appears **and** either a specific allergen is named or the text itself says "allergen"/"allergic". A bare labelling error with no allergen anywhere in it is a labelling violation, not an allergen recall, and is classified as such. A record can be both an allergen recall and a pathogen recall; the two columns are independent. ### 4.3 Non-pathogen contaminants When no pathogen matched, `contaminant` takes the first match from: Foreign material; Chemical or heavy metal; Temperature abuse; Insanitary conditions or process deviation; Upstream supplier recall; Labelling or regulatory violation, non-allergen. These exist to make the residual visible and to let a reporter say what the *rest* of the recall record is made of. They are not part of the gut-pathogen population. ### 4.4 Product category 22 ordered rules run against the product description (FDA) or product items plus title (FSIS). The ordering encodes one rule: **form beats flavour.** A product description usually names both a form ("yogurt", "bagel", "hummus") and a flavour ("strawberry", "jalapeño"). Form is tested first, so a strawberry yogurt is dairy rather than produce and a jalapeño bagel is bakery rather than produce. An earlier draft of this file did not do that and inflated "Other fresh produce" by roughly 140% with flavoured processed foods. If you change the rule order, re-check that bucket first. Categories whose vocabulary is unambiguous (infant formula, pet food, sprouts, named leafy greens) sit above the rest so they are never captured by a broader rule below. ### 4.5 Distribution scope and states affected FSIS gives a structured state list (`field_states`) and it is used directly. The FDA gives free text (`distribution_pattern`), which is parsed as follows: 1. Nationwide phrasing (`nationwide`, `nationally`, `all 50 states`, `throughout the United States`) → scope `Nationwide`, states count left empty. A nationwide recall has no meaningful state count and is not given a fake one. 2. Full state names, case-insensitive, word-bounded. 3. Two-letter uppercase tokens in the original casing. Codes that are also ordinary English words (`IN`, `OR`, `ME`, `OK`, `HI`, `DE`, `LA`, `PA`, `MA`, `MS`, `MD`, `AL`, `MI`, `MT`, `NE`, `SC`, `ID`, `AR`, `CO`, `WA`, `VA`, `OH`) are accepted **only** when they sit in an explicit list context — next to a comma, slash, semicolon or multiple spaces. Otherwise "distributed in Oregon **or** Washington" would score a phantom state. `states_confidence` records the outcome: `high` (at least one full state name or unambiguous code), `low` (only ambiguous codes in list context), `none`. **Every states statistic in the summary is computed on `high` confidence rows only, with nationwide rows excluded**, and the summary states how many events that leaves out of the total. Median is reported alongside the mean because the distribution is heavily skewed by a small number of very wide recalls. --- ## 5. What does not classify Nothing is dropped. Records the rules cannot place keep their raw `reason_text` and `product_description` in the CSV and are counted in the summary's "What did not classify" section. At this pull, of 12,670 records in window: - 892 (7.0%) have a reason text that matched no pathogen, no contaminant class and no allergen rule. These are a long tail of one-off reasons — "elevated water activity level in shelf-stable products", "lack of functioning temperature indicating devices in retorts" — not a systematic gap. - 1,295 (10.2%) have no product category. Mostly brand-name products whose description never says what the food is ("Velvet Vanilla Lovers Trio 56oz Carton"). - 387 gut-pathogen records (7.4%) have distribution text that could not be resolved to states. These numbers are printed to the console on every build and published in the summary. If a future rebuild moves them sharply, the source data changed shape and the rules need revisiting. --- ## 6. Sanity checks Six regression checks run on every build and are written into `recall-summary.json` and `recall-summary.md`. Each is an event Gut Health Times has already reported on. All six pass at this pull. They exist so that a future quarterly rebuild fails visibly if the normalisation drifts. **An outbreak is not a recall.** CDC and FDA open outbreak investigations that never produce a recall — no firm, no product, nothing to recall — and firms issue recalls where no outbreak was ever declared. Coverage of an outbreak will not necessarily have a row here, and that is the expected behaviour rather than a gap in the data. Two further mismatches to expect: our coverage date is the date the story broke, while the tracker's date is the Enforcement Report publication date, weeks later; and one story about "twin" recalls may be one FDA event carrying two records, as the Midwest Poultry brown-egg and white-egg recalls are. --- ## 7. Limitations Read these before citing any number. 1. **A recall count is not an illness count.** Nothing in this dataset measures how many people got sick, or whether anybody did. Most recalls in here are precautionary — the product tested positive, or a supplier's ingredient was recalled upstream — and were issued before any illness was reported. 2. **A recall count is not a safety rate.** There is no denominator. The volume of food produced, the number of establishments, the number of lots tested and the number of samples pulled are all absent. A year with more recalls is a year with more recall actions, and that is the whole of what it says. 3. **Recall counts track detection as much as contamination.** Whole-genome sequencing, routine environmental monitoring and more industry self-testing all raise the number of recalls without any change in how much contaminated food exists. Some of the movement in this table is the food-safety system working better. 4. **Reporting practice changes, and it shows in the data.** The FSIS series drops from 131 notices in 2019 to 50 in 2020 and stays at that lower level. That is a discontinuity in the record, and it is not explained inside this dataset. Do not narrate it as a change in food safety. 5. **The two agencies are genuinely different datasets.** Different schemas, different units, different classification processes, different scopes (FSIS: meat, poultry, processed egg products; FDA: essentially everything else, including shell eggs). Class I from one agency and Class I from the other are similar in definition and are not assigned by the same people or process. They are reported side by side in this tracker, never silently summed on a row basis. 6. **2026 is partial and lopsided.** FDA through 2026-09-16, FSIS through 2026-07-23. Do not compare 2026 to a full year, and do not compute an FDA-versus-USDA ratio for 2026. 7. **The most recent months are incomplete even within the FDA window**, because of the initiation-to-publication lag described in section 3. Expect the last quarter of any pull to grow. 8. **A single FDA event can carry records of more than one class**, so the classification table's columns run slightly above the event totals elsewhere. 9. **Percentage changes on small bases are noise.** Clostridium, Cronobacter, Hepatitis A, Norovirus and Other pathogen are all in the single or low double digits per three-year block. Report them as counts. 10. **Pathogen is classified from free text written by firms and agencies, not from a laboratory field.** A recall whose reason says only "product may be adulterated" is invisible to the pathogen rules even if an organism was involved. The reverse is rarer but possible: a reason naming an organism the firm was testing for, in a recall issued for another cause. 11. **The FSIS records come from an archived capture of the official endpoint**, not a live call, for the reason in section 1. The bytes are the agency's own; the recency is not current. 12. **openFDA is explicit that its own data is unvalidated** and should not be used for medical decisions. This tracker inherits that. --- ## 8. Rebuilding, quarterly ```bash cd wp-build/data python3 build-recall-tracker.py --refresh # re-pull both APIs python3 build-recall-tracker.py # rebuild from the cached pull ``` `--refresh` re-pulls everything and overwrites `recall-raw/`. Without it the build runs from the cached payloads, so the outputs are reproducible offline and a formatting change can be made without touching the network. On each quarterly run: 1. Check which FSIS route was taken (`recall-raw/fsis-provenance.json`). If it says `live`, the CDN block has lifted and the USDA coverage gap is closed. 2. Check that all six sanity checks still pass. 3. Check that the unclassified percentages have not moved sharply. 4. Update `COMPLETE_YEARS` in the script when a new year completes, so trend comparisons stop using a partial year. --- ## 9. Field dictionary — `recall-tracker.csv` | Column | Meaning | |---|---| | `source_agency` | `FDA` or `USDA-FSIS` | | `recall_number` | Agency recall number. FSIS rows are prefixed `FSIS-`. | | `event_id` | Groups records into one recall event. Collapse on this for event counts. | | `firm` | Recalling firm or establishment | | `firm_state` | Firm's own state (FDA only) | | `date_initiated` | Recall initiation date (FDA); notice date (FSIS) | | `date_reported` | Enforcement Report publication date (FDA); notice date (FSIS) | | `year_reported`, `month_reported` | Derived from `date_reported` | | `classification` | `I`, `II`, `III`, or empty for FSIS public health alerts | | `classification_raw` | Source string, including FSIS risk level | | `notice_type` | `Recall`, or FSIS `Public Health Alert` / `Closed Recall` / `Active Recall` | | `status` | Source status (Ongoing, Terminated, Completed, Active, Closed) | | `pathogen` | Normalised primary pathogen, empty when none named | | `pathogens_all` | Pipe-separated, all organisms matched | | `gi_pathogen_recall` | 1 when a pathogen was named. **The headline filter.** | | `contaminant` | Non-pathogen contaminant class; empty when a pathogen was named | | `undeclared_allergen` | 1/0. **Not a gut-pathogen recall — filter to 0 to exclude.** | | `allergens_named` | Pipe-separated allergens, or `unspecified` | | `product_category` | One of 22 normalised categories, or `Unclassified` | | `distribution_scope` | Nationwide / Multistate / Single state / International only / Unspecified | | `states_count` | Distinct US states resolved; empty for nationwide | | `states_list` | Pipe-separated two-letter codes | | `states_confidence` | `high` / `low` / `none`. **Filter to `high` for any states statistic.** | | `product_description` | Source text, truncated to 600 characters | | `reason_text` | Source reason text, truncated to 600 characters | | `source_url` | FSIS notice URL where available |