← Back to the homepage

Methodology

How items are sourced, ordered, labeled and corrected, and what is and is not in place today, followed by an original practical guide.

What this page covers. It explains how the service works today. Backprop Bulletin collects permitted publisher RSS feeds and research metadata, categorizes the items and shows each story once, with the publisher's own feed text attributed to its publisher. Backprop Bulletin adds no commentary to a card. The separately labeled original practical guide at the end of this page is general advice prepared with AI assistance; it is not a listing or a story about a listed item.

Where things stand today

The reader is updated automatically. A scheduled job checks the approved feeds every 15 minutes, following each source's own refresh interval, and adds new items to persistent storage, so earlier stories stay listed while new ones arrive. Each story is listed for up to seven days after the publication date its source supplies and is then removed, in line with the retention limits that apply to it.

Each card identifies the publisher, shows the publication date the source supplies and links to the original. No person checks an item against the original before it is listed. No advertising is shown, and the site has no accounts, comments or forms.

Sources

Items come from publisher RSS feeds and research metadata whose terms permit this use, such as Tech Xplore's news feeds, arXiv's preprint listings and government news feeds. Only the fields a source's terms allow are used: the headline, the link, the publication date, the author where supplied, and the feed's own description or abstract.

Publisher feed text is attributed to its publisher, with any credit or license statement the source requires. Tech Xplore headlines, links and descriptions are shown exactly as its feed supplies them, unchanged and in full, credited to “TechXplore.com”; an item that could not be shown unchanged is left out rather than altered. For other sources, a sentence of the feed text is used as their terms allow, and for research preprints a sentence of the abstract is adapted to name the authors.

A publisher's own image is shown only where that publisher's terms permit its reuse and the image passes automated checks of its source, type and size, with any credit the image requires. Otherwise the card shows a branded category illustration made for this site, described as such in its alternative text; it illustrates the topic, not the reported event. An image being reachable online is not treated as permission to reuse it.

Using a source is a decision that its terms permit this use, not a judgment that its content is correct. A source may change or remove items after they are collected, and a card may lag behind the source.

Kinds of source

This is the project's vocabulary for kinds of source. The current reader does not display these labels on listings; research preprints are labeled separately, as described under Claim status. We do not check whether an outlet is truly independent of the subject it covers.

Claim status

A claim status would describe what kind of statement an item makes. It does not say the item is true. The reader labels research preprints “Research preprint”, matching “Preprint — not peer reviewed” below; it does not display the other claim statuses or any verification badge.

Review and verification

Research preprints are labeled “Research preprint” and kept distinct from news: while other stories are available, at most eight of the forty stories on the front page are preprints. A preprint has not been peer reviewed, and its inclusion does not mean its findings have been independently verified.

Automated checks cover how content is handled and presented: that headlines, links, dates and required credits are carried through, that Tech Xplore text is unchanged, that each card names its publisher once, and that pages, filters and search work. They do not independently substantiate every publisher claim or reproduce research results, and the reader shows no “verified” badge.

Order and categories

The front page shows a lead story, top stories and the latest stories, newest source-supplied date first, mixing categories so that one topic does not fill the page. The order uses no reader data, no advertising and no commercial relationship, and it is not a measure of importance, novelty or whether a story is breaking.

Each item is placed in one of eight categories (Models, Agents, Tools, Business, Infrastructure, Robotics, Research and Policy) by automatic keyword rules within the topics its source covers. Categories are unverified annotations and can be wrong.

Duplicate stories are suppressed. An item listed by several feeds is shown once, and closely matching headlines from different publishers about the same event, published within three days of each other, are shown as one card that lists the other publishers under “Also reported by”. This matching is automatic and conservative, and it does not show whether outlets reported independently or who copied whom.

Corrections

Cards show what the source supplied when it was collected. If a publisher changes an item, a later update reflects the change. A story is removed when its listing period ends, and it is not added back if the source lists it again.

If you notice an error in how an item is listed, use the Contact Us section of the About and Contact page. No response time is promised.

Advertising

No advertising is shown. If advertising or sponsorship is added, it will be kept separate from editorial choices, will not affect which items are listed or how they are ordered, and will be labeled as advertising.

Attribution and linking

Each card names the publisher and links to the original article, and the service does not copy a publisher's full article. Headlines, feed text and links belong to their publishers, and a listing does not mean a publisher endorses or is affiliated with this site. This page describes how the service works; it is not a statement about copyright or legal compliance, and rights questions can be sent through Contact Us.

Research preprint metadata comes from arXiv. Thank you to arXiv for use of its open access interoperability.

Original practical guide: evaluating an AI agent claim

About this guide. This is general advice written for this site and prepared with AI assistance. It is not a listing, it is not about any item the reader lists, and it has not had editorial review. Its examples are illustrative.

A polished video of an AI agent finishing a task shows that the task can be finished at least once. It does not show how often the agent fails, how much help it needed, or what happens when something goes wrong partway through. These steps turn an impressive showcase into a question you can answer for your own work.

Start with your own task

Write down one task you repeat often enough to care about, as specifically as you can: for example, writing a reply to an invoice dispute using your own policy document, rather than handling email in general. Before you try anything, write an acceptance criterion, meaning what a result must contain or avoid for you to use it without rewriting it, and a baseline: how long the task takes a person today and how often that work needs correcting. Without both, faster and better have nothing to be measured against.

Count every attempt, not only the best one

A showcase is usually the best attempt, chosen afterward. What matters for your work is how often it succeeds. Run the same task several times on realistic inputs, and keep a tally of attempts started, attempts that met your criterion with no help, attempts that met it only after you stepped in, and attempts that failed or were abandoned. Count each intervention: if you had to nudge the agent along, that run was not autonomous, and your time is part of its cost.

Know what the agent could touch

Find out which tools and permissions it had, such as reading only, or also writing, sending, deleting or buying; what environment it ran in; and what was already set up for it, such as signed-in sessions or a hand-picked folder of examples. If that environment does not resemble yours, treat the result as a hypothesis rather than evidence. Grant the narrowest access that still lets the task be done, and widen it only after the agent behaves sensibly within it.

Ask what happens after an interruption

Real work gets interrupted. Check whether the agent resumes from a known point or starts again and repeats steps that already happened. The hardest case is an uncertain outcome: the agent pressed send or submit, then lost its connection before seeing a confirmation. A careful system checks whether the first action took effect before trying again, or stops and asks you. Keep an agent that simply repeats the action away from anything that cannot be undone, such as a payment or a sent message.

Record what you saw, and state its limits

For each trial, note the date, the input, the version or settings if they are visible, the tools it had, the outcome against your criterion, every intervention, and the time spent, including your own. This makes your conclusion checkable and shows whether behavior changes after an update. A handful of trials is a sanity check, not a benchmark: later inputs may be easier or harder, results can shift when the underlying model, tool or instructions change, and figures a vendor publishes were measured on its tasks, not yours. State your conclusion no more strongly than your own trials support.

Checklist before you rely on an agent