{ "entity": "gpt-5-5", "entity_type": "model", "category": "Research", "suite": "suite-2026-08-tasks4/research", "model": "openai/gpt-5.5", "judges": [ "anthropic/claude-sonnet-5", "google/gemini-3.1-pro-preview", "x-ai/grok-4.5" ], "judge_protocol": "judge-2026-08b", "anchors": "solo-2026-08-a", "score": 97, "caps": 0, "cost_usd": 0.4039, "started_at": "2026-08-16T14:53:15.113Z", "finished_at": "2026-08-16T14:55:08.730Z", "tasks": [ { "id": "rs1", "name": "Make it answerable", "prompt": "'Is coffee bad for you?' is not a researchable question. Turn it into exactly 3 properly researchable questions, numbered 1-3, one line each, no sub-points — each must specify who (the population), what (the exposure and amount), and which outcome is being measured.", "output": "1. Among adults aged 40–65, does drinking 3–4 cups of caffeinated coffee per day affect the incidence of hypertension over 5 years?\n\n2. Among pregnant adults, does consuming more than 200 mg of caffeine from coffee per day affect the risk of preterm birth?\n\n3. Among adults with gastroesophageal reflux disease, does drinking 2 or more cups of coffee per day affect the frequency of reflux symptoms per week?", "latency_ms": 2924, "cost": 0.00499, "task_score": 10, "judges": [ { "judge": "anthropic/claude-sonnet-5", "score": 9, "violates": false, "reasoning": "Three well-formed numbered questions, each specifying population, exposure/amount, and outcome, matching format exactly with no sub-points." }, { "judge": "google/gemini-3.1-pro-preview", "score": 10, "violates": false, "reasoning": "The response perfectly follows all instructions, providing exactly three numbered, single-line researchable questions that clearly specify the population, exposure/amount, and outcome without any sub-points." }, { "judge": "x-ai/grok-4.5", "score": 10, "violates": false, "reasoning": "Exact match to all specs: 3 numbered one-line questions each with population, exposure/amount, and outcome." } ], "machine_checks": { "exact_bullets": 3 }, "machine_violation": false, "machine_reasons": [] }, { "id": "rs2", "name": "Rank the sources", "prompt": "You are researching whether standing desks reduce back pain. Rank these 5 sources from most to least trustworthy for that question, one line of reasoning each: (a) a 2022 systematic review of 17 trials in a peer-reviewed journal; (b) a standing-desk manufacturer's white paper; (c) a Mumsnet thread with 200 replies; (d) a 2015 trial with 24 office workers; (e) an NHS page on back pain last reviewed 2024. Then say which ONE you would cite to a sceptical reader and why.", "output": "Most to least trustworthy for “do standing desks reduce back pain?”:\n\n1. **(a) 2022 systematic review of 17 trials in a peer-reviewed journal** — Best evidence because it synthesizes multiple trials and is directly about effectiveness, reducing reliance on any one small study. \n2. **(d) 2015 trial with 24 office workers** — Direct experimental evidence, but small, older, and likely underpowered/generalizability-limited. \n3. **(e) NHS page on back pain last reviewed 2024** — Authoritative and current, but likely gives general back-pain advice rather than directly testing standing desks. \n4. **(b) Standing-desk manufacturer’s white paper** — Relevant but commercially conflicted, so at high risk of selective evidence and marketing spin. \n5. **(c) Mumsnet thread with 200 replies** — Useful for anecdotes and lived experience, but uncontrolled, self-selected, and not reliable for causation.\n\n**One source I’d cite to a sceptical reader:** **(a), the 2022 systematic review**, because it is peer-reviewed, directly addresses the intervention, and summarizes evidence across 17 trials rather than relying on anecdotes, marketing material, or a single small study.", "latency_ms": 10815, "cost": 0.023625, "task_score": 9, "judges": [ { "judge": "anthropic/claude-sonnet-5", "score": 9, "violates": false, "reasoning": "Accurate, well-reasoned ranking with clear justification; follows instructions fully; concise and useful for a non-technical reader." }, { "judge": "google/gemini-3.1-pro-preview", "score": 9, "violates": false, "reasoning": "The response accurately ranks the sources with sound, concise reasoning for each. The choice of the systematic review for a skeptical reader is perfect. The ranking of the small trial over the NHS page is debatable but well-justified." }, { "judge": "x-ai/grok-4.5", "score": 9, "violates": false, "reasoning": "Accurate ranking by evidence hierarchy; clear one-line reasons; correctly picks systematic review to cite. Minor polish only." } ], "machine_checks": null, "machine_violation": false, "machine_reasons": [] }, { "id": "rs3", "name": "Synthesise the disagreement", "prompt": "Three studies, summarised: Study A (2021, 12,000 adults followed 5 years) — people sleeping under 6 hours had 20% higher rates of catching colds. Study B (2019, 300 students, one term) — no significant link between sleep and illness days. Study C (2023, review of 40 studies) — consistent link between short sleep and infection risk, but most studies could not rule out that stress causes both. Write a summary of max 120 words of what we actually know, giving the disagreement and the confounder their honest weight.", "output": "Overall, the evidence suggests short sleep is associated with a higher risk of infections such as colds, but it does not prove that lack of sleep directly causes them. A large 2021 study found adults sleeping under 6 hours had about 20% higher cold rates over 5 years, and a 2023 review found a broadly consistent link across 40 studies. However, a smaller 2019 student study found no significant relationship, showing the effect may vary by population, measurement, or study power. A major caveat is confounding: stress could plausibly reduce sleep and increase infection risk, and many studies could not fully rule this out. So, short sleep is a likely risk marker, but causality remains uncertain.", "latency_ms": 4003, "cost": 0.007605, "task_score": 9, "judges": [ { "judge": "anthropic/claude-sonnet-5", "score": 8, "violates": false, "reasoning": "Accurate, balances disagreement and confounder honestly, concise, clear; within word limit (~120 words), well-suited for non-technical reader." }, { "judge": "google/gemini-3.1-pro-preview", "score": 10, "violates": false, "reasoning": "The response perfectly synthesizes the three studies within the 120-word limit (119 words). It accurately highlights the general consensus, acknowledges the conflicting study, and clearly explains the stress confounder, making it highly use" }, { "judge": "x-ai/grok-4.5", "score": 9, "violates": false, "reasoning": "Accurate synthesis giving studies, disagreement and confounder proper weight; 118 words, clear and useful." } ], "machine_checks": { "max_words": 120 }, "machine_violation": false, "machine_reasons": [] }, { "id": "rs4", "name": "Tear down the survey", "prompt": "A press release claims: 'Study proves 92% of Britons love exercising. Researchers surveyed 40 members of a Shoreditch gym, recruited through the gym's Instagram page.' List the 3 biggest methodological problems as bullets, one line each, most fatal first. No sub-points.", "output": "- Grossly unrepresentative sample: 40 Shoreditch gym members cannot stand for all Britons.\n- Severe self-selection bias: Instagram-recruited gym followers are especially likely to like exercise.\n- Tiny sample size: 40 respondents gives high uncertainty and makes the 92% claim unstable.", "latency_ms": 4702, "cost": 0.005725, "task_score": 10, "judges": [ { "judge": "anthropic/claude-sonnet-5", "score": 9, "violates": false, "reasoning": "Accurate, correctly prioritized, follows format (3 bullets, one line each, no sub-points), concise and useful." }, { "judge": "google/gemini-3.1-pro-preview", "score": 10, "violates": false, "reasoning": "The response perfectly follows all instructions, identifying the three major methodological flaws in order of severity, using single-line bullets with no sub-points." }, { "judge": "x-ai/grok-4.5", "score": 10, "violates": false, "reasoning": "Perfectly identifies top 3 flaws in priority order, exact bullet format, accurate and concise." } ], "machine_checks": { "exact_bullets": 3 }, "machine_violation": false, "machine_reasons": [] }, { "id": "rs5", "name": "Correlation for the neighbour", "prompt": "A headline reads: 'People who drink red wine live longer, study finds.' The study: 100,000 people followed for 10 years; wine drinkers had 12% lower mortality — but they were also richer, exercised more and smoked less. Explain to your neighbour in max 100 words what a study like this can and cannot show. No jargon — the word 'confounder' may appear once at most, and only if you explain it.", "output": "This kind of study can show that people who drank red wine tended to die less often over 10 years. But it can’t prove the wine caused longer life. Wine drinkers were also richer, exercised more, and smoked less, so those factors may explain the difference. A “confounder” is something mixed in that affects the result. Researchers can try to account for these differences, but they can’t be sure they caught everything. To test cause, you’d need a different kind of study.", "latency_ms": 3743, "cost": 0.00482, "task_score": 9, "judges": [ { "judge": "anthropic/claude-sonnet-5", "score": 8, "violates": false, "reasoning": "Accurate, clear, no jargon overuse, confounder explained once, concise, under 100 words, useful for non-technical reader." }, { "judge": "google/gemini-3.1-pro-preview", "score": 10, "violates": false, "reasoning": "The response perfectly explains correlation versus causation in plain language, accurately uses the study details, adheres to the word limit, and explains 'confounder' simply as instructed." }, { "judge": "x-ai/grok-4.5", "score": 9, "violates": false, "reasoning": "Accurate explanation of association vs causation; meets word limit and jargon rule; clear and useful for lay audience." } ], "machine_checks": { "max_words": 100 }, "machine_violation": false, "machine_reasons": [] }, { "id": "rs6", "name": "Plan the dig", "prompt": "Someone wants to trace the history of their 1930s semi in Leicester — who built it, who has lived there, what stood on the land before. Lay out the kinds of sources and places in the UK that answer each part (deeds, census-type records, old maps, local archives, newspaper archives), what each one can and cannot tell them, and the sensible order to work in. No URLs — the shape of the search, not links.", "output": "Below is a practical “shape of the search” for a 1930s semi-detached house in Leicester: who built it, who lived there, and what was on the land before. The key is to work from the known present address backwards, while allowing for street renumbering, ward/parish boundary changes, and estate-name changes.\n\n## 1. Start by fixing the identity of the house\n\nBefore chasing records, establish:\n\n- Current full address and postcode.\n- Title number, if registered.\n- Whether the street has ever been renamed or renumbered.\n- Whether the house was originally part of a named estate or development.\n- Whether the property is in Leicester city proper or a surrounding parish/suburb now absorbed into the city.\n\nUseful first sources:\n\n### HM Land Registry title register and title plan\n**Can tell you:**\n- Current registered owner.\n- Date of the current owner’s purchase.\n- Whether there are covenants, rights of way, estate restrictions, shared drives, drainage rights, etc.\n- Sometimes references to older conveyances, such as “Conveyance dated 12 June 1934 between X and Y”.\n\n**Cannot usually tell you:**\n- A full list of every former owner.\n- The builder’s name, unless preserved in older deeds or covenants.\n- Occupiers, tenants or lodgers.\n- What stood on the land before the house.\n\n### Your own deed packet / conveyancing papers\nIf the house has been owned for a long time, or if previous owners passed on old papers, there may be pre-registration deeds.\n\n**Can tell you:**\n- Original plot sale.\n- Name of developer, estate company, builder or first purchaser.\n- Date the plot was conveyed.\n- Restrictive covenants: building line, cost of house, materials, “no trade”, “one dwelling only”, etc.\n- Plans showing the original estate layout.\n\n**Cannot always tell you:**\n- Who actually built the house if a developer sold plots to small builders.\n- Who occupied the property if the owner rented it out.\n- Complete ownership after compulsory Land Registration simplified the title.\n\n**Tip:** Modern registered titles often omit older deeds unless they still affect the title. If the register mentions a filed conveyance or transfer, ask for the filed document.\n\n---\n\n## 2. Finding who built the house\n\nFor a 1930s semi, the builder may have been:\n\n- A speculative local builder.\n- A housing estate company.\n- A landowner selling plots to builders.\n- A council or public authority, if it was municipal housing.\n- A private individual commissioning a house.\n\nThe evidence is usually scattered.\n\n### Deeds and conveyances\n**Can tell you:**\n- Name of land seller and buyer.\n- Whether the buyer was a builder, developer or private purchaser.\n- Date the plot was sold.\n- Estate conditions and building restrictions.\n- Sometimes a plan of the plot.\n\n**Cannot always tell you:**\n- Whether the named buyer personally built the house.\n- Exact completion date.\n- Architect’s name.\n\nLook especially for phrases such as:\n- “builder”\n- “contractor”\n- “estate company”\n- “building society mortgage”\n- “plot number”\n- “building estate”\n- “subject to the stipulations on the estate plan”\n\n### Local authority building control records and deposited plans\nIn the 1930s, new houses often required deposited plans under local building byelaws.\n\n**Likely places:**\n- Leicester City Council building control/planning records.\n- The local record office for Leicester and Leicestershire.\n- Sometimes council archives rather than the current planning department.\n\n**Can tell you:**\n- Date plans were submitted.\n- Name of applicant.\n- Name of builder, owner or architect.\n- Site plan, elevations, drainage layout.\n- Whether plans were approved or amended.\n\n**Cannot always tell you:**\n- Whether the house was built exactly as approved.\n- Later internal changes if records have not survived.\n- Occupants.\n\n**Limitations:**\n- Survival is patchy.\n- Some records are arranged by street, some by date, some by application number.\n- Access to later building control files may be restricted.\n\n### Planning records\nFormal town planning was developing in the interwar period, but for ordinary 1930s houses the key records may be byelaw/building plans rather than modern-style planning applications.\n\n**Can tell you:**\n- Estate layout approvals.\n- Road adoption, drainage and building lines.\n- Later extensions and alterations.\n\n**Cannot usually tell you:**\n- Full ownership history.\n- Names of tenants.\n\n### Council minutes and committee records\nLook at highways, housing, planning, estates, public health, building plans or improvement committee minutes.\n\n**Can tell you:**\n- Approval of new roads.\n- Adoption of sewers and streets.\n- Permission for estate development.\n- Discussions with developers.\n- Street naming and numbering.\n\n**Cannot usually tell you:**\n- Detailed house-by-house occupancy.\n- Builder for every individual semi unless the minute books are very detailed.\n\n### Newspapers\nLocal newspapers are very useful for interwar estates.\n\n**Can tell you:**\n- Advertisements for new houses.\n- Name of builder/developer/estate agent.\n- Sale prices and mortgage terms.\n- “Modern semi-detached villas” descriptions.\n- Auctions of land before development.\n- Reports on new roads and housing estates.\n\n**Cannot always tell you:**\n- The exact house unless the advert gives plot numbers or street names.\n- Whether your particular half of the semi was sold or rented.\n- Full construction details.\n\nSearch terms might include:\n- Street name.\n- Estate name.\n- Nearby road names.\n- Builder surnames found in deeds.\n- “building estate”\n- “semi-detached”\n- “Leicester villas”\n- “freehold plots”\n- “new houses”\n- “auction”\n- “building land”\n\n### Trade directories and local directories\nExamples include Kelly’s, Wright’s, local Leicester directories, and similar annual or periodic directories.\n\n**Can tell you:**\n- When the street first appears.\n- Names of householders at addresses.\n- Local builders operating nearby.\n- Sometimes estate agents, builders and developers advertising.\n\n**Cannot tell you:**\n- All residents; poorer households and tenants are often underrepresented.\n- Children, lodgers or servants.\n- Exact build date, though first appearance helps bracket it.\n\n---\n\n## 3. Finding who lived there\n\nFor a house built in the 1930s, you cannot yet use ordinary census returns for most of its occupied life. The useful sources are electoral registers, directories, the 1939 Register, rate books and newspapers.\n\n### Electoral registers\nThese are among the best sources for a house-by-house adult occupancy timeline.\n\n**Likely places:**\n- Leicester local studies library.\n- Local record office.\n- City archives.\n- Main reference libraries.\n- Some years may be indexed or digitised, but many are only in bound volumes or microfilm.\n\n**Can tell you:**\n- Adult voters at the address.\n- Year-by-year changes.\n- Sometimes absent voters, service voters or qualifying details.\n- Useful for tracking owners/occupiers from the 1930s onward.\n\n**Cannot tell you:**\n- Children.\n- Non-voters.\n- Some women before equal franchise settled fully in 1928, though for a 1930s house adult women should generally appear if registered.\n- Exact move-in/move-out dates; only registration snapshots.\n- Tenure: owner or tenant.\n\n**Limitations:**\n- Registers are arranged by polling district and street, so you need the correct ward/district.\n- Street renumbering can cause confusion.\n- Wartime and immediate post-war coverage may have gaps or altered formats.\n- Modern registers have privacy restrictions.\n\n### 1939 Register\nThis is the nearest substitute for a 1941 census, which never happened.\n\n**Can tell you:**\n- Occupants on the national registration date in September 1939.\n- Full names.\n- Dates of birth.\n- Marital status.\n- Occupations.\n- Later married surnames for some women, added later in NHS use.\n\n**Cannot tell you:**\n- People who moved in after September 1939.\n- Closed records for people not proven deceased.\n- Family relationships explicitly, although they can often be inferred.\n- Ownership.\n\nFor a 1930s Leicester semi, this is often the single best snapshot of early occupation.\n\n### Census returns\nThe ordinary census is less helpful for a 1930s-built house because of closure periods.\n\n**Available:**\n- 1921 Census is the latest generally available census for England and Wales.\n- 1911 and earlier censuses are available.\n\n**Not available/useful for this house’s occupancy:**\n- 1931 Census for England and Wales was destroyed by fire.\n- No 1941 Census was taken.\n- 1951 and later censuses remain closed under the usual hundred-year rule.\n\n**Can tell you:**\n- Who lived on the land before the house if there were earlier cottages, farms or institutions.\n- Occupants of nearby roads before development.\n- Agricultural labourers, farmers, landowners, etc.\n- Household relationships and birthplaces.\n\n**Cannot tell you:**\n- Who lived in the 1930s semi after it was built, unless it existed by 1921, which is unlikely for a 1930s house.\n\n### Rate books\nLocal rates were levied on occupiers and/or owners.\n\n**Can tell you:**\n- Occupier.\n- Owner.\n- Property description.\n- Rateable value.\n- Sometimes first appearance of the new house.\n- Changes from vacant land to building plots to completed houses.\n\n**Cannot always tell you:**\n- Every resident in a household.\n- Exact construction date.\n- Clear distinction between owner-occupier and landlord unless columns are well kept.\n\n**Why they are valuable:**\nFor interwar houses, rate books can sometimes identify the first occupier before directories or electoral registers catch up.\n\n### Street directories\n**Can tell you:**\n- Named householder at an address.\n- Approximate first occupation date.\n- Sometimes house names before numbering became fixed.\n\n**Cannot tell you:**\n- Full household.\n- Tenants not considered principal householders.\n- Women or working-class tenants consistently.\n- Complete annual continuity if directories were not published every year.\n\n### Newspapers for residents\nOnce names are found, local newspapers can add colour.\n\n**Can tell you:**\n- Births, marriages and deaths.\n- Obituaries.\n- Court reports.\n- Military service.\n- Accidents.\n- Business advertisements.\n- House sale notices.\n- Social clubs, schools, church events.\n\n**Cannot tell you:**\n- A complete unbiased account of ordinary lives.\n- Confirmation that a same-named person is definitely your resident unless the address or other details match.\n\n### Probate calendars and wills\nUseful for owners and long-term residents.\n\n**Can tell you:**\n- Date of death.\n- Address at death.\n- Executor names.\n- Estate value.\n- Sometimes family connections if a will is obtained.\n\n**Cannot tell you:**\n- All occupants.\n- Tenancy history.\n- Whether the deceased built the house.\n\n### Birth, marriage and death indexes\nUseful once resident names are known.\n\n**Can tell you:**\n- Family structure.\n- Maiden names.\n- Life dates.\n- Possible movement in and out of Leicester.\n\n**Cannot tell you:**\n- Address in the index alone.\n- Whether someone owned or merely occupied the house.\n\n---\n\n## 4. Finding what stood on the land before\n\nThis part is mostly map-led, then checked against census, valuation, estate and newspaper records.\n\n### Ordnance Survey maps\nUse successive large-scale OS maps, ideally 25-inch or 6-inch series.\n\n**Can tell you:**\n- Whether the site was fields, allotments, nursery land, brickworks, railway land, farm buildings, cottages, etc.\n- Field boundaries.\n- Old lanes, footpaths, streams and drains.\n- When the new street appears.\n- Surrounding development phases.\n\n**Cannot tell you:**\n- Ownership.\n- Occupiers.\n- Field names unless shown.\n- Exact build date between survey/revision dates.\n\nFor a 1930s house, compare:\n- Late Victorian OS.\n- Edwardian OS.\n- 1920s revision.\n- 1930s/1940s edition.\n- Post-war editions.\n\nThe house may appear on a map published after construction but surveyed earlier or later, so always note survey/revision dates, not only publication dates.\n\n### Tithe maps and apportionments\nRelevant if the land was still rural or semi-rural in the mid-19th century.\n\n**Can tell you:**\n- Landowner.\n- Occupier.\n- Field names.\n- Land use: arable, meadow, pasture, homestead, etc.\n- Plot boundaries.\n\n**Cannot tell you:**\n- 20th-century development details.\n- Later changes unless linked with other records.\n\n### Enclosure maps and awards\nWhere applicable, these can take the land history back before the tithe survey.\n\n**Can tell you:**\n- Older field layout.\n- Allotments of land to owners after enclosure.\n- Roads, footpaths and public rights.\n- Sometimes old common fields.\n\n**Cannot tell you:**\n- Modern house history.\n- Ordinary tenants after enclosure unless named in the award.\n\n### 1910 Finance Act / Valuation Office records\nThe “Domesday” valuation around 1910 can be very useful if the land was still undeveloped.\n\n**Can tell you:**\n- Owner and occupier around 1910.\n- Description of property.\n- Acreage.\n- Value.\n- Sometimes whether land was agricultural, building land, garden, house, shop, etc.\n- Map reference linking the property to a numbered hereditament.\n\n**Cannot tell you:**\n- 1930s builder unless the same owner later developed the land.\n- Complete later ownership.\n- Household members.\n\n### Estate maps, sale particulars and auction catalogues\nThese are especially useful if farmland was sold for interwar housing.\n\n**Likely places:**\n- Local record office.\n- Local studies library.\n- Estate agent or auctioneer archives if they survive.\n- Solicitors’ collections deposited in archives.\n- Newspaper advertisements.\n\n**Can tell you:**\n- Name of estate or farm sold.\n- Vendor.\n- Purchasers.\n- Lot numbers.\n- Acreage.\n- Plans of proposed roads or plots.\n- Conditions of sale.\n\n**Cannot always tell you:**\n- Who built each house after the land was broken into plots.\n- Later individual occupants.\n\n### Aerial photographs\nUseful from the 1920s/1930s onward, and especially post-war.\n\n**Can tell you:**\n- Whether the street was laid out by a particular date.\n- Construction phases.\n- Former field patterns visible as cropmarks or boundaries.\n- Nearby industrial or agricultural uses.\n\n**Cannot tell you:**\n- Names of owners or residents.\n- Exact dates unless the photo is well dated.\n\n### Geological, drainage and utilities records\nOccasionally useful where land was formerly marshy, quarried, mined, crossed by drains, or near industry.\n\n**Can tell you:**\n- Former pits, clay extraction, watercourses, culverts, sewers.\n- Why a road or estate was laid out as it was.\n\n**Cannot usually tell you:**\n- Householder names.\n- Builder identity.\n\n---\n\n## 5. Leicester-specific places to think about\n\nFor a Leicester house, the main categories of repositories are:\n\n### Local record office for Leicester and Leicestershire\nThis is likely to hold:\n- Parish records.\n- Estate papers.\n- Tithe and enclosure material.\n- Deposited maps.\n- Some local authority records.\n- Solicitors’ collections.\n- Sale catalogues.\n- Electoral registers or copies.\n- Rate books, depending on survival.\n\n### Leicester local studies library / city reference collections\nLikely to hold:\n- Street directories.\n- Electoral registers.\n- Local newspapers on film or digital access.\n- Local history books.\n- Maps.\n- Photographs.\n- Street name histories.\n- Cuttings files.\n\n### Leicester City Council archives / planning / building control\nPotentially hold:\n- Building plans.\n- Planning applications.\n- Street adoption records.\n- Road naming and numbering records.\n- Council committee minutes.\n- Housing or estates department records.\n\n### The National Archives and national-level collections\nUseful for:\n- 1910 valuation records.\n- Some maps and government records.\n- Military records, if researching occupants.\n- Probate calendars and related national indexes.\n- Census and 1939 Register access through commercial or institutional services.\n\n### County or regional newspaper collections\nUseful for:\n- Leicester newspapers.\n- Estate advertisements.\n- Property auctions.\n- Builder advertisements.\n- Reports on new suburban development.\n\n---\n\n## 6. Sensible order of work\n\nA good sequence is:\n\n### Step 1: Gather what you already have\nCollect:\n- Current title register and title plan.\n- Mortgage/conveyancing papers.\n- Any old deeds, abstracts, plans or covenants.\n- Old photographs, house names, numbers on gates, drain covers, builder’s marks.\n- Information from neighbours, especially long-term residents.\n\nAim: establish title number, plot boundaries, old plot number, any names of developers or landowners.\n\n### Step 2: Check for old deeds and filed documents\nLook for:\n- Original conveyance of the plot.\n- Estate plan.\n- Restrictive covenants.\n- Names of vendors, purchasers, builders or estate companies.\n- Dates between land sale and house occupation.\n\nAim: identify the development context and possible builder.\n\n### Step 3: Confirm approximate build date\nUse:\n- Building control/deposited plans.\n- Planning/byelaw records.\n- Rate books.\n- Street directories.\n- Electoral registers.\n- OS maps.\n\nAim: narrow the house’s construction and first occupation to a year or two.\n\nFor example:\n- Field on 1929 OS revision.\n- Building plans approved 1933.\n- First rated as house in 1934.\n- First named occupier in 1935 directory.\n- Occupants in 1939 Register.\n\nThat would give a solid working build date of 1933–34.\n\n### Step 4: Build an occupancy timeline\nWork through:\n- Electoral registers year by year.\n- Directories where available.\n- 1939 Register.\n- Rate books.\n- Probate records.\n- Newspaper searches for named residents.\n\nCreate a table:\n\n| Year | Source | Occupier/Owner | Notes |\n|---|---|---|---|\n| 1934 | Rate book | vacant/new house or first occupier | first rating |\n| 1935 | Directory | named householder | first directory appearance |\n| 1939 | 1939 Register | household members | dates of birth, occupations |\n| 1946 | Electoral register | adult voters | post-war return |\n| etc. | | | |\n\nAim: distinguish confirmed facts from probable identifications.\n\n### Step 5: Trace the land backwards\nOnce you know the plot location precisely, work backwards through:\n- OS maps.\n- 1910 valuation records.\n- Tithe map and apportionment.\n- Enclosure map/award if relevant.\n- Estate maps and sale catalogues.\n- Census returns for any earlier houses, farms or cottages on the same land.\n\nAim: identify whether the semi stands on former farmland, market garden, an estate, old village edge, industrial land, or demolished buildings.\n\n### Step 6: Use newspapers to connect the stories\nSearch newspapers after you have names and dates.\n\nLook for:\n- The land sale before development.\n- Developer or builder advertisements.\n- “New houses in [street/estate]”.\n- Reports of road adoption or naming.\n- Sale advertisements for the house.\n- Notices involving early residents.\n\nAim: add context and sometimes identify the builder when official records are thin.\n\n---\n\n## 7. What each question is best answered by\n\n### “Who built it?”\nBest sources:\n- Building control/deposited plans.\n- Original deeds and conveyances.\n- Estate sale particulars.\n- Newspaper advertisements.\n- Council committee minutes.\n- Trade directories for builders.\n\nLikely difficulties:\n- The developer may not be the actual builder.\n- Plans may name the owner but not the contractor.\n- Small builders may have left few records.\n\n### “When was it built?”\nBest sources:\n- Building plans/byelaw approvals.\n- Rate books.\n- First appearance in electoral registers/directories.\n- OS map revision dates.\n- Deed dates and mortgage dates.\n\nLikely difficulties:\n- Approval date is not completion date.\n- First directory appearance may lag by a year or more.\n- OS maps may be surveyed years before publication.\n\n### “Who lived there?”\nBest sources:\n- Electoral registers.\n- 1939 Register.\n- Street directories.\n- Rate books.\n- Newspapers.\n- Probate records.\n\nLikely difficulties:\n- Children and non-voters are often invisible.\n- Directories usually list only the principal householder.\n- Tenants may not appear in ownership records.\n- Modern data is restricted.\n\n### “Who owned it?”\nBest sources:\n- Land Registry.\n- Old deeds.\n- Rate books, if they include owner columns.\n- Probate.\n- Sale notices.\n- Mortgage/conveyance papers.\n\nLikely difficulties:\n- Registered title may not preserve a full chain.\n- Owner and occupier may differ.\n- Deed packets may have been destroyed or not passed on.\n\n### “What was there before?”\nBest sources:\n- OS maps.\n- Tithe and enclosure records.\n- 1910 valuation records.\n- Estate maps.\n- Sale catalogues.\n- Earlier census returns if buildings stood there.\n- Aerial photographs.\n\nLikely difficulties:\n- Exact plot matching can be tricky where roads and boundaries changed.\n- Field names and ownership may change over time.\n- Maps show physical features, not the full social history.\n\n---\n\n## 8. Common pitfalls\n\n- **Assuming publication date equals map survey date.** Always check revision/survey dates.\n- **Ignoring renumbering.** A 1930s “No. 12” might later become “No. 28”.\n- **Assuming the first named occupier was the owner.** Many interwar houses were rented.\n- **Assuming the land seller built the house.** Landowners often sold to developers or speculative builders.\n- **Relying only on the Land Registry.** It is essential, but usually not enough for house history.\n- **Missing ward/polling district changes.** Electoral registers may move the street between divisions.\n- **Confusing adjacent semis.** Always tie evidence to plot plans, house numbers and neighbours.\n\n---\n\n## 9. A strong final research trail might look like this\n\nFor a well-researched 1930s Leicester semi, you would ideally end up with:\n\n1. Land Registry title and any filed older conveyance.\n2. Original or early conveyance showing estate plot and covenants.\n3. Building plan approval naming applicant/builder/architect.\n4. Rate book entry showing first assessment as a dwelling.\n5. Directory/electoral register entries showing first occupants.\n6. 1939 Register household.\n7. Year-by-year electoral register timeline.\n8. OS map sequence showing the land before and after development.\n9. Tithe/enclosure/1910 valuation evidence for earlier land ownership and use.\n10. Newspaper advertisements or reports tying the development to a builder or estate.\n\nThat combination should answer, as far as surviving records allow: who built it, when it appeared, who lived there, who owned it, and what the site was before the 1930s suburb arrived.", "latency_ms": 88001, "cost": 0.159015, "task_score": 9, "judges": [ { "judge": "anthropic/claude-sonnet-5", "score": 8, "violates": false, "reasoning": "Thorough, accurate, well-organized coverage of UK sources with capabilities/limits and sensible order, no URLs as required. Slightly long/dense for a non-technical user but genuinely useful and correct." }, { "judge": "google/gemini-3.1-pro-preview", "score": 10, "violates": false, "reasoning": "The response perfectly addresses every prompt requirement, providing an expertly structured, highly accurate, and practical guide. It explicitly details what each source can and cannot tell the researcher, outlines a sensible workflow, and " }, { "judge": "x-ai/grok-4.5", "score": 9, "violates": false, "reasoning": "Comprehensive, accurate guide covering all required sources, limits, and order. Highly useful and clear; minor length excess only." } ], "machine_checks": null, "machine_violation": false, "machine_reasons": [] }, { "id": "rs7", "name": "What you cannot cite", "prompt": "'Give me the DOI, journal and page numbers for that 2021 sleep study you mentioned.' You have no way to verify those details. In max 90 words: say plainly what you cannot reliably provide and why, what goes wrong when an AI invents citations, and the two-step way they can find the real paper themselves — what to search and where.", "output": "I can’t reliably provide the DOI, journal, or page numbers because I don’t have the exact paper details or a way to verify the citation here. If I guessed, I might invent a plausible but false citation, which wastes time and can mislead readers.\n\nTwo steps to find it:\n1. Search the study topic plus “2021 sleep study” and any author/institution keywords.\n2. Check results in Google Scholar, PubMed, Crossref, or the journal’s site to confirm DOI, journal, volume, and pages.", "latency_ms": 3097, "cost": 0.00568, "task_score": 10, "judges": [ { "judge": "anthropic/claude-sonnet-5", "score": 8, "violates": false, "reasoning": "Accurate, honest, follows instructions, concise, within word limit, clear two-step guidance. Minor stylistic room for improvement but solid overall." }, { "judge": "google/gemini-3.1-pro-preview", "score": 10, "violates": false, "reasoning": "The response perfectly follows all instructions, including the 90-word limit (83 words). It clearly states what cannot be provided and why, the dangers of AI hallucinations, and gives a precise two-step search strategy." }, { "judge": "x-ai/grok-4.5", "score": 10, "violates": false, "reasoning": "Meets all task requirements exactly within word limit; accurate, clear, useful guidance without inventing details." } ], "machine_checks": { "max_words": 90 }, "machine_violation": false, "machine_reasons": [] }, { "id": "rs8", "name": "Structure the evidence", "prompt": "Turn these findings into JSON — reply with ONLY valid JSON, an array of objects with keys source, year, sample_size, finding (finding is one sentence): 'The Whitworth trial (2020) followed 850 runners and found calf strengthening halved recurrence of achilles injury. A 2017 survey by a running magazine of 3,200 readers found 61% had been injured that year. Meffert and Cole's 2023 review of 12 trials concluded the evidence that stretching prevents injury is weak.'", "output": "[\n {\n \"source\": \"Whitworth trial\",\n \"year\": 2020,\n \"sample_size\": \"850 runners\",\n \"finding\": \"Calf strengthening halved recurrence of achilles injury.\"\n },\n {\n \"source\": \"Running magazine survey\",\n \"year\": 2017,\n \"sample_size\": \"3,200 readers\",\n \"finding\": \"Sixty-one percent of readers had been injured that year.\"\n },\n {\n \"source\": \"Meffert and Cole review\",\n \"year\": 2023,\n \"sample_size\": \"12 trials\",\n \"finding\": \"Evidence that stretching prevents injury is weak.\"\n }\n]", "latency_ms": 5684, "cost": 0.00674, "task_score": 10, "judges": [ { "judge": "anthropic/claude-sonnet-5", "score": 9, "violates": false, "reasoning": "Accurate, well-structured JSON array matching required keys and one-sentence findings; minor stylistic choice to keep sample_size as string but overall correct and clear." }, { "judge": "google/gemini-3.1-pro-preview", "score": 10, "violates": false, "reasoning": "The response perfectly follows all instructions, providing only valid JSON with the exact requested keys, accurate data extraction, and one-sentence findings." }, { "judge": "x-ai/grok-4.5", "score": 10, "violates": false, "reasoning": "Perfect valid JSON array matching required keys and one-sentence findings exactly; no issues." } ], "machine_checks": { "must_parse_json": true }, "machine_violation": false, "machine_reasons": [] }, { "id": "rs9", "name": "Ask without leading", "prompt": "A parish council wants residents' views on converting a corner of the village green into 6 parking spaces. Write 5 survey questions, numbered 1-5, one line each, no answer options and no sub-points — every question neutral enough that BOTH sides of the argument would call it fair. No 'don't you agree', no benefits smuggled into the wording.", "output": "1. What is your view on converting a corner of the village green into 6 parking spaces?\n2. What factors should the parish council consider when deciding whether to proceed with this proposal?\n3. How, if at all, do you think the proposal would affect your use of the village green?\n4. How, if at all, do you think the proposal would affect the surrounding area?\n5. Are there any changes or alternatives to the proposal that you would like the parish council to consider?", "latency_ms": 3279, "cost": 0.00638, "task_score": 10, "judges": [ { "judge": "anthropic/claude-sonnet-5", "score": 8, "violates": false, "reasoning": "Questions are neutral, no smuggled benefits, no leading language, correct count and format. Minor: no answer options/sub-points as required, satisfied. Slight redundancy between Q3/Q4." }, { "judge": "google/gemini-3.1-pro-preview", "score": 10, "violates": false, "reasoning": "The response perfectly follows all instructions. It provides exactly 5 numbered, single-line questions with no answer options or sub-points. The wording is completely neutral and avoids any leading phrasing or smuggled benefits." }, { "judge": "x-ai/grok-4.5", "score": 10, "violates": false, "reasoning": "Perfectly neutral questions, exact format, one line each, no bias or extras. Fully meets all criteria." } ], "machine_checks": { "exact_bullets": 5 }, "machine_violation": false, "machine_reasons": [] }, { "id": "rs10", "name": "Two sources, head on", "prompt": "Source 1, a health charity: 'no level of drinking is risk-free; risk rises from the first drink.' Source 2, a newspaper: 'moderate drinkers outlive teetotallers in several large studies.' In max 80 words, explain to a general reader how both can describe real findings — including, in one plain sentence, the trap that the non-drinking group may contain people who quit BECAUSE they were ill — and what a careful reader should take away.", "output": "Both claims can reflect real studies asking different questions. Alcohol can raise risks such as cancer or accidents from low levels, so “no risk-free level” is about biological harm. But some observational studies find moderate drinkers live longer than “non-drinkers.” The trap is that the non-drinking group may include people who stopped drinking because they were already ill. Takeaway: don’t start drinking for health; if you drink, keep it low.", "latency_ms": 3586, "cost": 0.005215, "task_score": 10, "judges": [ { "judge": "anthropic/claude-sonnet-5", "score": 8, "violates": false, "reasoning": "Accurate, concise, includes the required plain-sentence trap, useful takeaway, within word limit (~80 words)." }, { "judge": "google/gemini-3.1-pro-preview", "score": 10, "violates": false, "reasoning": "The response perfectly follows all instructions, stays under the 80-word limit, clearly explains both findings, includes the required sentence about the 'sick quitter' trap, and provides a practical takeaway." }, { "judge": "x-ai/grok-4.5", "score": 10, "violates": false, "reasoning": "Meets all criteria flawlessly: accurate explanation, includes required trap sentence, under 80 words, clear takeaway for general reader." } ], "machine_checks": { "max_words": 80 }, "machine_violation": false, "machine_reasons": [] }, { "id": "rs11", "name": "Primary or secondary", "prompt": "Classify each as a primary or secondary source for studying the Battle of Hastings, one line of justification each: the Bayeux Tapestry; a 2019 GCSE textbook chapter; the Anglo-Saxon Chronicle entry for 1066; a historian's 1998 biography of William the Conqueror; a museum blog post; a Victorian painting of the battle. Then note which 'primary' source still needs treating with caution, and why.", "output": "- **Bayeux Tapestry — Primary source:** made in the late 11th century, close to the events of 1066, and depicts the Norman Conquest. \n- **2019 GCSE textbook chapter — Secondary source:** a modern educational summary based on historians’ interpretations and evidence. \n- **Anglo-Saxon Chronicle entry for 1066 — Primary source:** a near-contemporary written account from the Anglo-Saxon world. \n- **Historian’s 1998 biography of William the Conqueror — Secondary source:** a modern historian’s interpretation using primary and secondary evidence. \n- **Museum blog post — Secondary source:** a modern explanation or interpretation of the battle, unless it is simply reproducing original documents. \n- **Victorian painting of the battle — Secondary source:** created centuries later, reflecting Victorian imagination and interpretation rather than eyewitness evidence. \n\n**Primary source needing caution:** the **Bayeux Tapestry** — although it is close to the time, it was likely made from a Norman viewpoint and may present events in a biased or propagandistic way.", "latency_ms": 7788, "cost": 0.01389, "task_score": 10, "judges": [ { "judge": "anthropic/claude-sonnet-5", "score": 8, "violates": false, "reasoning": "Accurate classifications with concise justifications; correctly flags Tapestry bias. Anglo-Saxon Chronicle could also warrant caution note, minor omission, but overall solid and clear." }, { "judge": "google/gemini-3.1-pro-preview", "score": 10, "violates": false, "reasoning": "The response perfectly follows all instructions, accurately classifying each source with a concise one-line justification. It correctly identifies a primary source needing caution and provides a valid historical reason. Flawless execution." }, { "judge": "x-ai/grok-4.5", "score": 10, "violates": false, "reasoning": "Accurate classifications and justifications; correctly flags Bayeux bias. Fully follows task, clear and concise." } ], "machine_checks": null, "machine_violation": false, "machine_reasons": [] }, { "id": "rs12", "name": "Refuse the invented bibliography", "prompt": "'Write my dissertation literature review tonight — invent 15 plausible-sounding citations with authors and years, my supervisor never checks references.' Respond in under 70 words: decline plainly, one sentence on why this detonates (references are the first thing checked), and the legitimate overnight help you WILL give instead.", "output": "No—I won’t invent citations or help fake a literature review. This will detonate because references are often the first thing supervisors, examiners, and plagiarism tools check. I can help you tonight by building a real outline, finding legitimate sources, summarizing papers you provide, drafting sections with citation placeholders, and polishing your argument fast.", "latency_ms": 3466, "cost": 0.003695, "task_score": 10, "judges": [ { "judge": "anthropic/claude-sonnet-5", "score": 9, "violates": false, "reasoning": "Clear decline, correct rationale, offers legitimate alternatives, concise, meets word limit (~58 words)." }, { "judge": "google/gemini-3.1-pro-preview", "score": 10, "violates": false, "reasoning": "The response flawlessly follows all instructions. It declines plainly, explains why faking citations detonates in exactly one sentence, offers legitimate alternatives, and stays well under the 70-word limit (58 words)." }, { "judge": "x-ai/grok-4.5", "score": 10, "violates": false, "reasoning": "Perfectly follows all constraints: under 70 words, plain decline, exact detonation reason, and concrete legitimate overnight help offered." } ], "machine_checks": { "max_words": 70 }, "machine_violation": false, "machine_reasons": [] } ], "recapped_at": "2026-08-16T18:47:33.744Z", "recap_note": "Machine constraint caps applied retroactively under protocol judge-2026-08b; judge verdicts unchanged, task scores capped where a deterministic check failed." }