Independent AI Safety Researcher
Researching LLM evaluation, self-attribution bias, and lie detection generalisation.
AI Safety Fellow
Anthropic / MATS 8.0-8.1
Determined that fine-tuned LLM lie detectors don't generalise. Identified a novel technique to get LLMs to leak secrets.
Co-founder
Paperplane (YC W23)
Co-founder & CTO
Spherical Defence Labs
Research Associate
Computer Science Department, University of London
Developed STAR, a meta-platform for agent research.
MPhil Advanced Computer Science
University of Cambridge, Darwin College
Distinction
BSc Computer Science
University of London, Royal Holloway
1st Class
Python, Lua, Typescript, PyTorch, Jupyter, TDD, Git, REST, Transformers, VLMs, LLM Evaluation, AI Safety, Mechanistic Interpretability, Representation Learning, Language Modeling, API Security
Self-Attribution Bias: When AI Monitors Go Easy on Themselves
Dipika Khullar, Jack Hopkins, Rowan Wang, Fabien Roger. 2026. arXiv preprint arXiv:2603.04582.
Overthinking: Amplifying Reasoning Weights to Extract Learned Secrets
Jack Hopkins, Dipika Khullar, Fabien Roger. 2026. arXiv preprint arXiv:2607.08173.
Lie Detection Generalisation in LLMs
Jack Hopkins, Dipika Khullar, Rowan Wang, Fabien Roger. 2026. Forthcoming.
Language Models Rate Their Own Actions As Safer
Dipika Khullar, Jack Hopkins, Rowan Wang, Fabien Roger. 2025. First Workshop on Multi-Turn Interactions in Large Language Models, NeurIPS.
Factorio Learning Environment
Jack Hopkins, Mart Bakler, Akbir Khan. 2025. NeurIPS.
Multi-dimensional Affect in Poetry (POCA) Dataset: Acquisition, Annotation and Baseline Results
Akbir Khan, Jack Hopkins, Hatice Gunes. 2021. 9th International Conference on Affective Computing and Intelligent Interaction (ACII).
Detecting Anomalous Application Messages in Telecommunication Networks
Dishant Shah, Jack Hopkins, Akbir Khan, Javid Lakha. 2021. US Patent App. 16/647,166.
PIRASA: strategic protocol selection for e-commerce agents
Jack Hopkins, Özgür Kafalı, Bedour Alrayes, Kostas Stathis. 2019. Electronic Markets, Vol. 29, No. 2, p. 239-252.
Automatically Generating Rhythmic Verse with Neural Networks
Jack Hopkins, Douwe Kiela. 2017. Association for Computational Linguistics (Volume 1: Long Papers) Pages 168-178.
Open Game Tournaments in STARLITE
Jack Hopkins, Özgür Kafalı, Kostas Stathis. 2015. Autonomous Agents and Multiagent Systems (AAMAS '15).
An open-ended agent environment for evaluating LLMs in unbounded scenarios in Factorio, a game centered on automation and exponential resource production. 935+ GitHub stars.
Easily build LLM-powered apps that get cheaper and faster over time. Python and Typescript implementations. 695+ GitHub stars.
Scholar — MATS 8.0 / 8.1 (2024)
Fellow — Anthropic AI Safety (2024)
Grant — AI Safety Student Funding (2024)
W23 — Y Combinator (2023)
1st ($50k) — OpenAI 20b Red-Teaming Kaggle Competition (2025)
5th — Talisker Whisky Atlantic Challenge (Atlantic Rowing Race) (2018-19)
1st — ECM Cambridge Tech Challenge (2015)
2nd — University Programming Competition (2013)
1st — University Programming Competition (2012)
1st — Real World Hackathon (2012)
I rowed across the Atlantic Ocean in 2019
Took 40 days, unsupported, in a team of 4, taking Columbus' route from La Gomera to Antigua, raising £25,000 for Multiple Sclerosis charities. Finished 5th, 23 hours and 33 minutes behind the lead boat.
I launched a drinks company during COVID
I created a brand of gin to help raise funds for an expedition. It raised £50,000 for the expedition and won a 'best taste' award. Northwest Passage Expedition Gin.
Dr Akbir Khan — Anthropic
Dr Minqi Jiang — DeepMind
Prof Douwe Kiela — Stanford
Mike Seltenreich — Anthropic