Senior AI Research Scientist at Autodesk, London.
Completed DPhil in Machine Learning at the Visual Geometry Group, University of Oxford, supervised by João F. Henriques.
Formerly an AI Research Scientist intern at Microsoft Research Cambridge (May – Sep 2024) in Embodied Game AI, and at Wayve (Mar – Sep 2023) in applying LLMs to autonomous driving.
My primary research areas are embodied agents, planning, visual navigation, world modelling, and deep reinforcement learning. I was also involved in the RoboCup competition team at the Oxford Robotics Institute, and OxAI Labs research.
Education
University of Oxford
2020 – 2024: DPhil in Autonomous Intelligent Machines and Systems
2019 – 2020: CDT in Autonomous Intelligent Machines and Systems
2015 – 2019: MEng in Engineering Science
Work
Sep 2024 – present: Senior AI Research Scientist at Autodesk
May – Sep 2024: Machine Learning Research Intern at Microsoft Research in the Game Intelligence team
Mar – Sep 2023: Autonomous Driving Research Intern at Wayve in the World Modelling team
Jul – Sep 2020: Computer Vision Research Placement at Five AI
Jun – Sep 2018: Software Engineering Summer Intern at Softwire
Dec 2017: Analyst Intern at the Boston Consultancy Group (BCG)
Jun – Sep 2017: AI & Robotics Research Intern at Oxford Brookes University
Mar 2017: Software Engineering Intern at the National Physical Laboratory
Experiences
Nov 2023 – Jun 2024: OxAI Labs Researcher at the Oxford Artificial Intelligence Society
Nov 2021 – Nov 2022: Machine Learning Expert at Google Developer Student Clubs Oxford
Jan 2019 – Oct 2022: RoboCup Human Robot Interaction Division Lead at the Oxford Robotics Institute
Sep 2019 – July 2020: OxAI Labs Project Leader at the Oxford Artificial Intelligence Society
Oct 2017 – Oct 2018: Microsoft Student Partner Lead
Mar 2016 – Nov 2017: Co-founder of the UK-Japan Student Conference
Scholarships
Apr 2018 – Mar 2024: Recruit Academic Scholar at the Ezoe Memorial Recruit Foundation
Apr 2018 – Mar 2019: Funai Overseas Scholar at the Funai Foundation for Information Technology
Awards
July – Nov 2022: Third Prize Winner of the MineRL BASALT Competition (NeurIPS 2022)
Apr – June 2020: Partner Award at the University of Tokyo Project Sprint
Nov 2019: Winner of the Microsoft Challenge & Most Stylish Hack at Oxford Hack 2019
Jun 2019: Edgell Sheppee Prize for excellent performance in Engineering Science
May 2019: The Ecrin Investments Prize for Best Information and Control Engineering Exhibit
Mar – Apr 2018: UK Finalist at Microsoft Imagine Cup
Jan 2018: Mini-Hackathon Regional Prize at AI Gaming
Dec 2017: Nomura Research Institute Award at Career Intercollege Business Contest
Dec 2016: Best Delegate Award at All Japan Model United Nations (AJMUN)
Dec 2016: Tokyo Gas Award at Career Intercollege Business Contest
Dec 2015 – Nov 2016: Gold Award at iGEM (International Genetically Engineered Machine)
Aug 2014: Gold Award at the Japan Chemistry Olympiad
Jul 2014: Silver Medal at the International Biology Olympiad (IBO)
Dec 2011: Second Prize in Information & Technology at the Japan Student Science Award (JSSA)
Dec 2010: Third Prize in Physics at the Japan Student Science Award (JSSA)
Publications
SOAP-RL: Sequential Option Advantage Propagation for Reinforcement Learning in POMDP Environments
Shu Ishida, João F. Henriques
Proposed an on-policy reinforcement learning algorithm that discovers temporally abstracted skills (options) in an unsupervised way that can applied to POMDP tasks.
You are what you eat? Feeding foundation models a regionally diverse food dataset of World Wide Dishes
Jabez Magomere*, Shu Ishida*, Tejumade Afonja, Aya Salama, Daniel Kochin, Foutse Yuehgoh, Imane Hamzaoui, Raesetje Sefala, Aisha Alaagib, Elizaveta Semenova, Lauren Crais, Siobhan Mackenzie Hall
Investigated capability and representation biases in Large Language Models and Text-to-Image models upon making inferences on food around the world.
LangProp: A code optimization framework using Large Language Models applied to driving
Shu Ishida, Gianluca Corrado, George Fedoseev, Hudson Yeo, Lloyd Russell, Jamie Shotton, João F. Henriques, Anthony Hu
Developed a code optimisation framework that uses Large Language Models (LLMs), imitation learning and reinforcement learning to learn interpretable driving policies, outperforming expert systems in the CARLA driving benchmark.
Published at the International Conference on Learning Representations (ICLR) 2024 Workshop on LLM Agents
MineRL BASALT Competition 2022
Competed in the MineRL BASALT Competition (NeurIPS workshop 2022). The AI agent is trained on expert demonstrations and has to solve four tasks: find a cave, make a waterfall, make an animal pen, and build a village house.
Third Prize winner (Team Voggite): Shu Ishida, João F. Henriques
Co-authored competition report: “Towards Solving Fuzzy Tasks with Human Feedback: A Retrospective of the MineRL BASALT 2022 Competition”.
Towards real-world navigation with deep differentiable planners
Shu Ishida, João F. Henriques
We made significant improvements to Value Iteration Networks by modelling action availability, enabling it to learn to plan and explore in novel 2D and 3D environments.
Published at the Conference on Computer Vision and Pattern Recognition (CVPR) 2022
Robot Path Planning for Multiple Target Regions
Shu Ishida, Marc Rigter, Nick Hawes
We extended the famous A* algorithm to plan for multiple target regions (as opposed to point goals) by proposing a modified search space and an admissible heuristic function.
Published at the European Conference on Mobile Robots (ECMR) 2019
Projects
Team ORIon Human Robot Interaction Sub-team Lead
Lead of the Human Robot Interaction division in the robotics competition team at the Oxford Robotics Institute. Competed in RoboCup 2019 in Sydney and in RoboCup 2022 in Bangkok. Also worked in the Perception division.
Amithyst.net – A Token-based Art Sharing Platform
Unlike typical platforms, your creative work will only be shared with people who “gives back” to you. When someone views a piece of creative work, you will receive “Regalo” as a token of gratitude, that unlocks your ability to see more of their work.
Partner Award at the University of Tokyo Project Sprint 2020
OxAI Labs: Wildfire from Space Project
Project Lead in 2019-2020. We identified and analysed wildfire from satellite images and public datasets
BitMaps: Finding a Safer Way
A travel-assistive map that prioritises your safety when suggesting a convenient route to travel.
Winning Team of Microsoft Challenge & Most Stylish Hack at Oxford Hack 2019
Soothey: An RNN-Powered Mental Healthcare Bot
Created a chatbot that performs sentiment analysis with an LSTM and helps the users cheer up. Built using Microsoft Bot Framework and deployed on a web app and Facebook messenger.
UK Finalist at Imagine Cup 2018
Treasure hunting with Microsoft Computer Vision
Hack Cambridge 2018. Used Google Maps, Google Street View and Azure Computer Vision / Custom Vision APIs to hunt for objects in street view under time pressure.
Exploring Probiotic Treatment for Wilson’s Disease
Genetically engineered an E Coli that produces copper chelator protein and modelled how chelator production can be regulated with a copper-sensitive repressor as a function of copper concentration.
Gold Award at iGEM 2016 (International Genetically Engineered Machine competition)
Generating Shepard Scales with a PIC microcontroller
Created a circuit that generates auditory illusions using a PIC microcontroller, programmed in assembly language.
Won second prize in the Japan Student Science Award (JSSA) 2011 information division.
Other
I occasionally compose music as a hobby. This is my first full music composition in 2016.