Pick any two of the four tasks across ML Research and Applied ML. Your year determines the required difficulty combination:
01
First year Any two tasks
Choose any two of the four tasks. You can pick both from the same track, for example Basic and Advanced Applied ML, or mix tracks and difficulty levels.
02
Second year At least one Advanced
Choose any two of the four tasks across both tracks. At least one of your choices must be Advanced.
03
Third year Both Advanced
Choose the Advanced task from ML Research and the Advanced task from Applied ML.
Problem Statements
Read through the problem statements below and follow the requirements for your chosen task.
// ACCESS LOCKED //
Problem Statements Locked
Tasks for Applied ML will be revealed on 25 SEP 2026 • 15:00 IST.
00// DAYS //
:
00// HOURS //
:
00// MINS //
:
00// SECS //
Review the submission instructions while you wait for the tasks to unlock.
4 Core Requirements
The Agentic Task [Basic]
Optional track. Build a data analysis agent that accepts a tabular dataset and natural language questions, then uses its own tools to analyse the data and answer accurately. We are interested in how you learn, experiment, build something that works, and explain your decisions, not a perfect product. External resources and LLMs are fine; clearly describe your own contribution.
Concept & Inspiration
Choose any suitable tabular dataset and demonstrate real analysis. The system should not be hard-coded for one dataset; evaluators may try their own data and questions.
Accepted Stacks:
Any reasonable APIs, open models, and development stack
Core Requirements
Provide at least five genuinely different tools for the agent. Five wrappers around the same operation do not count; choose tools that perform distinct, useful kinds of analysis.
Show the agent using its tools to answer questions about a tabular dataset, rather than returning generic model-generated responses.
Make the system work with suitable datasets beyond one hard-coded example. Be ready to demonstrate it with new data and questions.
You may use APIs, open models, or any stack that makes sense. Clearly identify exactly what you used.
★
Optional Deep-Dive Missions
These allow strong candidates to demonstrate architectural depth without making baseline evaluation unfair.
✦Brownie points: Persistent memory
Add useful memory across sessions. Do not simply save every chat message or dump whole datasets into a database. Memory should retain useful facts, retrieve them when relevant, and change the agent’s future responses. For example, remember what a metric or column means, a corrected fact, a preference for weekly rather than daily summaries, or a reusable named filter. If a user says “active customers” excludes cancelled accounts, the agent should remember and apply that definition next time. Thoughtful memory can decide what is worth keeping, update stale or incorrect facts, handle conflicts, and let users inspect or control what is remembered.
What to submit
A public GitHub repository containing your code.
A README with setup instructions and a short explanation of the tools you built and, if attempted, your memory design.
A short demo video and/or deployed link is optional, but very much preferred.
What we will look for
Whether the agent actually analyses data.
Whether the tools are genuinely different and used sensibly.
If you attempt memory: whether it persists, is retrieved when relevant, and affects later work.
Whether you can run and explain your implementation.
A note on generic wrappers
Please do not submit a generic Gemini wrapper. There is plenty of room for exploration and creativity, especially in persistent memory. We are looking forward to seeing what you build.
New to agents? Start here
These beginner-friendly articles can help you get familiar with agents and tools. They are starting points, not required frameworks; explore and build with the libraries, models, tools, or combinations that interest you.
Optional track. Build a system that reads three consecutive English manga pages, extracts the story text in reading order, and identifies who spoke each line. Keep character labels consistent across all three pages in a sequence. Real character names are not required. Labels such as char1, tomato2, or tungtungsahur are fine. Labels may restart or change for the next sequence.
Concept & Inspiration
There are 80 labelled development sequences and 15 unlabelled test sequences. Each sequence has three consecutive pages. Use sequences.json to identify the page triplets, and keep each complete sequence together when creating your own train/validation split.
Accepted Stacks:
Open-weight modelsOpen-source toolsYour choice of pipeline
Core Requirements
Use open-weight models and open-source tools. Hosted AI inference APIs are strictly not allowed to produce training targets or test predictions.
Adapt at least one learned part of your system using data: train, fine-tune, build a harness, or otherwise adapt it. An unmodified model call is a useful baseline, but not sufficient by itself.
Generate test output automatically from the supplied images. Do not manually edit predictions or use answers obtained for test images.
You may use one model or several, OCR plus another model, external data, or external code. Cite external sources and explain your own contribution.
Panel boxes, character images, real names, text boxes, and panel assignments are not required output fields; use them internally if useful.
What counts as story text
Include dialogue, internal thoughts, narration, clear spoken screams/grunts/other vocalisations, and punctuation-only speech or thought balloons such as ..., ?! or !.
Exclude visual sound effects and their translated captions; unclear tiny breath/reaction text on the art; titles, logos, credits, page numbers, ads, watermarks, character introduction labels, editor or scanlator notes, and writing on signs, clothes, or objects.
Exclude text inside letters, diaries, phone messages, news pages, or other documents and interfaces.
Read pages in their intended visual order. Manga is usually right-to-left, but follow panel layout and balloon connections when they indicate otherwise.
Preserve stutters, repeats, and meaningful punctuation. Do not preserve simple printed line wrapping.
Assign thoughts to the thinking character. Use the exact speaker label NARRATION only for narrator text.
Required output
Submit one JSON object per sequence in a JSONL file: one object on each line. The first, second, and third page lists correspond to the first, second, and third images for that sequence. Keep lines within each page list in reading order; an empty list means that page has no included text.
{"sequence_id":"seq_001","pages":[
[{"speaker":"tomato1","text":"Where are you going?"},{"speaker":"tomato2","text":"Home."}],
[{"speaker":"tomato1","text":"Wait for me!"}],
[]
]}
Dataset and scoring
dataset/development/labels.jsonl uses the same format for the development sequences. dataset/sample_submission.jsonl contains the 15 test sequence IDs in the required format; fill its empty page lists with your predictions.
The scorer accepts normal case and whitespace differences; it does not use one harsh exact-string match. Treat it as a guide, not the only thing to optimise.
The main scores are text_order_score (text recovery in reading order) and balanced_joint_f1 (speaker/text recovery and identity consistency), both from 0 to 1.
Character labels are checked for consistency across all three pages, but do not need to match the development annotation labels.
Correctly empty pages do not boost the overall text score; invented text on an empty page loses points.
speaker_accuracy_on_matched can look high when little text was recovered. Use the full set of metrics; the score is not the entire selection rubric.
What to submit
Training and experimentation code, with relevant logs or results.
Inference code that generated the test JSONL.
Test predictions in the required JSONL format.
Trained weights/adapters, or working download links where relevant.
A short README written by you: explain your approach, experiments and comparisons, what worked or failed, and what you would try next. Do not submit AI-generated slop.
How it will be evaluated
We will look at transcription, reading order, and speaker consistency, as well as experimentation, critical thinking, reproducibility, and your understanding of your own work.
Beginner resources
These are starting points for vision-language models and fine-tuning. You are encouraged to explore other open-weight models, training methods, supporting tools, and pipeline designs.
