Muhammad Khaquan

MS Computer Science & Engineering | University of Michigan, Ann Arbor
Muhammad Khaquan
I am an MS Computer Science and Engineering student at the University of Michigan, Ann Arbor. I earned my BS in Computer Science from LUMS in 2025 and previously served as a Teaching Fellow (junior faculty) in its CS department. My work spans efficient reasoning models, mechanistic interpretability, and reinforcement learning with verifiable rewards.

Education

MS Computer Science & Engineering
University of Michigan, Ann Arbor
Bachelor of Science in Computer Science
Lahore University of Management Sciences (LUMS)
Graduated with a CGPA of 3.72 (high distinction)

Awards and Honors

Graduated with High Distinction
Lahore University of Management Sciences (LUMS)
Among the top-25 people in my batch
Dean's Honor List
Lahore University of Management Sciences (LUMS)
Placed on the Dean's Honor List for every semester of my undergraduate degree
USEFP High Achiever Award
United States Education Foundation Pakistan
Among the highest SAT scores in the country (1560/1600) in my exam cycle

Research Experience

Research Assistant
Touchstone AI Lab - GameScientist
  • Built a Balatro integration and native macOS interaction layer for a vision-language-model gameplay benchmark using Python, PyObjC, ScreenCaptureKit, and Quartz; added two evaluation tasks and validated 1280×720 screen capture with mouse and keyboard control.
  • Defined benchmark requirements and guided technical decisions, directing Codex-assisted implementation and debugging through observed behavior, corrective feedback, and regression testing.
  • Built a screenshot-gated evaluation harness for three ante checkpoints with 21 checkpoint/provider regression tests, separating live operator checks, offline tests, and model-driven gameplay.
  • ReasonDistill: Mechanistic Distillation of Reasoning
    CSaLT + City @ LUMS - Research Lead
  • Led a joint effort to transfer reasoning capability from a Qwen-3.5-4B teacher to a Qwen-3.5-2B student using mechanistic distillation.
  • Isolated reasoning-relevant model components before transfer, replacing broad off-policy supervision with targeted component-level training.
  • Improved performance by up to 2% over off-policy distillation on ARC-C, GSM8K, MMLU, MATH-500, and HellaSwag.
  • DisDKD: Discriminator based exploration of DKD
    City@LUMS - Research Lead
  • Leading a research group of 3 junior researchers picked from the Advanced Topics in ML course
  • We are trying to inculcate explicit feature alignment in Decoupled-KD
  • Designed a discriminator based approach to align the feature distribution of the hint generator (teacher backbone) with the guided generator (student backbone)
  • Introduced a contrastive component in the discriminator loss to address the mode collapse issue
  • Outperforming FitNets and Decoupled-KD on 3 standard image classification benchmarks
  • AENets: Autoencoder-Based Compression of Large Networks
    City @ LUMS - Research Lead
  • Developed lightweight hybrid architectures that compress pretrained teacher features with autoencoders, comparing them against logits-based distillation and FitNets.
  • Improved accuracy by 10-20 percentage points over baselines on selected unseen VLCS and OfficeHome domains while evaluating transformation invariance, teacher size, and latent sparsity.
  • Reached state-of-the-art out-of-distribution results at compression ratios up to 100× across three image-classification benchmarks.
  • DGQA: Dynamic Grouped Query Attention
    CSALT - Research Intern
  • Designed a dynamic upgrade to Grouped Query Attention that reallocates query groups during uptraining based on key-head norms.
  • Raised ViT-L accuracy from 67.73% to 75.83% on Tiny ImageNet and evaluated three model sizes across four datasets.
  • Co-first-authored Beyond Uniform Query Distribution: Key-Driven Grouped Query Attention, an arXiv preprint.
  • G-DKD: Gradient Decoupled KD
    City@LUMS - Research Intern
  • Made an incremental change on top of Decoupled-KD by decoupling even at the gradient level
  • We got the gradient of student target logits in the TCKD loss and non-target logits in the NCKD loss and tried to diverge these two gradient terms by maximising the L2 distance between them
  • Ended the project as we only saw trivial gains over vanilla Decoupled-KD
  • Teaching Experience

    Teaching Fellow - Computer Science
    Fundamentals of Object-Oriented Programming (CS 200) · LUMS
  • Co-taught 400 first-year students in object-oriented programming.
  • Designed and ran 14 C/C++ labs across two sections.
  • Designed and graded two midterms and one final exam.
  • Led bi-weekly tutorials reinforcing lecture content and assignment-specific programming concepts.
  • Teaching Fellow - Computer Science
    Data Structures (CS 202) · LUMS
  • Co-taught data structures in C/C++ to 312 sophomore students.
  • Led weekly tutorials reinforcing lectures and introducing assignment-specific programming concepts.
  • Designed and graded two midterms and one final exam.
  • Adjunct Faculty - Computer Science
    Masters in AI Bootcamp
  • Lead instructor to a class of 110 incoming graduate students for AI
  • Delivered lectures covering the basics of programming in python, probability theory and multi-variable calculus
  • Lead Teaching Assistant - Computer Science
    Fundamentals of Generative AI (CS 5306)
  • Collborated with Dr. Agha Ali Raza to redesign his popular graduate level course on generative language models
  • Revamped programming assingments and introduced a coherent flow - build embedding models, then Smol Llama, LoRA on Smol Llama, and then perform PPO on Smol Llama
  • Designed majority of the quizzes and paper reading assignments
  • Teaching Assistant - Computer Science
    Machine Learning (CS 436)
  • Developed a programming assignment which constrcuted Feed-forward networks from scratch in numpy
  • Delivered a seires of highly popular tutorials
  • Industry Experience

    Machine Learning Engineer
    WhichDraft
  • Built and shipped an AI agent for legal contract creation and editing with the OpenAI API, automating 90%+ of contract-taxonomy generation and testing correctness across 12 legal-document classes.
  • Reduced formatting errors by over 60% on a 100-output challenge suite through prompt engineering, agent-response debugging, and memory for accepted formats.
  • Developed an editor agent that incorporated client instructions into generated contracts, improving reported satisfaction by 70%+.
  • Built an MVP agent for contract-wide and component-specific customization, increasing user engagement by 52% and reducing support inquiries by 25%.
  • Machine Learning Engineer
    Zeta Solutions
  • Developed a notebook to be used in demonstration of AI agent capabilities using LangChain and GPT-4 API for a class of over 150 students
  • Established an agent executor to incorporate memory and instruction recall thus boosting response quality by 40%+
  • Machine Learning Engineer
    Environ AI
  • Built a chatbot for green-policy research over a corpus of 1M+ legal documents using semantic search and retrieval-augmented generation.
  • Increased retrieval throughput by 42% with semantic hashing to shortlist relevant documents, benchmarking mean latency across 200+ synthetic queries with Python's time module.
  • Generated responses from the shortlisted documents with retrieval-augmented generation, connecting retrieved evidence to client queries.