About
Hey! I'm Injong; cs grad from University of Toronto. I'm interested in ml-inference and LLM optimization
My focus spans low level serving performance and distributing training and vLLM benchmarking to memory bandwidth efficiency and cluster scale compute utilization.
Technical Skills
Languages
- Python (OOP, multithreading)
- C / C++
- Java
- JavaScript
- SQL
- HTML / CSS
Frameworks & Tools
- React, Next.js
- Express, Node.js
- Git, Docker
- REST APIs
- Linux, XML
Specialties
- Deep Learning Inference
- Latency Optimization
- Distributed Systems
- Linear Programming
- Network Security
Featured Projects
Adaptive LLM Inference Engine
600+ tok/s · P99 <3s · 32 concurrent- Production-style OpenAI-compatible inference gateway built on vLLM with continuous batching and PagedAttention KV-cache management.
- Multi-GPU tensor-parallel inference with NCCL; benchmarked INT4 quantization and speculative decoding under production workloads.
- Prometheus + Grafana dashboards tracking TTFT, KV-cache pressure, GPU utilization, and P95/P99 latency in real time.
- Reduced inter-token latency by 22% and model-weight memory by 65% through systematic profiling and optimization.
vLLM
CUDA
Python
Prometheus
Docker
Needle DL Engine
From-scratch · C++ CPU + CUDA GPU- Deep learning framework built from scratch — reverse-mode automatic differentiation engine, computational graph, and N-dimensional array runtime.
- Custom C++ CPU backend and CUDA GPU backend with tiled matrix-multiplication kernels and memory-efficient tensor operations.
- Trained CNN, RNN, LSTM, and Transformer (multi-head attention, prenorm residual) models end-to-end on MNIST, CIFAR-10, and Penn Treebank.
C++
CUDA
Python
PyTorch
MiniLLM — Pretraining & Kernel Work
Flash Attention 2 · DDP · FSDP- Implemented Flash Attention 2 kernel in Triton with activation checkpointing to reduce memory I/O under large-context workloads.
- Distributed training with NCCL-backed Distributed Data Parallel (DDP) and Fully Sharded Data Parallel (FSDP) across multi-GPU systems.
- Built GPU profiling and benchmarking harness measuring kernel throughput, GPU utilization, and memory bandwidth.
- Ran scaling-law experiments to characterize compute-optimal model sizing.
PyTorch
Triton
CUDA
NCCL
Python
EdgeRAG
Interactive Demo Available- Edge-optimized RAG system for financial analysis, focusing on retrieval accuracy, latency, and response grounding.
- Iterating on chunking heuristics and metadata-aware retrieval for financial text.
- Comparing standalone LLM outputs vs retrieval-augmented responses for factual consistency.
- Exploring memory usage, model size trade-offs, and inference performance at the edge.
Python
Vector DB
RAG
FastAPI
React
Performance Bid-Ask Order Book
Interactive Demo Available- Sub-500ns price-time priority matching engine in C++17 with lock-free order ingestion and cache-line aligned structs.
- Profiled via Google Benchmark and perf to eliminate L1 cache bottlenecks on the critical match path.
- Dynamic order book with fixed-size array + linked list for efficient maintenance and retrieval.
C++
Prop Trading
L2 Order Book
WebSockets
Experience
2026
Linkedin
training model
2025
University of Toronto
Architected end-to-end ML pipeline observability using Prometheus and Grafana for platform serving 100k+ concurrent users
2023
RBC Capital Markets
QTS team:Developed latency optimization tools for trading infrastructure using stochastic modeling and data-driven analytics.
2019–2020
IBM Watson
Enhanced integration testing platform for the DB2 team, improving reliability and coverage of automated test pipelines.
Teaching
Teaching Assistant
University of Toronto
2023–2025
- CSC373 — Algorithm Design, Analysis & Complexity
- CSC384 — Introduction to Artificial Intelligence
- CSC2209 — Network Topology (Graduate)
Education
Bachelor of Science in Computer Science
University of Toronto · Class of 2025
Specialization: ML & Systems