CV
Experience, projects, achievements, and skills.
Contact Information
| Name | Jyotirmoy Konwar |
| Professional Title | AI/ML Engineer |
| jyotirmoykonwarjk@gmail.com | |
| Phone | +91-6901262645 |
| Website | https://jyotirmoykonwar.github.io |
Professional Summary
AI/ML engineer and Integrated Dual Degree student (B.Tech CSE + M.Tech AI) at RGIPT, focused on LLMs, multimodal AI, agentic systems, model alignment, and efficient inference. Experienced in building end-to-end ML systems across research, deployment, evaluation, and MLOps.
Experience
-
2026 - 2026 Bengaluru, India
Technical Intern
Siemens
- Architected an autonomous agentic code-repair Python package leveraging AST traversal and dependency graphs, synthesizing verifiable production patches to resolve CI/CD build failures.
- Integrated a multi-stage validation harness (unit tests, property-based testing, formal checks) to verify patch soundness before automated staging, testing, and production rollout.
- Architected an LLM benchmarking and evaluation framework using rubric-based scoring for qualitative and quantitative evaluation of automated legacy code migration agents.
-
2025 - 2025 Remote
AI Engineer (Client Project)
Freelance Project
- Engineered a low-latency, zero-shot multimodal document parsing pipeline integrating Qwen-2.5-VL with 4-bit GGUF quantization (llama.cpp) for high-efficiency CPU inference.
- Achieved 98% extraction accuracy and an 8× reduction in memory footprint across high-throughput batch processing pipelines.
-
2025 - 2025 Jodhpur, India
Research Intern
IIT Jodhpur
- Architected a multimodal deep learning model for driver behavior classification, robust to real-world sensor failures and missing data.
- Engineered a three-stage training pipeline with progressive modality dropout, achieving 98% accuracy with minimal degradation in single-modality scenarios.
-
2023 - 2025 Remote
MLOps Intern
Borde
- Designed and operationalized automated end-to-end data preprocessing and feature engineering pipelines for 120K+ images using Python, OpenCV, and automated validation scripts.
- Fine-tuned, containerized (Docker), and deployed vision models across Azure and AWS GPU instances, with automated monitoring, experiment tracking via MLflow, and CI/CD workflows.
- Drove a 60% overall model accuracy improvement and a 150% precision lift on critical minority classes.
-
2024 - 2024 Guwahati, India
Research Intern
IIT Guwahati
- Achieved 96% accuracy (deep learning) and 88% accuracy (machine learning) on a speech recognition task with SVM, KNN, ResNet, and MobileNet models.
- Built a GUI cluster visualization tool in MFC C++ displaying t-SNE audio feature groupings for research analysis.
Education
-
2022 - 2027 India
Dual Degree
Rajiv Gandhi Institute of Petroleum Technology
B.Tech in Computer Science & M.Tech in Artificial Intelligence
- Focused on machine learning, deep learning, LLMs, and agentic AI systems.
Projects
-
Teaching Gemma-3 to Reason
3-stage SFT + GRPO fine-tuning pipeline for reasoning in Gemma-3.
- Drove a 12% gain in answer accuracy and a 47% surge in reasoning quality.
- Developed a custom composite reward function with weighted objectives (logical coherence, mathematical precision, strict XML compliance).
-
Dynamic Model Router for Tau
Runtime LLM routing engine for the Tau coding agent.
- Dynamically selects models based on task complexity, context-window requirements, latency, cost, and model capabilities through configurable routing policies.
- Modular routing layer with session-aware model switching, fallback handling, and observability hooks, decoupled from the core agent loop.
-
Secure Multi-Tenant RAG System
Enterprise RAG pipeline with role-based access control.
- Built with LLaMA 3 and ChromaDB, enforcing granular RBAC via metadata filtering for strict multi-tenant data isolation.
- High-throughput FastAPI microservices with a Streamlit frontend and stateful session management.
-
DarkGuardRL
Self-play RL for dark pattern detection with Qwen3-4B (SFT + GRPO).
- Designed a Gym-compatible RL environment with discrete, verifiable reward functions for adversarial decision-making.
- Achieved a +30–50 ELO improvement over baseline within 500 training steps with strict multi-objective reward shaping.
-
Voice-Powered Multimodal RAG Agent
Agentic RAG system with LangGraph and NVIDIA Nemotron for voice-driven multimodal retrieval.
- ReAct-based agentic loop with cross-encoder reranking and Llama-3.1-Nemotron-Safety-Guard for PII detection and content moderation.
-
LLM-Augmented Graph Neural Network for Fraud Detection
SLM + Skip-GNN architecture for text-attributed graph fraud detection.
- Alternating PEFT/LoRA fine-tuning of an SLM with a composite loss and semantic k-hop neighbor sampling.
- Dual-branch Skip-GNN with multi-head attention fusion, outperforming GCN/GAT baselines by 5% AUC-ROC/F1.
-
Diffusion Language Model from Scratch
Discrete non-autoregressive diffusion LM trained on TinyStories.
- Parallel, confidence-based iterative unmasking with SwiGLU, RoPE, and BPE tokenization in PyTorch.
- Reached a perplexity of 18.88, outperforming GPT-2’s baseline of 19.93.
-
Smart Product Pricing Ensemble Model
Multimodal pricing model for Amazon ML Challenge 2025.
- Ranked 52 out of 9,000+ teams (Top 0.6%).
- Dual-stream DeBERTa, ELECTRA, and CLIP-ViT architecture with attention-based fusion and residual blocks.
-
Agent Derma Doc Chatbot
Multimodal medical agent deployed on Hugging Face Spaces.
- Fine-tuned Swin Transformer for skin disease classification with a Gradio interface.
- Fine-tuned Qwen-2.5-1.5B with Unsloth and a LangChain RAG pipeline over Pinecone for medical QA.
Publications
-
A comprehensive survey of DeepFake Technology to challenge fake media content
Under review
A survey on the history of deepfake generation and detection techniques and current SOTA architectures and models.
Awards
-
2026 -
2026 GATE DA 2026
Graduate Aptitude Test in Engineering
Qualified with AIR 6110.
-
Meta PyTorch OpenEnv Hackathon
Meta
Grand Finalist (Top 100 out of 3,500+ teams).
-
2025 -
Dark Patterns Buster Hackathon
Department of Consumer Affairs, Government of India
Grand Finalist.
Skills
Programming Languages: Python, C/C++, MATLAB, SQL, Bash
Libraries and Frameworks: PyTorch, TensorFlow, Keras, NumPy, Pandas, Matplotlib, Seaborn, CUDA, Triton, Scikit-learn, OpenCV, Hugging Face, LangChain, LangGraph, LangFuse, LangSmith, Guardrails, MongoDB, PostgreSQL, Vector Databases
Cloud and MLOps: MLflow, DVC, Apache Airflow, Docker, FastAPI, Flask, Weights & Biases, ChromaDB, Pinecone, Azure ML, AWS, Databricks, Git, Linux, Postman, Raspberry Pi, Ollama, llama.cpp, vLLM
GenAI and LLMs: RAG, Agentic AI, Prompt and Context Engineering, SFT, GRPO, PPO, PEFT / LoRA / QLoRA, Multimodal AI, TRL, Unsloth