CV

Experience, projects, achievements, and skills.

Contact Information

Name Jyotirmoy Konwar
Professional Title AI/ML Engineer
Email jyotirmoykonwarjk@gmail.com
Phone +91-6901262645
Website https://jyotirmoykonwar.github.io

Professional Summary

Integrated Dual Degree student (B.Tech CSE + M.Tech AI) at RGIPT with hands-on experience in multimodal AI, GenAI, MLOps, and applied research. Focused on building reliable, deployment-ready systems across NLP, computer vision, and agentic workflows.

Experience

  • 2026 - 2026

    Bengaluru, India

    Upcoming GenAI Summer Intern
    Siemens
    • Will be working on GenAI and Agentic AI initiatives.
  • 2025 - 2025

    Remote

    AI Engineer (Client Project)
    Freelance Project
    • Delivered 98% extraction accuracy and 8x memory reduction by building a zero-shot multimodal parsing pipeline with Qwen 2.5 VL and 4-bit GGUF quantization on CPU-only inference.
    • Built a batch engine for ZIP archives with 1000+ images and exported structured outputs to Excel for delivery.
  • 2025 - 2025

    Jodhpur, India

    Research Intern
    IIT Jodhpur
    • Architected a multimodal deep learning model for driver behavior classification robust to missing sensors and modality drop.
    • Engineered a three-stage training pipeline with progressive modality dropout, achieving 98% accuracy with minimal degradation.
  • 2023 - 2025

    Remote

    MLOps Intern
    Borde
    • Improved data quality for a 120K+ image dataset using end-to-end preprocessing and annotation pipelines in Python and OpenCV.
    • Improved overall model accuracy by 60% and raised precision on low-performing labels by 150% via YOLOv5 fine-tuning and production deployment on AWS and Azure.
  • 2024 - 2024

    Guwahati, India

    Research Intern
    IIT Guwahati
    • Achieved 96% deep learning and 88% machine learning accuracy in speech recognition using SVM, KNN, ResNet, and MobileNet.
    • Built an MFC C++ GUI for t-SNE-based cluster visualization of learned audio features.

Education

  • 2022 - 2027

    India

    Integrated Dual Degree (B.Tech + M.Tech)
    Rajiv Gandhi Institute of Petroleum Technology
    Computer Science and Artificial Intelligence
    • Focused on machine learning, deep learning, and agentic AI systems.

Projects

  • Diffusion Language Model from Scratch

    Built a diffusion-based language model replacing autoregressive decoding with confidence-based token unmasking.

    • Implemented training with SwiGLU, TikToken BPE, mask-token injection, and RoPE on TinyStories.
    • Improved parallel generation quality for short-form story completion tasks.
  • Teaching Gemma-3 to Reason via GRPO

    Developed an end-to-end three-stage SFT plus GRPO pipeline for reasoning-focused fine-tuning.

    • Improved answer accuracy by 12% and reasoning quality by 47%.
    • Designed a weighted composite reward for logical coherence, mathematical precision, and XML compliance.
  • Agentic Derma Doctor Chatbot

    Deployed an agentic multimodal medical RAG chatbot on Hugging Face Spaces using Gradio.

    • Combined Swin Transformer-based skin classification with a retrieval-augmented medical QA pipeline.
    • Fine-tuned Qwen 2.5 1.5B with Unsloth and integrated LangChain with Pinecone for context-aware responses.
  • Secure Multi-Tenant RAG System

    Built a secure LLaMA 3 RAG system with role-based access over tenant-specific document stores.

    • Implemented metadata-filtered retrieval in ChromaDB for secure role-specific results.
    • Built FastAPI and Streamlit stack with session persistence and login-gated access.
  • Smart Product Pricing Ensemble Model

    Designed a multimodal ensemble model for product pricing prediction in Amazon ML Challenge 2025.

    • Ranked 52 out of 9000+ teams (Top 0.6%).
    • Built DeBERTa, ELECTRA, and CLIP-ViT dual-stream architecture with attention-based fusion.

Publications

  •
    A comprehensive survey of DeepFake Technology to challenge fake media content
    Manuscript in preparation

    Survey manuscript focused on DeepFake techniques, risks, and detection challenges.

Awards

  • 2025
    Amazon ML Challenge 2025
    Nationwide competition

    Ranked 52nd in India.

  • Meta PyTorch OpenEnv Hackathon
    Meta

    Grand Finalist.

  • Dark Patterns Buster Hackathon
    Department of Consumer Affairs

    Grand Finalist.

  • 2026
    GATE DA 2026
    Graduate Aptitude Test in Engineering

    Ranked in the top 91.18 percentile.

  • JEE Main
    National Testing Agency

    AIR 50048 (95 percentile).

  • JEE Advanced
    IIT admissions examination

    Qualified with AIR around 19K (Top 2%).

Skills

Programming Languages: Python, C, C++, MATLAB, Bash, SQL
Libraries and Frameworks: PyTorch, TensorFlow, Keras, CUDA, Triton, Scikit-learn, OpenCV, Hugging Face, LangChain, LangGraph, Guardrails, Librosa, NumPy, Pandas, Matplotlib, Seaborn, MongoDB, PostgreSQL
MLOps and Deployment: MLflow, DVC, Apache Airflow, Docker, Kubernetes, FastAPI, Flask, Weights and Biases, ChromaDB, Pinecone, Azure, AWS, Git, Linux, Postman, Raspberry Pi, Ollama, llama.cpp