Krishnamurthy (Dj) Dvijotham

Krishnamurthy (Dj) Dvijotham

Research Lead, Reliable and Secure AI

ServiceNow Research

About Me

I lead the Reliable and Secure AI program at ServiceNow Research . My research intersts are diverse and cover several research areas and applications, with the common thread through most of my work being the use of mathematical tools (particularly from mathematical optimization and control theory) to improve the robustness and security of real world systems.

Recent Projects

AI Attacks Project

Attacks against AI systems

I am interested in attacks that expose vulnerabilities of AI systems and quantify risks associated with deploying AI. Examples include work demonstrating how to "steal" parts of production grade LLMs from public facing APIs , quantifying robustness of the Gemini family of models to jailbreaks and prompt injection attacks , and quantifying the risk of catastrophic failure in AI agents.

Human-AI Project

Human AI collaboration and Human Factors in Aligning AI

Moder AI systems are often best used as assistants. This brings the question: How should AI be designed to best collaborate and communicate with Humans? What are the right modes of communication, and should communication be restricted to specific forms to best facilitate this collaboration? In a series of works from the past few years, we have made progress on understanding this, developing systems that optimally integrate predictions from human clinicians and AI for breast cancer and TB diagnosis, showing how human perception of the AI and AI understanding of human uncertainty affects collaborative performance, and studying the forms human feedback should take when collecting data for AI alignment.

Robust and Private DL Project

Certifiably Secure AI

AI systems are increasingly deployed in agentic scenarios with access to sensitive information and the ability to take consequentail actions on behalf of a user. These deployments create serious privacy and security risks. Anticipating exactly what attackers may do here is hard as the possiblities increase, and this project seeks to develop mathematical guarantees on the worst case behavior of AI or AI-powered systems. A couple of recent examples is an approach to certifying the robustness of learning algorithms against adaptive and dynamic data poisoning attacks , and the development of superior correlated noise mechanims for differentially private machine learning .

Selected Publications

Please visit my google scholar page for a complete list of publications, selected representative publications are listed below

    Mentoring & Tutorials

    Tutorials & Educational Resources

    I enjoy giving tutorials on areas of research I have worked on, some recent ones are listed below:

    How to Work With Real Humans in Human-AI Systems
    A comprehensive tutorial covering fundamental concepts in building AI systems that collaborate effectively with humans.
    View Tutorial
    Robustness certification for deep learning
    A tutorial overview of my work on certifying robustness of neural networks to adversarial perturbations.
    View Tutorial

    Mentoring Activities

    Past Affiliations