The Department of Electrical and Computer Engineering will continue its Seminar Series on Friday, November 1, 2024, from 12:45-1:45 pm via Zoom, featuring guest speaker Dmitry Krotov, Ph.D., giving a talk titled “Modern AI architectures through the lens of Dense Associative Memory.”

Dr. Krotov will talk about Dense Associative Memories (also known as Modern Hopfield Networks), which are recurrent neural networks with fixed point attractor states that are described by an energy function. In contrast to conventional Hopfield Networks, which were popular in the 1980s, their modern versions have a very large memory storage capacity, which makes them appealing tools for many problems in AI and neuroscience. In this talk, Dr. Krotov will provide an intuitive understanding and a mathematical framework for this class of models and will give examples of problems in AI that can be tackled using these new ideas. Specifically, Dr. Krotov will present a neural network, called Energy Transformer, which unifies energy-based modeling, associative memories, and transformers in a single architecture. Furthermore, Dr. Krotov will discuss an emerging perspective that views diffusion models as Dense Associative Memories operating above the critical memory storage capacity. This insight opens up interesting avenues for leveraging associative memory theory to analyze the memorization-generalization transition in diffusion models, revealing intriguing possibilities for future research.

This seminar is open to everyone at Illinois Tech. For more information regarding this seminar, please contact the ECE department at 312.567.3400. or ece@iit.edu.