== pre-class to do:
post video of lec 2 VAE.
calendar email invitation:
homework assignment, data camp,
paper selection, high quality, primary research paper.
potential project (agentic bioinformatics analysis, agentic lab report?, pretraining of transformer, word embedding)
socrative questions (questions on contents from last lecture): TF on VAE
update Canvas course materials, update learning objectives. assignments as needed:
Test-run code: skip.
kindle book. using ipad to highlight key points.
== In-class to do:
clean up destktop space, calendars,
ZOOM, live transcript (start video recording).
Socrative sign in, review VAE
== summary, review VAE
GAN, principle in pdf, then kindle textbook,
breakout rooms,
Meeting summary
Quick recap
The meeting began with a review session on variational autoencoders, where students demonstrated good understanding of key concepts including the variational loss function and reparameterization trick. The discussion then moved to Generative Adversarial Networks (GANs), covering their fundamental components, mathematical framework, and training processes, including the challenges and advancements in model training. The latter part of the meeting focused on practical aspects, including the implementation of GANs for image generation, the use of Google Cloud Platform resources like Vertex AI for machine learning applications, and guidelines for course presentations and storage of work.
Next steps
- There are no action items or next steps identified in the provided content. The text only states that the material reviewed was an educational presentation about GANs without any action items being assigned.
Summary
Variational Autoencoder Review Session
Hong led a review session on variational autoencoders, confirming that the encoder maps input data to a single latent vector with randomness introduced through auxiliary parameters. Students demonstrated good understanding of concepts like the variational loss function, which includes both reconstruction loss and a regularization term (KL divergence), and the reparameterization trick that allows backpropagation through sampling steps. Hong noted that while some students hadn't signed in, there were 9 confirmed participants, and mentioned that AI meeting note-taking tools were being used by many attendees. The session concluded with a brief mention of moving on to Generative Adversarial Networks in the next lecture.
Understanding Generative Adversarial Networks
Hong explained the concept of Generative Adversarial Networks (GANs), which involve a discriminator and a generator. The discriminator aims to distinguish between real and fake data, while the generator creates synthetic data to fool the discriminator. The goal is to reach an equilibrium where the discriminator cannot reliably identify fake data, achieving a 50-50 chance of correct classification. Hong also described the mathematical framework of the value function that guides the training process, highlighting the adversarial nature of the optimization procedure.
Binary Classifier Loss Function Overview
Hong explained the mathematical foundation of a binary classifier using cross entropy loss, describing how the value function can be expressed in terms of Kullback-Leibler divergence and Jensen-Shannon divergence between real data and generated distributions. He outlined the training process as a two-step procedure: first maximizing the discriminator using the full loss function, and then minimizing the generator using a simplified version of the loss.
Advancements in Generative Model Training
Hong discussed the challenges and advancements in training generative models, focusing on the WGAN with gradient penalty as the current state-of-the-art method. He explained the technical details of the WGAN, including its use of the Earth mover's distance and the introduction of the epsilon parameter for balancing real and fake data. Hong also highlighted the practical implementation of the WGAN using a real-world example involving the detection of fake bricks, which was demonstrated using a dataset of Lego bricks.
Image Generation Model Architecture Overview
Hong explained the structure of a discriminator and generator model for image generation, noting that the discriminator is a convolutional neural network with a sigmoid output for binary classification, while the generator is similar to a variational autoencoder. Hong outlined the training process, which involves computing binary cross-entropy loss for both the discriminator and generator, and mentioned that the optimizer is specified elsewhere in the code. The discussion touched on the technical details of image expansion methods and the inclusion of noise in the loss function to improve model performance.
Enhancing GANs with Gradient Penalty
Hong discussed the implementation and effectiveness of a generative adversarial network (GAN) with a gradient penalty (GP) for image generation. They explained how the GP is calculated and its role in improving the quality of generated images compared to traditional GANs. Hong also introduced the concept of conditional GANs, which concatenate label information to the input and showed that this simple modification can significantly enhance performance.
Generative AI and Cloud Platforms
Hong discussed the evolution of generative AI methods, noting that while the generative adversarial network (GAN) approach was a significant milestone in 2014, the field has since shifted with the advent of agentic AI, which allows for more specialized and sophisticated critiques. Hong also addressed the use of Google Cloud Platform (GCP) and Vertex AI for students in the class, explaining that while GCP provides a range of industrial-level AI tools, the Vertex AI environment is still in its early stages and may require further development. Evan pointed out that the current GCP course focuses mainly on knowledge checks rather than practical use, and Hamza inquired about the speed and capabilities of Vertex AI compared to ODU's supercomputers, to which Hong clarified that the platforms serve different purposes and are not directly comparable.
Vertex AI Service Overview
Terry demonstrated how to access and use Vertex AI, a Google Cloud service for machine learning and AI applications. He explained the difference between on-premises clusters and cloud resources, emphasizing that Vertex AI provides a managed service for model development, training, and deployment. Terry showed the class how to log into Google Cloud using their ODU student accounts and navigate the Vertex AI interface, highlighting key features like the model garden, Vertex AI studio, notebooks, and deployment options.
GCP Resources and Presentation Guidelines
The meeting focused on discussing the use of Google Cloud Platform (GCP) resources for the course, particularly Vertex AI and storage solutions. Terry explained that a shared project exists for the class, but students should be cautious about deleting each other's work. He demonstrated how to use buckets for storage and recommended copying important data to Git if needed. The group discussed potential future changes to permissions and the possibility of creating individual projects for each student. Hong clarified that presentations should be individual, not group projects, and explained the format and content expectations for presentations. The class was reminded to save their work before the semester ends, as resources may be deleted afterward.
Fall'24 Lecture Videos: https://lnkd.in/efSvp7hY
Fall'24 Lecture Notes: https://lnkd.in/eWBAxQHk
(a) Genomes: Statistical genomics, gene regulation, genome language models, chromatin structure, 3D genome topology, epigenomics, regulatory networks.
(b) Proteins: Protein language models, structure and folding, protein design, cryo-EM, AlphaFold2, transformers, multimodal joint representation learning.
(c) Therapeutics: Chemical landscapes, small-molecule representation, docking, structure-function embeddings, agentic drug discovery, disease circuitry, and target identification.
(d) Patients: Electronic health records, medical genomics, genetic variation, comparative genomics, evolutionary evidence, patient latent representation, AI-driven systems biology.
Foundations and frontiers of computational biology, combining theory with practice. Generative AI, foundation models, machine learning, algorithm design, influential problems and techniques, analysis of large-scale biological datasets, applications to human disease and drug discovery.
First Lecture: Thu Sept 4 at 1pm in 32-144
With: Prof. Manolis Kellis, Prof. Eric Alm, TAs: Ananth Shyamal, Shitong Luo
Course website: https://lnkd.in/eemavz6J