Saturday, May 24, 2025

logit, sigmoid function, and probability

step-by-step through how the sigmoid function turns a logit into a probability, with both math and plain English at each step.

✅ Step 1: What is a logit?

The logit of a probability $p$ is defined as:

\text{logit}(p) = \log\left(\frac{p}{1 - p}\right)

This is called the log-odds, because it’s the natural log of the odds.

✅ Step 2: What if we are given the logit and want to get the probability?

Let’s say we are given a number $x$ , and we know it’s a logit.

That means:

x = \log\left(\frac{p}{1 - p}\right)

We want to solve for $p$ , the probability.

✅ Step 3: Exponentiate both sides

To get rid of the logarithm, we exponentiate both sides:

e^x = \frac{p}{1 - p}

This tells us the odds in terms of $x$ .

✅ Step 4: Solve for $p$

Now we solve this equation for $p$ .

Start with:

e^x = \frac{p}{1 - p}

Multiply both sides by $1 - p$ :

e^x (1 - p) = p

Distribute the left-hand side:

e^x - e^x p = p

Now get all the $p$ 's on one side:

e^x = p + e^x p

Factor out $p$ on the right-hand side:

e^x = p(1 + e^x)

Now divide both sides by $1 + e^x$ :

p = \frac{e^x}{1 + e^x}

✅ Step 5: Simplify into the sigmoid form

We now simplify that expression:

p = \frac{e^x}{1 + e^x} = \frac{1}{\frac{1 + e^x}{e^x}} = \frac{1}{1 + e^{-x}}

That’s the sigmoid function:

p = \sigma(x) = \frac{1}{1 + e^{-x}}

✅ Final Result

So if you’re given a logit value $x$ , the sigmoid function:

\sigma(x) = \frac{1}{1 + e^{-x}}

turns it into a probability $p \in (0, 1)$ .

🔁 Why this is useful

In logistic regression, we compute:
$x = w^T x + b \quad \text{(a linear score)}$
Then use the sigmoid to convert that score (logit) into a probability of class 1:
$p = \frac{1}{1 + e^{-x}}$

This way, the model’s output is interpretable as a probability — suitable for binary classification.

Let me know if you want a plot or visualization to make it even clearer!

Wednesday, June 26, 2024

TinyML and Efficient Deep Learning Computing

https://hanlab.mit.edu/courses/2023-fall-65940

Efficient AI Computing,
Transforming the Future.

TinyML and Efficient Deep Learning Computing

6.5940

• Fall

• 2023

• https://efficientml.ai

About Logistics All Courses

This course focuses on efficient machine learning and systems. This is a crucial area as deep neural networks demand extraordinary levels of computation, hindering its deployment on everyday devices and burdening the cloud infrastructure. This course introduces efficient AI computing techniques that enable powerful deep learning applications on resource-constrained devices. Topics include model compression, pruning, quantization, neural architecture search, distributed training, data/model parallelism, gradient compression, and on-device fine-tuning. It also introduces application-specific acceleration techniques for large language models and diffusion models. Students will get hands-on experience implementing model compression techniques and deploying large language models (Llama2-7B) on a laptop.

Live Streaming:
https://live.efficientml.ai/
Time:
Tuesday/Thursday 3:35-5:00pm Eastern Time
Location:
36-156
Office Hour:
Thursday 5:00-6:00 pm Eastern Time, 38-344 Meeting Room
Discussion:
Piazza
Homework Submission:
Canvas
Contact:
- For external inquiries, personal matters, or emergencies, you can email us at efficientml-staff [at] mit.edu.
- If you are interested in getting updates, please sign up here to join our mailing list to get notified!

Instructor

Song Han

Associate Professor

Teaching Assistants

Han Cai

Ph.D

Ji Lin

Ph.D

Announcements

2023-12-15
Final project: reports, slides and demo videos

2023-12-14
Final report and course evaluation due

2023-11-09
Mid-term survey: https://forms.gle/xMgCohDLX73cd4af9

2023-10-31
Lab 5 is out.

2023-10-19
Lab 4 is out.

Schedule

Date

Lecture

Logistics

Sep 7

Lecture

Saturday, May 24, 2025

logit, sigmoid function, and probability

✅ Step 1: What is a logit?

✅ Step 2: What if we are given the logit and want to get the probability?

✅ Step 3: Exponentiate both sides

✅ Step 4: Solve for pp

✅ Step 5: Simplify into the sigmoid form

✅ Final Result

🔁 Why this is useful

Wednesday, June 26, 2024

TinyML and Efficient Deep Learning Computing

Efficient AI Computing,Transforming the Future.

TinyML and Efficient Deep Learning Computing

6.5940

•

Fall

•

2023

•

https://efficientml.ai

Instructor

Song Han

Teaching Assistants

Han Cai

Ji Lin

Announcements

Schedule

Date

Lecture

Logistics

Sep 7

Introduction

Sep 12

Basics of Deep Learning

Chapter I: Efficient Inference

Sep 14

Pruning and Sparsity (Part I)

Sep 19

Pruning and Sparsity (Part II)

Sep 21

Quantization (Part I)

Sep 26

Quantization (Part II)

Sep 28

Neural Architecture Search (Part I)

Oct 3

Neural Architecture Search (Part II)

Oct 5

Knowledge Distillation

Oct 10

Student Holiday — No Class

Oct 12

MCUNet: TinyML on Microcontrollers

Oct 17

TinyEngine and Parallel Processing

Chapter II: Domain-Specific Optimization

Oct 19

Transformer and LLM (Part I)

Oct 24

Transformer and LLM (Part II)

Oct 26

Vision Transformer

Oct 31

GAN, Video, and Point Cloud

Nov 2

Diffusion Model

Chapter III: Efficient Training

Nov 7

Distributed Training (Part I)

Nov 9

Distributed Training (Part II)

Nov 14

On-Device Training and Transfer Learning

Nov 16

Efficient Fine-tuning and Prompt Engineering

Nov 21

Basics of Quantum Computing

Nov 23

Thanksgiving — No Class

Chapter IV: Advanced Topics

✅ Step 4: Solve for $p$

Efficient AI Computing,
Transforming the Future.