Derivatives Explained: From Definition to Application

The derivative measures how fast a function is changing at a single point. If $f(x)$ describes the position of a car at time $x$, then $f^{\prime }(x)$ is the car's speed at time $x$. If $f(x)$ is the cost of producing $x$ units of a product, then $f^{\prime }(x)$ is the marginal cost — the cost of producing one more unit when you are already producing $x$. If $f$ is the loss function of a neural network as a function of its parameters, then the derivative tells you how to adjust the parameters to reduce the loss.

This article covers what a derivative actually measures, the formal limit definition with a worked example, all the differentiation rules you will use daily (power, product, quotient, chain), the table of derivatives of common functions, implicit differentiation, higher-order derivatives, and applications in physics, economics, and machine learning.

What a derivative actually measures

The intuitive picture: at any point on the graph of a function, draw the tangent line — the straight line that touches the curve at exactly that point. The slope of this tangent line is the derivative at that point.

Why this matters. The slope tells you how steeply the function is changing. A large positive slope means the function is increasing rapidly. A negative slope means it is decreasing. A slope of zero means the function is momentarily flat — a peak, a valley, or an inflection point.

Concretely: if $f(x)=x^2$, then $f^{\prime }(x)=2x$. At $x=3$, the slope is 6. This means: in a neighborhood of $x=3$, when $x$ increases by a small amount $\Delta x$, the function increases by approximately $6\Delta x$. The derivative is the rate of change, and it predicts the function's behavior locally.

The derivative is the foundation of nearly every quantitative discipline. It captures the idea of change at an instant — which is at the core of physics (motion), economics (marginal quantities), engineering (rates), biology (growth), and machine learning (loss optimization).

The formal limit definition

The slope of a line through two points $(x,f(x))$ and $(x+h,f(x+h))$ is:

$\frac{f(x+h)-f(x)}{h}$

This is the average rate of change of $f$ over the interval from $x$ to $x+h$. It is the slope of a secant line — a line through two points on the curve.

The derivative is what happens when $h$ shrinks to zero — the secant line becomes a tangent line:

$f^{\prime }(x)={\lim }_{h\to 0}\frac{f(x+h)-f(x)}{h}$

This is the formal definition. Every other formula for derivatives is derived from it.

Example 1: Compute $f^{\prime }(x)$ from the definition when $f(x)=x^2$.

$f^{\prime }(x)={\lim }_{h\to 0}\frac{(x+h{)}^2-x^2}{h}={\lim }_{h\to 0}\frac{x^2+2xh+h^2-x^2}{h}={\lim }_{h\to 0}\frac{2xh+h^2}{h}={\lim }_{h\to 0}(2x+h)=2x$

So $f^{\prime }(x)=2x$, confirming the rule.

Example 2: Compute $f^{\prime }(x)$ from the definition when $f(x)=\frac{1}{x}$.

$f^{\prime }(x)={\lim }_{h\to 0}\frac{\frac{1}{x+h}-\frac{1}{x}}{h}={\lim }_{h\to 0}\frac{x-(x+h)}{h\cdot x(x+h)}={\lim }_{h\to 0}\frac{-h}{h\cdot x(x+h)}={\lim }_{h\to 0}\frac{-1}{x(x+h)}=-\frac{1}{x^2}$

So $\frac{d}{dx}\left(\frac{1}{x}\right)=-\frac{1}{x^2}$.

Working from the definition is tedious. In practice you use the rules below.

The differentiation rules

These rules allow you to compute the derivative of almost any function algebraically, without going back to the limit definition.

The power rule:

$\frac{d}{dx}(x^n)=n{x}^{n-1}$

for any real number $n$.

The sum and constant rules:

$\frac{d}{dx}(f+g)=f^{\prime }+g^{\prime }$ $\frac{d}{dx}(cf)=c{f}^{\prime }$

for any constant $c$.

The product rule:

$\frac{d}{dx}(fg)=f^{\prime }g+f{g}^{\prime }$

The quotient rule:

$\frac{d}{dx}\left(\frac{f}{g}\right)=\frac{f^{\prime }g-f{g}^{\prime }}{g^2}$

The chain rule:

$\frac{d}{dx}(f(g(x)))=f^{\prime }(g(x))\cdot g^{\prime }(x)$

The chain rule is the most important rule. It tells you how to differentiate composed functions. In machine learning, the chain rule applied recursively is exactly the backpropagation algorithm.

Example 3: Differentiate $f(x)=(3x^2+1{)}^5$.

Apply the chain rule. Let $u=3x^2+1$, so $f=u^5$. Then $f^{\prime }=5u^4\cdot u^{\prime }=5(3x^2+1{)}^4\cdot 6x=30x(3x^2+1{)}^4$.

Example 4: Differentiate $f(x)=x^3\sin x$.

Apply the product rule:

$f^{\prime }(x)=3x^2\sin x+x^3\cos x$

Example 5: Differentiate $f(x)=\frac{e^x}{x^2+1}$.

Apply the quotient rule:

$f^{\prime }(x)=\frac{e^x(x^2+1)-e^x\cdot 2x}{(x^2+1{)}^2}=\frac{e^x(x^2-2x+1)}{(x^2+1{)}^2}=\frac{e^x(x-1{)}^2}{(x^2+1{)}^2}$

Example 6 — multiple rules: Differentiate $f(x)=\sin (x^2)\cdot e^{3x}$.

Product rule on the outside, chain rule on each factor:

$f^{\prime }(x)=\cos (x^2)\cdot 2x\cdot e^{3x}+\sin (x^2)\cdot 3e^{3x}=e^{3x}(2x\cos (x^2)+3\sin (x^2))$

Table of common derivatives

These should be memorized.

Implicit differentiation

Sometimes a function is defined implicitly by an equation that cannot be solved for $y$ in terms of $x$. Example: $x^2+y^2=25$ (the unit circle scaled).

To find $\frac{dy}{dx}$ at a point on this curve, differentiate both sides with respect to $x$, treating $y$ as a function of $x$:

$2x+2y\frac{dy}{dx}=0$

Solve for $\frac{dy}{dx}$:

$\frac{dy}{dx}=-\frac{x}{y}$

At the point $(3,4)$: $\frac{dy}{dx}=-\frac{3}{4}$. (You can verify this is the slope of the tangent to the circle $x^2+y^2=25$ at this point.)

Implicit differentiation is essential for problems where $y$ is locked inside the equation — common in physics (constraints) and economics (utility curves).

Higher-order derivatives

The derivative of a derivative is the second derivative: $f^{\prime \prime }(x)=\frac{d}{dx}(f^{\prime }(x))$. Geometrically, it measures how the slope itself is changing — the curvature of the function.

In physics: position $\to$ velocity (first derivative) $\to$ acceleration (second derivative).

In optimization: the second derivative tells you whether a critical point (where $f^{\prime }=0$) is a minimum (where $f^{\prime \prime }>0$), a maximum (where $f^{\prime \prime }<0$), or an inflection point (where $f^{\prime \prime }=0$).

Higher-order derivatives appear in Taylor series, in physics (jerk is the third derivative of position), and in numerical analysis (Newton's method uses both first and second derivatives).

Applications

Physics

The most direct application. Newton's second law: $F=ma$, where $a=\frac{d^{2}x}{d{t}^2}$. Any equation of motion is a differential equation involving derivatives of position with respect to time.

Economics

Marginal quantities. If $C(q)$ is the total cost of producing quantity $q$, then $C^{\prime }(q)$ is the marginal cost — the cost of producing one more unit at the current production level. Marginal revenue, marginal utility, marginal product of labor — all are derivatives.

Optimization

To find the maximum or minimum of a function, set the derivative equal to zero and solve. Then use the second derivative to determine whether the critical point is a maximum, minimum, or saddle point.

Example 7: A box with no top is to be made from a square piece of cardboard 12 inches on a side by cutting squares of side $x$ from each corner and folding up the sides. What value of $x$ maximizes the volume?

The volume is $V(x)=x(12-2x{)}^2$. Set $V^{\prime }(x)=0$:

$V^{\prime }(x)=(12-2x{)}^2+x\cdot 2(12-2x)(-2)=(12-2x)((12-2x)-4x)=(12-2x)(12-6x)$

Setting this to zero gives $x=6$ or $x=2$. The first makes the box have zero volume; the second gives the maximum. So $x=2$ inches.

Machine learning

Gradient descent. The optimization algorithm that trains nearly every neural network. Given a loss function $L(\mathbf{\theta })$ as a function of model parameters $\mathbf{\theta }$, gradient descent updates the parameters by

$\mathbf{\theta }\leftarrow \mathbf{\theta }-\alpha \nabla L(\mathbf{\theta })$

where $\alpha$ is a learning rate and $\nabla L$ is the gradient — the vector of partial derivatives of $L$ with respect to each parameter.

Backpropagation. The algorithm that computes the gradient of the loss with respect to every parameter in a neural network. It is the chain rule of calculus applied recursively, layer by layer, from the output back to the input. Every parameter update in deep learning is a chain rule computation.

Common mistakes

Mistake 1 — Confusing $f(x)$ with $f^{\prime }(x)$ in the chain rule. When you compute $\frac{d}{dx}(f(g(x)))$, the rule is $f^{\prime }(g(x))\cdot g^{\prime }(x)$, not $f^{\prime }(g^{\prime }(x))$. The outer derivative is evaluated at $g(x)$, not at $g^{\prime }(x)$.

Mistake 2 — Forgetting the product rule when both factors involve $x$. $\frac{d}{dx}(x\cdot \sin x)$ is not $\sin x$. You have to use the product rule: $\sin x+x\cos x$.

Mistake 3 — Treating $\frac{dy}{dx}$ as a fraction. It is not a fraction in the strict sense. You cannot generally cancel $dx$ in the denominator with a $dx$ elsewhere. (There is a more advanced treatment in differential forms, but for the level here, treat $\frac{dy}{dx}$ as a single notation.)

Mistake 4 — Computing $\frac{d}{dx}(x^x)$ with the power rule. The power rule applies to $x^n$ where $n$ is a constant. For $x^x$, you have to take the log first: $\ln (x^x)=x\ln x$, then differentiate implicitly.