4.1.1 Rank as the count of independent directions

The rank of a matrix A is the number of linearly independent rows, which equals the number of linearly independent columns. For an m by n matrix, rank is at most the smaller of m and n. Full rank for a square n by n matrix means rank n. Rank deficiency means at least one row or column is a linear combination of the others, so the matrix compresses space and destroys at least one dimension of information.

Geometrically, an n by n matrix maps n-dimensional vectors to n-dimensional vectors. Full rank preserves the full dimension. Rank r less than n collapses the image onto an r-dimensional subspace. The null space, the set of vectors x with A x equal to zero, then has dimension n minus r. This dimension count is the rank-nullity relation, and it governs everything about linear systems.

A near-singular matrix deserves attention because it appears in measurement-fitted models. Its rows are almost dependent, so its determinant is tiny but nonzero. Algebraically it remains invertible. Numerically it behaves almost like a singular matrix, amplifying small data errors into large solution errors. The ratio of largest to smallest singular value, called the condition number, quantifies this sensitivity. A large condition number warns that the computed solution may be dominated by noise.

Rank can be found by row reduction to echelon form. The number of nonzero pivots is the rank. Row operations of swapping, scaling by a nonzero constant, and adding a multiple of one row to another do not change the rank, because they do not change the row space. They only present the same space in a simpler basis.

4.1.2 Determinant: volume, orientation, and singularity

The determinant of a square matrix A, written det A or vertical bars around A, measures the signed volume scaling of the linear map. A unit cube is mapped to a parallelepiped whose signed volume equals det A. The sign records orientation. A positive determinant preserves orientation, a negative determinant reverses it, and a zero determinant flattens the cube into a lower-dimensional object of zero volume.

This volume picture motivates every row property. Swapping two rows swaps two edges of the parallelepiped and reverses orientation, so the determinant changes sign. Multiplying one row by a scalar k stretches one edge by k, so the volume scales by k. Adding a multiple of one row to another shears the parallelepiped without changing its volume, so the determinant is unchanged. These three properties together explain how row reduction affects the determinant: swaps contribute minus signs, scalings contribute factors, and shear eliminations contribute nothing.

From these properties follow the central product and scaling laws. The product rule states det(A B) equals det A times det B. The reasoning is compositional: B scales volumes by det B, then A scales the result by det A, so the combined map scales by the product. An immediate corollary is det(A transpose) equals det A, since row properties become column properties under transposition and the volume interpretation is symmetric. Another corollary is the scaling law for an n by n matrix: det(k A) equals k to the power n times det A, because every one of the n rows is stretched by k. This is a frequent source of error when k is factored out of a whole matrix rather than a single row.

Invertibility is decided by the determinant. A square matrix A is invertible exactly when det A is nonzero. In one direction, if A has an inverse, the product rule gives det A times det(A inverse) equals det of the identity, which is unity, so det A cannot be zero. In the other direction, if det A is nonzero, row reduction reaches the identity without a zero pivot, and the recorded operations assemble the inverse. A zero determinant means a zero pivot appears, a nontrivial null vector exists, and no inverse can undo the flattening.

4.1.3 Singularity and the zero eigenvalue

A scalar lambda is an eigenvalue of A with eigenvector v, nonzero by definition, when A v equals lambda v. Setting lambda to zero gives A v equals zero with v nonzero. That is precisely the statement that the null space is nontrivial, which is precisely the statement that A is singular. Hence a square matrix is singular exactly when zero is one of its eigenvalues. Equivalently, det A equals the product of all eigenvalues counting algebraic multiplicity, so det A vanishes exactly when at least one eigenvalue vanishes. This connection lets singularity be read off from the spectrum and lets the spectrum be sanity-checked against the determinant.

4.2.1 The characteristic equation

Eigenvalues are found from the condition that A minus lambda I must be singular, where I is the identity matrix. If A minus lambda I were invertible, the equation (A minus lambda I) v equals zero would force v to be zero, contradicting the requirement that eigenvectors are nonzero. Therefore lambda is an eigenvalue exactly when det(A minus lambda I) equals zero. This scalar equation in lambda is the characteristic equation, and the left side is the characteristic polynomial.

For a 2 by 2 matrix A with entries a, b on the first row and c, d on the second row, the computation is fully explicit:

det(A minus lambda I) equals det of the matrix with a minus lambda, b on the first row and c, d minus lambda on the second row, which equals (a minus lambda)(d minus lambda) minus b c.

Expanding gives lambda squared minus (a plus d) lambda plus (a d minus b c). The coefficient a plus d is the trace of A, written tr A, the sum of diagonal entries. The constant term a d minus b c is det A. Hence the characteristic equation is:

lambda squared minus tr(A) lambda plus det(A) equals zero.

This is not a mnemonic. It is the expanded determinant. Every 2 by 2 eigen computation is this quadratic.

4.2.2 Trace and determinant relations, proved for 2 by 2

Let the two roots of the quadratic, counting multiplicity and allowing complex values, be lambda1 and lambda2. A monic quadratic with these roots factors as (lambda minus lambda1)(lambda minus lambda2), which expands to lambda squared minus (lambda1 plus lambda2) lambda plus lambda1 lambda2. Matching coefficients with the characteristic polynomial gives the two relations:

lambda1 plus lambda2 equals tr A.

lambda1 lambda2 equals det A.

The sum of eigenvalues equals the trace, and the product equals the determinant. These relations are proved by direct comparison of coefficients, and they hold for repeated and complex-conjugate roots as well. They provide an instant check on any computed eigenvalues. If the sum or product disagrees with trace or determinant, the roots are wrong.

As an illustration inside the prose, take A with rows 4, 1 and 2, 3. The trace is 7 and the determinant is 12 minus 2, namely 10. The characteristic equation is lambda squared minus 7 lambda plus 10 equals zero, factoring as (lambda minus 5)(lambda minus 2). The roots 5 and 2 sum to 7 and multiply to 10, confirming the relations. The larger eigenvalue stretches its eigendirection by a factor of 5, the smaller by a factor of 2, and the area scaling 10 is their product.

For n by n matrices the same determinant defines a degree-n polynomial. The trace still equals the sum of all eigenvalues and the determinant still equals their product, counting algebraic multiplicity. The signs for intermediate coefficients alternate in the standard way, but the trace and determinant endpoints are the ones used most.

Quick example — eigenvalues via trace and determinant: For $A = [[5, 2], [2, 2]]$, $\mathrm{tr}(A) = 7$ and $\det(A) = 6$ give $\lambda^2 - 7\lambda + 6 = 0$, so $\lambda = 6$ and $\lambda = 1$, with sum $7$ and product $6$.

4.2.3 Cayley-Hamilton theorem and what it buys

The Cayley-Hamilton theorem states that every square matrix satisfies its own characteristic equation, with the matrix itself substituted for lambda and the constant term interpreted as a multiple of the identity. For 2 by 2:

A squared minus tr(A) A plus det(A) I equals the zero matrix.

The content is that the scalar relation among eigenvalues lifts to a matrix identity. Its value is computational. First, it reduces powers: A squared equals tr(A) A minus det(A) I with no matrix multiplication. Higher powers follow by repeated reduction, so every power of a 2 by 2 matrix collapses to a linear combination of A and I. Second, it gives the inverse directly. Moving terms gives A times (tr(A) I minus A) equals det(A) I, so when det A is nonzero:

A inverse equals (tr(A) I minus A) divided by det A.

This reproduces the classical adjugate formula for 2 by 2 matrices and explains why division by the determinant is unavoidable. When det A is zero the formula breaks down, exactly as it must, because no inverse exists.

4.2.4 Diagonalizable versus defective matrices

A square n by n matrix is diagonalizable when there exists a basis of eigenvectors, equivalently when n linearly independent eigenvectors exist. In that basis the matrix acts as independent scalings along each eigendirection, and powers, exponentials, and decoupled system responses become transparent. Diagonalizability holds automatically when all n eigenvalues are distinct, because eigenvectors for distinct eigenvalues are linearly independent.

A matrix is defective when it has fewer than n independent eigenvectors. The gap arises from repeated eigenvalues. The algebraic multiplicity of lambda is its multiplicity as a root of the characteristic polynomial. The geometric multiplicity is the dimension of the corresponding eigenspace, the number of independent eigenvectors for that lambda. Geometric multiplicity never exceeds algebraic multiplicity. Defectiveness means strict inequality for at least one eigenvalue.

The canonical illustration is a 2 by 2 matrix with a repeated eigenvalue whose eigenspace is one-dimensional. Only one independent eigendirection exists, so no eigenbasis exists. Such a matrix cannot be diagonalized, though it can be placed in Jordan form with a single Jordan block. The missing direction is supplied by a generalized eigenvector, and matrix functions then involve polynomial-times-exponential terms rather than pure exponentials. This is the linear-algebraic origin of the factor x multiplying e to the r x in critically damped systems and repeated-root differential equations. Recognizing defectiveness prevents the error of assuming two free eigenvector constants when only one eigendirection exists.

Symmetric matrices with real entries are never defective in this sense. They possess a full orthonormal eigenbasis with real eigenvalues. That is why vibration and inertia matrices, which are symmetric, admit clean modal decompositions.

4.3.1 Rank of A versus rank of the augmented matrix

The system A x equals b, with A an m by n coefficient matrix and b an m-vector, is analyzed through two ranks: the rank of A and the rank of the augmented matrix [A vertical bar b] formed by appending b as an extra column. Appending a column can either leave the rank unchanged or raise it by one. It can never raise it by more, and it can never lower it. This single observation yields the complete classification.

If rank[A vertical bar b] exceeds rank A, the vector b contributes a genuinely new direction outside the column space of A. The system is inconsistent and has no solution. Row reduction exposes a row of the form zero equals nonzero, an explicit contradiction.

If the two ranks are equal, b lies in the column space of A and at least one solution exists. Uniqueness then depends on whether the null space of A is trivial. If rank A equals n, the number of unknowns, the null space contains only zero, and the solution is unique. If rank A is less than n, the null space has dimension n minus rank A, a positive number, and infinitely many solutions exist: one particular solution plus arbitrary combinations of null-space vectors.

4.3.2 Geometric meaning of the three cases

Each algebraic case has a direct geometric picture. In two unknowns, each equation is a line. A unique solution is two nonparallel lines meeting at one point. Infinitely many solutions are coincident lines, the same constraint written twice. No solution is parallel distinct lines that never meet. In three unknowns, each equation is a plane. A unique solution is three planes meeting at a point. Infinitely many solutions are planes sharing a line or a common plane. No solution is an arrangement with empty common intersection, such as two parallel distinct planes or three planes pairwise intersecting with no triple point.

The homogeneous system A x equals zero always has the trivial solution. It has nontrivial solutions exactly when rank A is less than n, equivalently when A is square with zero determinant. The nonhomogeneous system inherits this structure: its full solution, when it exists, is one particular vector plus the entire homogeneous solution space. The null-space dimension counts the degrees of freedom.

As an illustration, a square 3 by 3 system with full rank 3 has a unique solution given by the inverse times b. If one equation is a combination of the others and b respects the same combination, the effective rank drops and a line or plane of solutions remains. If b violates the combination, the augmented rank exceeds the coefficient rank and the equations contradict one another. Checking ranks before computing avoids wasted elimination and identifies which physical constraint is redundant or inconsistent.

4.4.1 Maxima and minima theory

Let f be differentiable at an interior point xc of its domain. If f attains a local maximum or minimum at xc, then f prime at xc equals zero. The reasoning is that the difference quotient must be nonnegative from one side and nonpositive from the other when xc is extremal, forcing the two-sided derivative to vanish. This is Fermat necessary condition. It says candidates for interior extrema lie among critical points where the derivative vanishes or fails to exist, plus possibly the domain endpoints, which are not interior and must be checked separately.

The second-derivative test classifies critical points using curvature. If f prime at xc is zero and f double prime at xc is negative, the graph is concave down near xc and xc is a strict local maximum. If f double prime is positive, the graph is concave up and xc is a strict local minimum. If f double prime is zero, the test is inconclusive: xc may be a maximum, a minimum, or an inflection with a flat tangent, and the sign change of f prime across xc decides the matter. The exam-relevant value is f at xc, the extreme value itself, not merely its location, and global extrema on a closed interval require comparing all local candidates with endpoint values.

As an illustration, f(x) equals x cubed minus 3 x squared minus 9 x plus 5 has derivative 3 x squared minus 6 x minus 9, which factors as 3 times (x minus 3)(x plus 1). The critical points are x equals 3 and x equals minus 1. The second derivative 6 x minus 6 takes values 12 and minus 12 respectively, so x equals 3 is a local minimum with value minus 22 and x equals minus 1 is a local maximum with value 10. The sign pattern of the derivative, negative between the roots and positive outside, confirms the classification independently of the second derivative.

4.4.2 Rolle theorem and Lagrange mean value theorem

Rolle theorem: if f is continuous on the closed interval [a, b], differentiable on the open interval (a, b), and f(a) equals f(b), then there exists at least one c in (a, b) with f prime at c equal to zero. Intuition: a continuous curve that starts and ends at the same height and has no corners must turn around somewhere, producing a horizontal tangent. The equal-endpoint hypothesis forces the turnaround. Continuity alone is insufficient without differentiability, since a corner can supply the turnaround with no horizontal tangent, and differentiability alone is insufficient without continuity at the endpoints.

Lagrange mean value theorem: under the same continuity and differentiability hypotheses but without requiring equal endpoint values, there exists at least one c in (a, b) with f prime at c equal to (f(b) minus f(a)) divided by (b minus a). Intuition: the secant chord joining the endpoints has some slope, and somewhere the tangent is parallel to that chord. The proof applies Rolle theorem to an auxiliary function that subtracts the secant line from f, reducing the general slope to the zero-slope case. Geometrically the theorem guarantees that average slope over an interval is realized as instantaneous slope at an interior point. Analytically it bounds differences: the absolute difference of function values is at most the maximum derivative magnitude times the interval length, a workhorse estimate for error bounds and for proving that a bounded derivative implies uniform continuity.

4.4.3 Taylor series with remainder

Taylor expansion approximates a smooth function by its osculating polynomials. If f has n plus 1 derivatives near a point a, then:

f(x) equals f(a) plus f prime(a)(x minus a) plus f double prime(a)(x minus a) squared over 2 factorial plus continuing terms up to order n, plus a remainder Rn.

The Maclaurin form is the special case a equals zero. The standard series e to the x equals 1 plus x plus x squared over 2 factorial plus x cubed over 3 factorial and so on, and sin x equals x minus x cubed over 3 factorial plus x fifth over 5 factorial and so on, are Maclaurin expansions valid for all real x, with alternating signs in the sine series reflecting successive differentiation cycling through sine and cosine.

The remainder is what makes Taylor theory quantitative rather than formal. Lagrange form of the remainder states that Rn equals f to the (n plus 1)-th derivative at some intermediate point xi times (x minus a) to the power n plus 1 over (n plus 1) factorial. Meaning: the error of an n-th order truncation behaves like the next neglected power, scaled by a derivative somewhere between a and x. When derivatives grow slowly, few terms give high accuracy near a. The illustration e to the 0.1 approximated by 1 plus 0.1 plus 0.005 equals 1.105 differs from the true value by roughly the next term 0.1 cubed over 6, about 0.00017, exactly as the remainder predicts. Every finite-difference and quadrature rule in numerical methods is a Taylor polynomial with its remainder converted into an error order.

4.4.4 Partial derivatives, total differential, and chain rule

For z equals f(x, y), the partial derivative f subscript x is the ordinary derivative with y held fixed, and f subscript y holds x fixed. Geometrically these are slopes of surface cross-sections parallel to the coordinate planes. Second partials describe curvature in each direction and twist across directions. For smooth functions the mixed partials agree, so the order of differentiation does not matter.

The total differential dz equals f subscript x dx plus f subscript y dy is the linear part of the increment of f. Meaning: small input deviations dx and dy propagate to first order through the local slopes. This is the error-propagation law used when instrument tolerances pass through a formula. Dividing through by a parameter t gives the chain rule: dz over dt equals f subscript x times dx over dt plus f subscript y times dy over dt. Each path from t to z through x and through y contributes its slope product, and the total rate is the sum over paths. The multivariable chain rule generalizes this path-summing idea to any number of intermediate variables, and the total differential is its compact encoding.

4.5.1 Linear first-order equations and the integrating factor derived

The standard linear first-order form is:

dy over dx plus P(x) y equals Q(x),

where P and Q are known functions of x only and y is the unknown. The obstruction is that the left side is not visibly a single derivative because of the product-like term P y. The integrating factor removes this obstruction by multiplication with a weight mu(x) chosen so that the left side becomes an exact derivative.

Require mu times y prime plus mu P y to equal the derivative of (mu y). Expanding d(mu y) over dx by the product rule gives mu y prime plus mu prime y. Matching coefficients of y forces mu prime equals mu P, hence mu prime over mu equals P. Integrating gives ln mu equals the integral of P, so:

mu(x) equals e to the integral of P(x) dx.

Any nonzero constant multiple works, and the simplest choice takes the constant of integration as zero. With this mu, the equation becomes d(mu y) over dx equals mu Q, and one quadrature finishes the solution:

mu y equals the integral of mu Q dx plus C, then divide by mu.

As an illustration, y prime plus 2 y equals e to the x has P equals 2 and mu equals e to the 2 x. Multiplying gives e to the 2 x y prime plus 2 e to the 2 x y on the left, which is exactly d(y e to the 2 x) over dx, while the right side becomes e to the 3 x. Integration gives y e to the 2 x equals e to the 3 x over 3 plus C, so y equals C e to the minus 2 x plus e to the x over 3. The first term is the transient, decaying with the homogeneous rate minus 2, and the second term is the forced response tracking the excitation. Setting x to zero gives y(0) equals C plus one third, showing how the constant absorbs the initial condition.

Quick example — integrating-factor ODE: For $y^\prime + 3y = e^{2x}$, $\mu = e^{3x}$ gives $\frac{d}{dx}(y e^{3x}) = e^{5x}$, so $y e^{3x} = \frac{e^{5x}}{5} + C$ and $y = C e^{-3x} + \frac{e^{2x}}{5}$.

4.5.2 Constant-coefficient second order equations via the exponential ansatz

For a y double prime plus b y prime plus c y equals zero with constant a, b, c and a nonzero, the translation invariance of constant coefficients suggests exponential trial solutions, since differentiation reproduces exponentials up to a factor. Trying y equals e to the r x gives y prime equals r e to the r x and y double prime equals r squared e to the r x. Substitution and cancellation of the nonzero exponential yields the auxiliary equation:

a r squared plus b r plus c equals zero.

Each root r supplies a solution e to the r x, and the general solution combines them according to the discriminant D equals b squared minus 4 a c.

Distinct real roots (D positive): r1 not equal to r2, both real. The general solution is y equals C1 e to the r1 x plus C2 e to the r2 x. The two exponentials are independent, one decaying faster, and their combination describes overdamped return to equilibrium with no oscillation.

Repeated root (D zero): r1 equals r2 equals r. Only one exponential e to the r x arises directly, but a second-order equation needs two constants. The missing independent solution is x e to the r x, giving y equals (C1 plus C2 x) e to the r x. One way to see the extra x is reduction of order: seeking y equals u(x) e to the r x forces u double prime to vanish when r is double, so u is linear. Another way is to view the repeated root as the limit of two coalescing roots, where the difference quotient of the two exponentials tends to x e to the r x. Forgetting the factor x is the standard error in this case.

Complex conjugate pair (D negative): r equals alpha plus or minus i beta. Euler formula converts the complex exponentials into real oscillation: y equals e to the alpha x times (C1 cos beta x plus C2 sin beta x). The envelope e to the alpha x grows or decays according to the sign of alpha, while beta sets the oscillation frequency. Stability of the homogeneous response requires alpha negative, so both roots lie in the left half-plane. This is the underdamped regime and the algebraic reason natural frequency and damping ratio appear in vibration analysis.

4.5.3 Resonance preview for forced equations

With a forcing term on the right, the full solution is homogeneous plus particular. When the forcing is exponential, sinusoidal, or polynomial, the method of undetermined coefficients tries a particular of the same family with unknown multipliers. Resonance occurs when the trial particular duplicates a homogeneous solution, which happens exactly when the forcing exponent matches a root of the auxiliary equation. The remedy is to multiply the trial by x, or by x squared for a double root, restoring independence. Physically this is why excitation at the natural frequency produces a response growing linearly with time in the undamped model: each cycle deposits energy in phase with the motion. Damping caps the growth at a large but finite steady amplitude, and the linear-in-time growth is the transient signature of exact resonance.

4.6.1 Definition and operating logic

For t greater than or equal to zero, the Laplace transform of f is:

F(s) equals the integral from zero to infinity of e to the minus s t f(t) dt,

where defined by convergence of the improper integral. The transform is linear: constants factor out and sums transform termwise. Its operating logic is algebraization: differentiation in t becomes multiplication by s (minus initial-condition terms), and convolution in t becomes multiplication in s. Constant-coefficient differential equations with initial conditions therefore become algebraic equations in s, solved by division, then returned to t by inversion using a table of pairs plus the shift theorems. Existence requires roughly exponential order, meaning f grows no faster than some exponential, which covers all standard engineering signals.

4.6.2 The eight core pairs derived or motivated

Unit constant 1 transforms to 1 over s. Direct integration gives the integral of e to the minus s t from zero to infinity, which is 1 over s for s positive. This is the seed of the whole table.

Power t to the n, for nonnegative integer n, transforms to n factorial over s to the n plus 1. Repeated integration by parts peels one power of t at a time, each step bringing down a factor and raising the power of s in the denominator. The case n equals zero reproduces the unit-constant pair.

Exponential e to the a t transforms to 1 over (s minus a). The integrand becomes e to the minus (s minus a) t, so the same computation as the constant case applies with s shifted to s minus a. Convergence needs s greater than a, the abscissa of convergence.

Sine sin(b t) transforms to b over (s squared plus b squared). Cosine cos(b t) transforms to s over (s squared plus b squared). One derivation writes the sinusoids via complex exponentials and uses the exponential pair: the transform of e to the i b t is 1 over (s minus i b), whose real and imaginary parts are s and b over s squared plus b squared. Equivalently, transforming the differential equation y double prime plus b squared y equals zero with appropriate initial data yields the same denominators. The shared denominator s squared plus b squared is the algebraic fingerprint of undamped oscillation at frequency b.

Hyperbolic sine sinh(b t) transforms to b over (s squared minus b squared). Since sinh is half the difference of e to the b t and e to the minus b t, linearity plus the exponential pair gives half of 1 over (s minus b) minus 1 over (s plus b), which simplifies to the stated form. The sign change in the denominator relative to sine reflects growth rather than oscillation.

Delayed unit step u(t minus c), equal to zero before t equals c and unity after, transforms to e to the minus c s over s. The integral runs effectively from c to infinity, and substituting tau equals t minus c factors out e to the minus c s, leaving the unit-constant integral. Delay in time becomes exponential multiplication in s.

Dirac impulse delta(t) at zero transforms to 1. Heuristically the impulse has unit area concentrated at the origin, so the integral of e to the minus s t against it samples the exponential at zero, which is unity. Formally the impulse is the limit of tall narrow unit-area pulses, each of whose transforms tends to 1. The impulse is the identity element of convolution, and its transform 1 is the identity of multiplication, exactly as the convolution theorem demands.

4.6.3 Shift theorems with reasoning

The s-shift theorem states that L of e to the a t f(t) equals F(s minus a). Reasoning: inserting e to the a t into the defining integral merges with e to the minus s t to give e to the minus (s minus a) t, so the whole computation repeats at the shifted frequency variable. Effect: damping or growth in time slides the s-domain picture. As an illustration, starting from L of sin 2 t equals 2 over (s squared plus 4) and multiplying by e to the 3 t shifts every s to s minus 3, giving 2 over ((s minus 3) squared plus 4).

The t-shift theorem states that L of u(t minus c) g(t minus c) equals e to the minus c s G(s). Reasoning: the delayed waveform is the original shifted right by c, so substituting tau equals t minus c translates the integral and leaves the factor e to the minus c s. Effect: delay in time is phase-like multiplication in s. The argument of g must be written in shifted form t minus c before reading off G, otherwise the delay and the waveform shape are misaligned. Together the two shifts generate damped sinusoids, delayed switch-ons, and piecewise excitations from the eight seed pairs.

4.7.1 Axioms and the meaning of probability

Probability assigns numbers to events, subsets of a sample space S, obeying the Kolmogorov axioms. First, nonnegativity: P(E) is greater than or equal to zero for every event E. Second, normalization: P(S) equals unity, so certainty has probability one. Third, countable additivity: for pairwise disjoint events, the probability of their union is the sum of their probabilities. From these follow the complement rule P(not E) equals 1 minus P(E), monotonicity (larger events have larger probability), and the addition law P(E union F) equals P(E) plus P(F) minus P(E intersect F), subtracting the double-counted overlap. The axioms say nothing about physics. They constrain any consistent quantification of uncertainty, whether interpreted as long-run frequency or as degree of belief.

4.7.2 Conditional probability and independence

Conditional probability restricts the universe. For P(F) positive:

P(E given F) equals P(E intersect F) divided by P(F).

Meaning: discard everything outside F, renormalize F to probability one, and measure E within it. Rearranged, this is the product rule P(E intersect F) equals P(E given F) P(F), symmetric in E and F. Events E and F are independent when conditioning is irrelevant: P(E given F) equals P(E), equivalently P(E intersect F) equals P(E) P(F). Independence is a property of the probability measure, not of causal unrelatedness in the colloquial sense, and assuming it without justification is a persistent modelling error.

4.7.3 Bayes theorem derived

Let H and not H partition the possibilities, and let E be observed evidence. From the product rule applied two ways, P(H intersect E) equals P(E given H) P(H) and also equals P(H given E) P(E). Equating and dividing by P(E) gives:

P(H given E) equals P(E given H) P(H) divided by P(E).

Expanding the denominator by total probability, P(E) equals P(E given H) P(H) plus P(E given not H) P(not H), yields the familiar form:

P(H given E) equals P(E given H) P(H) divided by [P(E given H) P(H) plus P(E given not H) P(not H)].

Vocabulary: P(H) is the prior before seeing E, P(E given H) is the likelihood of the evidence under H, the denominator is the total evidence, and P(H given E) is the posterior after seeing E. Bayes theorem flips the conditioning direction. Its lesson is that a highly accurate test can still produce mostly false positives when the prior prevalence is small, because the absolute number of healthy subjects dwarfs the absolute number of ill subjects. As an illustration, with 1 percent prevalence, 95 percent sensitivity, and 90 percent specificity, the posterior after a positive result is 0.95 times 0.01 divided by (0.95 times 0.01 plus 0.10 times 0.99), about 0.088. Fewer than one in eleven positives is a true case despite the test being 95 percent sensitive, because the prior dominates.

Quick example — Bayes flip: With prior $0.02$, sensitivity $0.99$, and specificity $0.95$, $P(\text{pos}) = 0.99 \times 0.02 + 0.05 \times 0.98 = 0.0688$, so $P(\text{ill} \mid \text{pos}) = \frac{0.0198}{0.0688} \approx 0.288$.

4.7.4 Binomial, Poisson, and normal models

The binomial model counts successes in n independent trials with common success probability p. Its mean is n p and its variance is n p (1 minus p). Shape: for p near one half the distribution is nearly symmetric; for extreme p it skews toward the nearer endpoint. Variance is maximal at p equals one half and vanishes at certainty.

The Poisson model counts rare events occurring at mean rate m over a fixed exposure. Its mass function is P(k) equals e to the minus m m to the k over k factorial for k equals 0, 1, 2, and so on. Both mean and variance equal m. Shape: skewed right for small m, increasingly symmetric as m grows. The Poisson arises as the limit of the binomial when n is large and p is small with n p held at m: many nearly impossible opportunities produce a finite rate. It is the model for defect counts, arrivals, and breakdowns per interval when events occur singly and independently at a steady average rate.

The normal model is the symmetric bell curve parametrized by mean mu and standard deviation sigma. Its density is highest at mu and falls off as exp(minus (x minus mu) squared over 2 sigma squared), scaled to unit total area. By the central limit theorem, averages and sums of many small independent contributions are approximately normal regardless of the underlying distribution, which is why measurement error and aggregate variation so often look Gaussian.

4.7.5 The 68-95-99.7 rule and standard error

For a normal variable, about 68 percent of probability lies within mu plus or minus one sigma, about 95 percent within mu plus or minus two sigma (more precisely 1.96 sigma), and about 99.7 percent within mu plus or minus three sigma. Meaning: one sigma is the typical deviation scale, two sigma is the rough bounds of usual variation, and three sigma encloses essentially everything except rare outliers. The bands are nested areas under the bell curve, widest in probability mass near the centre and thinning in the tails.

The standard error concerns the mean of n independent observations with common standard deviation sigma. The sample mean has the same expectation mu but its variance is sigma squared over n, so its standard deviation, called the standard error, is sigma over root n. Intuition: independent errors partly cancel in averaging, but only at the root-n rate because variances add while deviations scale as the square root. Averaging four readings halves the uncertainty; averaging one hundred readings cuts it tenfold. This square-root law explains why repeated measurement works and why further replication has diminishing returns. Combining the two ideas, the interval sample mean plus or minus two standard errors is an approximate 95 percent confidence interval for mu under normality.

4.8.1 Quadrature: why trapezoidal and Simpson rules work

Numerical quadrature approximates the integral of f over [a, b] by integrating a simple interpolant fitted through sampled ordinates. Divide [a, b] into n equal strips of width h equals (b minus a) over n with ordinates y0 through yn at the nodes.

The trapezoidal rule joins consecutive ordinates by straight chords and sums the resulting trapezoids:

I approx equals (h over 2) times [y0 plus 2(y1 plus continuing plus y_{n minus 1}) plus yn].

Interior ordinates are shared by two adjacent trapezoids, hence the weight 2, while endpoints belong to one. The rule is exact for linear f because chords then coincide with the graph. For curved f, each strip erring follows local curvature, and Taylor expansion shows the global error scales as h squared: halving h quarters the error. The rule systematically overestimates on concave-down portions and underestimates on concave-up portions.

Simpson one-third rule fits parabolas through successive triples of points instead of chords:

I approx equals (h over 3) times [y0 plus 4 y1 plus 2 y2 plus 4 y3 plus continuing plus 4 y_{n minus 1} plus yn].

The alternating 4, 2 pattern reflects overlapping parabolic fits: odd-indexed ordinates are midpoints of their panels with weight 4, even-indexed interior ordinates are shared panel endpoints with weight 2. With three ordinates (n equals 2) this reduces to (h over 3)(y0 plus 4 y1 plus y2). As an illustration, the integral of x squared from 0 to 2 is exactly 8 over 3. With h equals 1, the trapezoidal estimate is (1 over 2)(0 plus 2 plus 4) equals 3.0, while Simpson gives (1 over 3)(0 plus 4 plus 4) equals 8 over 3, exact. The chord overestimates the convex parabola, while the fitted parabola reproduces it.

Quick example — Simpson with three ordinates: For $\int_0^2 x^3 dx = 4$, $h = 1$ with $y_0 = 0$, $y_1 = 1$, $y_2 = 8$ gives $\frac{1}{3}(0 + 4 + 8) = 4$, so the parabola fit is exact on this cubic.

4.8.2 Why Simpson needs even n and why it is exact through cubics

Each parabolic panel spans two strips (three points determine a parabola), so the panels tile [a, b] only when n is even. An odd n leaves one strip uncovered, and no consistent 4-2 weighting exists. Hence Simpson one-third rule requires n even.

Exactness through cubics is the deeper fact. A parabola fit would seem exact only through quadratics, yet Simpson integrates cubics exactly as well. Reason: over a symmetric two-strip panel centred at its midpoint, the cubic component of f is antisymmetric (odd) about the centre, so its positive and negative lobes cancel exactly in the true integral, and the Simpson weights, symmetric about the same centre, reproduce that cancellation. The leading error therefore comes from the fourth derivative, and the global error scales as h to the fourth: halving h cuts the error sixteenfold for smooth f. Cubic exactness is why Simpson handled the x-cubed illustration exactly and why it converges so much faster than the trapezoidal rule on smooth integrands.

4.8.3 Newton-Raphson iteration: geometric derivation and caveats

To solve f(x) equals zero, start from a guess xk and linearize. The tangent to the graph at (xk, f(xk)) has slope f prime(xk) and equation y equals f(xk) plus f prime(xk)(x minus xk). Its x-intercept, where y equals zero, is the natural improved guess:

x_{k plus 1} equals xk minus f(xk) over f prime(xk).

Geometrically each step slides down the tangent line to the axis and re-evaluates. Near a simple root with nonzero derivative, the error roughly squares each iteration (quadratic convergence), so correct digits approximately double, as seen when iterating on x squared minus 2 from 1: 1.5, then about 1.4166667, then about 1.4142157, closing rapidly on root 2.

Convergence carries caveats. The formula divides by f prime(xk), so a flat tangent with near-zero derivative throws the next iterate far away. A poor starting point can cycle, diverge to infinity, or converge to an unintended root when several exist. Oscillatory or kinked functions violate the local-linearity assumption on which the tangent step rests. Multiple roots, where f and f prime vanish together, reduce convergence to linear because the graph kisses rather than crosses the axis. The practical safeguards are to start near the target root, monitor the step size and residual, and fall back to bracketing methods when the derivative is small or the iterates misbehave.

Quick example — Newton-Raphson iterations: For $f(x) = x^2 - 3$ from $x_0 = 2$, $x_1 = 2 - \frac{1}{4} = 1.75$ and $x_2 = 1.75 - \frac{0.0625}{3.5} \approx 1.7321$, closing on $\sqrt{3} \approx 1.7321$.

4.8.4 Euler method for initial-value problems

For dy over dx equals f(x, y) with initial value y(x0) equal to y0, Euler method advances in steps of size h along the local tangent:

y_{n plus 1} equals yn plus h f(xn, yn), with x_{n plus 1} equals xn plus h.

Each step is one tangent-line hop: freeze the slope at the current point and walk it forward over width h. The local truncation error per step scales as h squared (the first neglected Taylor term), and accumulating over (interval length over h) steps gives global error of order h. Halving h halves the error but doubles the work. The method is first-order, simple, and the conceptual ancestor of all Runge-Kutta schemes, which sample additional slopes within each step to cancel higher Taylor terms and achieve higher order at the cost of more function evaluations per step.

• Rank counts independent directions and, together with the augmented rank, classifies every linear system as unique, infinite, or inconsistent, with a matching lines-and-planes picture.
• Determinant is signed volume scaling: row swaps flip the sign, single-row scaling scales it, shears leave it unchanged, det(A B) equals det A det B, det(k A) equals k to the n det A for n by n, and singularity is exactly det zero, equivalently a zero eigenvalue.
• Eigenvalues of 2 by 2 matrices come from lambda squared minus trace lambda plus determinant equals zero, with sum equal to trace and product equal to determinant by coefficient matching; Cayley-Hamilton turns this into matrix identities for fast powers and the inverse; missing eigenvectors mean defectiveness, with Jordan structure and x-times-exponential terms replacing a full eigenbasis.
• Interior extrema force zero derivative, curvature classifies them, and endpoints must be checked separately; Rolle theorem forces a horizontal tangent between equal heights, and Lagrange mean value theorem realizes average slope as an interior tangent slope with quantitative difference bounds.
• Taylor polynomials approximate with Lagrange remainder controlled by the next derivative power; the total differential dz equals f subscript x dx plus f subscript y dy propagates small errors, and the chain rule sums slope products over all paths.
• First-order linear equations fall to the integrating factor mu equals exp(integral P dx), derived by forcing the left side into d(mu y) over dx; constant-coefficient second-order equations follow the exponential ansatz with overdamped, critically damped (with the extra x), and underdamped envelope-oscillation cases; forcing at a natural frequency creates resonance with linearly growing particular response.
• Laplace transformation converts differentiation to multiplication by s: the eight pairs for constant, power, exponential, sine, cosine, hyperbolic sine, delayed step, and impulse follow from direct integration, complex exponentials, and limiting pulses, while s-shift slides the picture for damping and t-shift multiplies by exp(minus c s) for delay.
• Probability obeys nonnegativity, normalization, and additivity; conditioning renormalizes to the known event, Bayes theorem inverts conditioning through prior times likelihood over total evidence, binomial counts trials, Poisson counts rare events with mean equal to variance, normal describes aggregate variation with 68-95-99.7 bands, and averaging shrinks uncertainty as sigma over root n.
• Quadrature integrates interpolants: trapezoidal sums chords with h-squared error, Simpson sums parabolas on even n with h-fourth error and exactness through cubics by symmetric cancellation; Newton-Raphson slides down tangents with quadratic convergence near simple roots subject to flat-derivative, bad-start, and multiple-root caveats; Euler method hops along tangents with first-order global error.