Lab U2: Understanding a Linear Map

Unit: Unit 2, Anatomy of a Linear Map
Textbook sections: 2.1 Understanding a linear map: a preview; 2.2 Linear systems; 2.3 Subspaces; 2.4 Independence, bases, and rank; 2.5 Inverses and determinants; 2.6 Applications and computation recap
Core path: projection, reachable outputs, row reduction, plane intersections, homogeneous systems, null-space directions, difference matrices, redundant features, affine-layer bias, rank and nullity, determinants, and inverse checks
This lab is about reading short NumPy and SymPy snippets as linear algebra. The goal is not to write long programs. The goal is to predict small outputs, identify shapes, translate code into mathematics, and explain what the computation says about the linear map (\mathbf{x}\mapsto A\mathbf{x}).

Part 0 is a warm-up. Parts 1-10 are core. The final part contains review code-reading checks with answer sketches.

The main questions are object, operation, and output; the prediction and interpretation questions below keep returning to those three ideas.

Computational tools used in this lab

Before starting, review these parts of Appendix B, NumPy and SymPy Quick Reference for the Labs:

• Appendix B.2: NumPy arrays, vectors, matrices, and shapes
• Appendix B.4: Elementwise arithmetic versus linear algebra
• Appendix B.9: Numerical checks and roundoff
• Appendix B.10: NumPy linear algebra commands
• Appendix B.11: SymPy symbolic computation

The goal is to interpret the mathematical computation, not to memorize every command.

Part 0. NumPy and SymPy warm-up: shapes, @, and rref

Math reminder. In NumPy, A @ x means matrix-vector multiplication when A is a matrix and x is a compatible vector. In SymPy, .rref() row-reduces exactly and also reports pivot columns.

Predict before running. What is the shape of A0? What is the shape of x0? Is A0 @ x0 a dot product, a matrix-vector product, or entry-by-entry multiplication? What should the second row of the rref of M0 become?

import numpy as np
import sympy as sp

np.set_printoptions(precision=3, suppress=True)

A0 = np.array([
    [1.0, 2.0],
    [3.0, 4.0],
])
x0 = np.array([1.0, -1.0])

M0 = sp.Matrix([
    [1, 2, 3],
    [2, 4, 6],
])

A0.shape, x0.shape, A0 @ x0, M0.rref()

Run and compare. A0 has shape (2,2), x0 has shape (2,), and A0 @ x0 is a matrix-vector product. The second row of the rref of M0 is zero because the second row is a multiple of the first.

Interpretation check. Row reduction detected redundant information. The two rows of M0 impose only one independent linear condition.

Common mistake. The output of .rref() is a pair: the row-reduced matrix and the tuple of pivot columns.

Part 1. A map that forgets height

Math reminder. The projection (P:\mathbb R^3\to\mathbb R^2) keeps the first two coordinates and forgets the third coordinate, interpreted here as height: [ P(x,y,z)=(x,y). ] A nonzero vector (\mathbf{z}) with (P\mathbf{z}=\mathbf{0}) is a forgotten direction.

Predict before running. Which two inputs below should have the same output? Which nonzero input should go to zero?

P = np.array([
    [1, 0, 0],
    [0, 1, 0],
])

x1 = np.array([2, 1, 0])
x2 = np.array([2, 1, 5])
z = np.array([0, 0, 1])

P @ x1, P @ x2, P @ z

Run and compare. The outputs are [2, 1], [2, 1], and [0, 0].

Interpretation check. The inputs represented by x1 and x2 differ only in the third coordinate. The vector represented by z is invisible to (P), so the output does not determine the original input uniquely.

Common mistake. A zero output does not mean the input was zero. It can mean the input lies in the null space.

Part 2. Reachable outputs

Math reminder. The equation (A\mathbf{x}=\mathbf{b}) asks whether the target output (\mathbf{b}) is reachable. For the matrix below, [ A\mathbf{x}=(x_1,x_2,x_1+x_2). ] So a target (\mathbf{b}=(b_1,b_2,b_3)) is reachable exactly when (b_3=b_1+b_2).

Predict before running. Which target below is reachable: b_good or b_bad? Which augmented matrix should produce an inconsistent row?

A = np.array([
    [1, 0],
    [0, 1],
    [1, 1],
])

b_good = np.array([1, 2, 3])
b_bad = np.array([1, 2, 4])

M_good = sp.Matrix([
    [1, 0, b_good[0]],
    [0, 1, b_good[1]],
    [1, 1, b_good[2]],
])

M_bad = sp.Matrix([
    [1, 0, b_bad[0]],
    [0, 1, b_bad[1]],
    [1, 1, b_bad[2]],
])

M_good.rref(), M_bad.rref()

Run and compare. The first target is reachable because (3=1+2). The second target is not reachable because (4\neq 1+2). The rref for the second augmented matrix has a pivot in the last column, which represents an impossible equation.

Interpretation check. The column space of (A) is the plane in (\mathbb R^3) described by (b_3=b_1+b_2).

Common mistake. A target vector can have the correct size and still be unreachable.

Part 3. Row reduction and solution sets

Math reminder. A linear system has no solution, one solution, or infinitely many solutions. Row reduction tells which case occurs.

Predict before running. The three systems below are small variants of the same setup. Which should have one solution? Which should have infinitely many solutions? Which should have no solution?

systems = {
    "one solution": sp.Matrix([
        [1, 1, 18],
        [2, 4, 56],
    ]),
    "infinitely many solutions": sp.Matrix([
        [1, 1, 18],
        [4, 4, 72],
    ]),
    "no solution": sp.Matrix([
        [1, 1, 18],
        [4, 4, 56],
    ]),
}

for name, M in systems.items():
    R, pivots = M.rref()
    print("\n" + name)
    print(R)
    print("pivot columns:", pivots)

Run and compare. The first system has one solution. The second has a zero row after reduction, so it has infinitely many solutions. The third has a pivot in the last column, so it is inconsistent.

Interpretation check. A free variable means a family of solutions. A pivot in the augmented column means no solution.

Common mistake. The number of equations alone does not decide the number of solutions. Redundancy and inconsistency matter.

Part 4. Vector geometry and systems

Math reminder. A line in (\mathbb R^3) can appear as the intersection of two planes. Solving the two plane equations is solving a linear system.

The planes are [ x+y+z=1,\qquad -x+2y-3z=-1. ]

Predict before running. Should the solution set be a point, a line, a plane, or empty? Which variable do you expect to be free?

M_planes = sp.Matrix([
    [1, 1, 1, 1],
    [-1, 2, -3, -1],
])

M_planes.rref()

Run and compare. The rref is [ \begin{bmatrix} 1&0&5/3&1\\ 0&1&-2/3&0 \end{bmatrix}. ] So (z) is free, (x=1-\frac53z), and (y=\frac23z).

Interpretation check. With (z=t), the solution set is [ (x,y,z)=(1,0,0)+t\left(-\frac53,\frac23,1\right). ] This is a line, matching the geometric picture of two nonparallel planes intersecting.

Common mistake. A system with three variables and two independent equations usually has a free variable, not a unique solution.

Part 5. Homogeneous systems and null-space directions

Math reminder. A homogeneous system has the form (H\mathbf{x}=\mathbf{0}). It always has the trivial solution (\mathbf{x}=\mathbf{0}). Nontrivial solutions are null-space directions.

Predict before running. The matrix below has two rows and four columns. Should there be free variables? Should the null space contain nonzero vectors?

H = sp.Matrix([
    [2, 0, 4, 6],
    [1, 2, 0, 7],
])

H.rref(), H.nullspace()

Run and compare. The rref has pivots in the first two columns and free variables in the third and fourth columns. SymPy returns two null-space basis vectors: [ (-2,1,1,0),\qquad (-3,-2,0,1). ]

Interpretation check. Every linear combination of these two vectors is sent to (\mathbf{0}) by (H). These are input directions, not output directions.

Common mistake. The null space lives in the input space. For a (2\times4) matrix, null-space vectors have four entries.

Part 6. Difference matrices forget level

Math reminder. A first-difference matrix records changes between neighboring entries. Adding the same constant to every entry changes the level but not the differences.

Predict before running. Should D @ x and D @ (x + 10*ones) be the same or different? What should D @ ones be?

D = np.array([
    [-1, 1, 0, 0],
    [0, -1, 1, 0],
    [0, 0, -1, 1],
])

x = np.array([2, 5, 9, 10])
ones = np.ones(4)

D @ x, D @ (x + 10*ones), D @ ones

Run and compare. The first two outputs agree. The vector D @ ones is zero.

Interpretation check. The map (D) measures consecutive differences. The constant direction ((1,1,1,1)) is forgotten.

Common mistake. A matrix can preserve changes while forgetting absolute level.

Part 7. Redundant features and same predictions

Math reminder. Dependent columns create nonzero null vectors. If (X\mathbf{z}=\mathbf{0}), then [ X\mathbf{c}=X\mathbf{c}' ] whenever (\mathbf{z}=\mathbf{c}-\mathbf{c}'). So different coefficient vectors can give the same prediction.

Predict before running. The third feature column of X is the sum of the first two feature columns. What vector z should be in the null space?

X = np.array([
    [1, 2, 3],
    [2, 4, 6],
    [0, 1, 1],
    [1, -1, 0],
], dtype=float)

c = np.array([2, 1, 0], dtype=float)
c_prime = np.array([1, 0, 1], dtype=float)
z = c - c_prime

X @ c, X @ c_prime, z, X @ z, np.linalg.matrix_rank(X)

np.allclose(X @ c, X @ c_prime), np.allclose(X @ z, 0)

Run and compare. The products X @ c and X @ c_prime are the same. The vector z is [1, 1, -1], the product X @ z is zero, and the rank is 2.

Interpretation check. The three columns contain only two independent feature directions. The null vector represents a hidden way to change coefficients without changing predictions on the observed data.

Common mistake. Redundant columns do not mean the output is zero. They mean some nonzero coefficient changes are invisible.

Part 8. Affine layer bias and forgotten directions

Math reminder. An affine layer has the form (\mathbf{h}\mapsto W\mathbf{h}+\mathbf{b}). The bias shifts outputs, but forgotten directions are still determined by (W\mathbf{d}=\mathbf{0}).

Predict before running. If W @ d is zero, should adding the same bias vector change whether h and h + d have the same affine-layer output?

W = np.array([
    [1, 1, 0],
    [0, 1, 1],
], dtype=float)

b_bias = np.array([0.5, -0.5])
h = np.array([2, 0, 1], dtype=float)
d = np.array([1, -1, 1], dtype=float)

W @ h + b_bias, W @ (h + d) + b_bias, W @ d

Run and compare. The two affine-layer outputs agree because W @ d is zero.

Interpretation check. The bias shifts both outputs by the same amount. Forgotten directions are determined by (W\mathbf{d}=\mathbf{0}), not by the bias vector.

Common mistake. A bias can move the output location, but it does not make an invisible input direction visible.

Part 9. Rank and nullity audit

Math reminder. Rank counts how many independent directions get through. Nullity counts how many independent input directions are forgotten.

Predict before running. Which matrix below should have full column rank? Which should definitely forget a nonzero input direction?

matrices = {
    "projection P": P,
    "difference D": D,
    "redundant X": X,
    "affine matrix W": W,
    "identity I3": np.eye(3),
    "zero 2x3": np.zeros((2, 3)),
}

for name, M in matrices.items():
    rank = np.linalg.matrix_rank(M)
    nullity = M.shape[1] - rank
    print(name, M.shape, "rank =", rank, "nullity =", nullity)

Run and compare. The identity matrix has full column rank and nullity zero. The projection, difference matrix, redundant matrix, affine-layer matrix, and zero matrix all forget at least one nonzero input direction.

Interpretation check. For an (m\times n) matrix, rank-nullity says [ \operatorname{rank}(A)+\operatorname{nullity}(A)=n. ] Rank is the number of transmitted input directions; nullity is the number of forgotten input directions.

Common mistake. Rank is not just the number of rows or columns. It is the number of independent directions transmitted.

Part 10. Inverses and determinants

Math reminder. For a square matrix, determinant and rank are invertibility checks. A square matrix is invertible exactly when it forgets no nonzero input direction.

Predict before running. Which transformations in the gallery below should be invertible? Which are projections and should have determinant zero?

gallery = {
    "projection onto y-axis": np.array([[0., 0.],
                                        [0., 1.]]),
    "reflection across y-axis": np.array([[-1., 0.],
                                          [0., 1.]]),
    "horizontal shear": np.array([[1., 1.],
                                  [0., 1.]]),
    "rotation": np.array([[0., -1.],
                          [1., 0.]]),
    "projection onto x-axis": np.array([[1., 0.],
                                        [0., 0.]]),
}

for name, M in gallery.items():
    print(name)
    print("det =", np.linalg.det(M))
    print("rank =", np.linalg.matrix_rank(M))
    print()

Run and compare. The reflection, shear, and rotation have nonzero determinant and full rank, so they are invertible. The two projections have determinant zero and rank one, so they are not invertible.

Interpretation check. A projection collapses one direction to zero. Once a direction is forgotten, the input cannot be recovered uniquely.

Common mistake. A determinant close to zero in floating-point arithmetic may require care. For exact symbolic checks, use SymPy.

Solving with an inverse check

Math reminder. The command np.linalg.solve(A, b) solves (A\mathbf{x}=\mathbf{b}) when (A) is square and nonsingular.

Predict before running. Why should the final product A_solve @ x_solution reproduce b?

A_solve = np.array([
    [1., 1.],
    [0., 1.],
])

b = np.array([3., 2.])

x_solution = np.linalg.solve(A_solve, b)
x_solution, A_solve @ x_solution

Run and compare. The computed solution is [1, 2], and multiplying by A_solve returns [3, 2].

Interpretation check. The final product checks the solution by substituting it back into (A\mathbf{x}=\mathbf{b}).

Common mistake. solve is for square nonsingular systems. A singular matrix may have no solution or infinitely many solutions, depending on the target.

Part 11. Review code-reading bank

These are short checks for discussion or self-review. They are meant to be answered without writing long programs. A few items are review/extension prompts from the Unit 2 exercises rather than new core computations.

Review questions

1. Shape check. What is the shape of the projection matrix P in Part 1?
2. Domain and codomain. What input dimension and output dimension does P represent?
3. Forgotten direction. What does P @ z = [0, 0] mean when z is nonzero?
4. Same output. Why do P @ x1 and P @ x2 agree in Part 1?
5. Reachability. For the matrix in Part 2, what condition must (\mathbf{b}=(b_1,b_2,b_3)) satisfy to be reachable?
6. Column space. Geometrically, what is (\operatorname{col}(A)) in Part 2?
7. Inconsistent row. What does a row [0, 0, 1] in an augmented rref mean?
8. Pivot variable. In an rref output, what does a pivot column tell you?
9. Free variable. What does a free variable usually mean for the number of solutions?
10. Plane intersection. Why does the system in Part 4 describe a line?
11. Homogeneous system. Why does (H\mathbf{x}=\mathbf{0}) always have at least one solution?
12. Null-space basis. Are the vectors returned by H.nullspace() input directions or output directions?
13. Nontrivial null vector. If (A\mathbf{z}=\mathbf{0}) with (\mathbf{z}\neq\mathbf{0}), what does this say about uniqueness of inputs?
14. Difference matrix. What does D @ x measure?
15. Level shift. Why does adding 10*ones not change the first differences?
16. Constant direction. What does D @ ones = 0 say?
17. Rank meaning. What does rank count?
18. Nullity meaning. What does nullity count?
19. Rank-nullity. If (A) has four columns and rank three, what is its nullity?
20. Redundant columns. If (\mathbf{c}_3=\mathbf{c}_1+\mathbf{c}_2), give one nonzero null vector.
21. Same predictions. If (B\mathbf{z}=\mathbf{0}) and (B\mathbf{c}=\mathbf{y}), what is (B(\mathbf{c}+5\mathbf{z}))?
22. Parameter nonuniqueness. What does a nonzero null vector mean for a fitted model with coefficient vector (\mathbf{c})?
23. Determinant zero. What does determinant zero say about a square matrix?
24. Nonzero determinant. What does a nonzero determinant say about a square matrix?
25. Projection matrix. Why is a projection not invertible when it sends a nonzero direction to zero?
26. Solve command. What does np.linalg.solve(A, b) compute?
27. Solve caution. Why should np.linalg.solve not be used blindly on a singular matrix?
28. Column-space basis bug (review/extension). What is wrong with using pivot columns of rref(A) as a basis for (\operatorname{col}(A))?
29. Row operation warning. What do row operations preserve?
30. Numerical check. What does np.allclose(A @ z, 0) check?
31. Bias check. Why does the bias (\mathbf{b}) in (W\mathbf{h}+\mathbf{b}) not change the null space of (W)?
32. Audit summary. Name three audit questions one can ask about a linear map (\mathbf{x}\mapsto A\mathbf{x}).

Answer sketches

1. (2,3).
2. Inputs are in (\mathbb R^3); outputs are in (\mathbb R^2).
3. The direction represented by z is forgotten by (P).
4. They differ only in the third coordinate, which (P) forgets.
5. The target must satisfy (b_3=b_1+b_2).
6. It is the plane (b_3=b_1+b_2) in (\mathbb R^3).
7. It represents the impossible equation (0=1), so the system is inconsistent.
8. It identifies a leading variable and an independent condition.
9. If the system is consistent, a free variable gives infinitely many solutions.
10. Two independent plane equations in three variables leave one free parameter.
11. The zero vector is always a solution.
12. Input directions.
13. Inputs are not unique: (\mathbf{x}) and (\mathbf{x}+\mathbf{z}) have the same output whenever (A\mathbf{x}) is defined.
14. Consecutive differences.
15. Differences do not change when the same constant is added to every entry.
16. The constant direction is in the null space of (D).
17. Independent directions transmitted by the matrix.
18. Independent input directions forgotten by the matrix.
19. Nullity is (4-3=1).
20. ((1,1,-1)).
21. (\mathbf{y}).
22. Different coefficient vectors can give the same predictions.
23. The matrix is singular and forgets a nonzero direction.
24. The matrix is invertible and forgets no nonzero direction.
25. A forgotten direction cannot be recovered.
26. The unique solution of (A\mathbf{x}=\mathbf{b}), when (A) is square and nonsingular.
27. A singular system may have no solution or infinitely many solutions.
28. Row operations do not preserve the original columns. Use pivot positions from rref, but take the corresponding columns of the original matrix.
29. Solution sets of linear systems and row space, not the original column vectors.
30. Whether A @ z is approximately zero in floating-point arithmetic.
31. The bias is added after multiplication by (W); invisible directions are determined by (W\mathbf{z}=\mathbf{0}).
32. What outputs are reachable? What directions are forgotten? Can the map be reversed?