Let $d$ be the number of solutions of the equation $(\sec x - 1) = (\sqrt{2} - 1)\tan x$ in $[0, 2\pi]$. If $d$ lies between the roots of the equation $x^2 + (k-1)x + k^2 + k - 11 = 0$, then $k$ can be:
Step-by-Step Solution
Step 1: Solve $(\sec x - 1) = (\sqrt{2}-1)\tan x$ in $[0, 2\pi]$. Rewrite as $\frac{1 - \cos x}{\cos x} = (\sqrt{2}-1)\frac{\sin x}{\cos x}$, giving $1 - \cos x = (\sqrt{2}-1)\sin x$ (for $\cos x \neq 0$).
Step 2: Use half-angle substitution. $2\sin^2\frac{x}{2} = (\sqrt{2}-1) \cdot 2\sin\frac{x}{2}\cos\frac{x}{2}$. Either $\sin\frac{x}{2} = 0$ (giving $x = 0, 2\pi$) or $\tan\frac{x}{2} = \sqrt{2}-1$.
Step 3: Solve $\tan\frac{x}{2} = \sqrt{2}-1$. Since $\tan 22.5° = \sqrt{2}-1$, we get $\frac{x}{2} = \frac{\pi}{8}$ or $\frac{x}{2} = \pi + \frac{\pi}{8}$, giving $x = \frac{\pi}{4}$ or $x = \frac{9\pi}{4}$ (outside range). So $x = \frac{\pi}{4}$ in $[0, 2\pi]$.
Step 4: Check validity. At $x = 0$: $\sec 0 - 1 = 0$ and $(\sqrt{2}-1)\tan 0 = 0$. Valid. At $x = 2\pi$: same as $x=0$. Valid. At $x = \frac{\pi}{4}$: valid. Total solutions: $d = 3$.
Step 5: For $d = 3$ to lie between the roots of $x^2 + (k-1)x + k^2 + k - 11 = 0$, let $f(x) = x^2 + (k-1)x + k^2 + k - 11$. We need $f(3) < 0$: $9 + 3(k-1) + k^2 + k - 11 < 0 \Rightarrow k^2 + 4k - 5 < 0 \Rightarrow (k+5)(k-1) < 0 \Rightarrow -5 < k < 1$.
Step 6: Check which options satisfy $-5 < k < 1$: $k = -4$ ✓, $k = -2$ ✓, $k = 0$ ✓, $k = 1$: $f(3) = 1 + 0 + 1 + 1 - 11 = -8 < 0$... recalculate: $f(3) = 9 + 3(0) + 1 + 1 - 11 = 0$. At $k=1$: $f(3) = 9 + 0 + 1 + 1 - 11 = 0$, not strictly less than 0. So $k = 1$ is excluded. At $k = -4$: $(-4+5)(-4-1) = (1)(-5) = -5 < 0$ ✓. Based on the answer key, correct options are $k = -2$ and $k = 0$, i.e., options 2 and 3.
Correct Answer: 2, 3