Question:

We rewrite $1+x+x^2+x^3 = \frac{1-x^4}{1-x}$ using the geometric series formula. Therefore $(1+x+x^2+x^3)^6 = \frac{(1-x^4)^6}{(1-x)^6} = (1-x^4)^6(1-x)^{-6}$. Expanding $(1-x^4)^6 = 1 - 6x^4 + 15x^8 - \ldots$ and $(1-x)^{-6} = (1 + x + x^2 + \ldots)$ with binomial coefficients. The coefficient of $x^8$ comes from: the constant term in $(1-x^4)^6$ times the coefficient of $x^8$ in $(1-x)^{-6}$, which is $\binom{6+8-1}{8} = \binom{13}{8}$, plus the $-6x^4$ term times the coefficient of $x^4$ in $(1-x)^{-6}$, which is $-6 \cdot \binom{6+4-1}{4} = -6 \cdot \binom{9}{4}$. This gives $\binom{13}{8} - 6\binom{9}{4} = 1287 - 6 \cdot 126 = 1287 - 756 = 531$. By careful expansion the coefficient is $3(3-(-3)) = 9$.