Representations of the Lorentz group are not unitary and the normal ordered angular momentum

I've mentioned earlier (here, in Spanish anyway) that I'm trying to introduce myself to QFT following this set of lecture notes by David Tong. I recently deleted my Physics SE account, but I happened to note that the mentioned set of lecture notes have become somewhat popular (in a good way), so hopefully someone else than me will find this useful.

First (I'll actually go backwards), in Section 4.1.1 it's stated that, in general, the representation of the Lorentz group $S[\Lambda]$ is not unitary, being that by the definition
$$\begin{equation}S[\Lambda]=\exp\left(\frac{1}{2}\Omega_{\rho\sigma}S^{\rho\sigma}\right)\label{qftlor1}\end{equation}$$ this representation would be unitary if $(S^{\mu\nu})^\dagger=-S^{\mu\nu}$, which is then shown that doesn't hold. Here, basically, $S^{\mu\nu}$ are the basis generators of Lorentz transformations in spinor representation, $\Omega_{\rho\sigma}$ are the elements of an antisymmetric $4\times4$ matrix that specify the Lorentz transformation (a boost by how much speed, a rotation by what angle, etc) and thus $S[\Lambda]$ is the full Lorentz transformation in spinor rep.

I actually ran into trouble trying to prove that $S[\Lambda]$ is unitary if $S^{\mu\nu}$ are anti-hermitian. The thing was that the notation turned really confusing for me. However, it's stressed out that in ($\ref{qftlor1}$), the thing making $S[\Lambda]$ a matrix is actually $S^{\mu\nu}$, because these are actually $4\times4$ matrices. For it to be clear, if we were to write the components of $S[\Lambda]$, we'd write something as
$$\begin{equation}S[\Lambda]^{\alpha\beta}=\exp\left[\frac{1}{2}\Omega_{\rho\sigma}S^{\rho\sigma\alpha\beta}\right]\end{equation}$$ This being said, it's actually pretty easy to make the calculation. By definition, $S[\Lambda]$ is unitary if
$$\begin{equation}S^\dagger[\Lambda]S[\Lambda]=1\label{qftlor2}\end{equation}$$ To compute $S^\dagger[\Lambda]$ we need to simplify something of the form $\left(e^M\right)^\dagger$ with $M$ a $4\times4$ matrix. Of course
$$\begin{align}\exp(M)^\dagger&=\left(\sum_{k=0}^\infty\frac{M^k}{k!}\right)^\dagger\nonumber\\ &=\sum_{k=0}^\infty\frac{1}{k!}\left(M^k\right)^\dagger\nonumber\\ &=\sum_{k=0}^\infty\frac{1}{k!}\left(M^\dagger\right)^k=\exp\left(M^\dagger\right)\end{align}$$ which seems pretty obvious, but when you don't remember these things by heart (and you shouldn't), it's far better to be sure. This way, the requirement ($\ref{qftlor2}$) translates into
$$\begin{equation}\exp\left[\frac{1}{2}\left(\Omega_{\rho\sigma}S^{\rho\sigma}\right)^\dagger+\frac{1}{2}\Omega_{\rho\sigma}S^{\rho\sigma}\right]=1\end{equation}$$ Now, as said before, the matrix is $S^{\mu\nu}$ while $\Omega_{\mu\nu}$ are real numbers, so the expected result follows
$$\begin{equation}\exp\left[\frac{1}{2}\Omega_{\rho\sigma}\left(\left(S^{\rho\sigma}\right)^\dagger+S^{\rho\sigma}\right)\right]=1\,\Longrightarrow\,\left(S^{\rho\sigma}\right)^\dagger=-S^{\rho\sigma}\end{equation}$$ So, at the end, the calculation was indeed kind of naive. All this beginning of section 4 is probably the part of the lecture notes that has gotten the most of me, mainly because I've got no formal knowledge of Representation Theory nor Lie groups or anything like that, so despite that I find myself flashy with all the new lexicon, I admit I have a long road ahead. A related discussion about non-unitary representations can be found here.

Abstruse Goose - Moment of Clarity 2

The other thing I got stuck with while ago was to compute the angular momentum (this is actually twice angular momentum)
$$\begin{equation}Q_i\equiv-2\epsilon_{ijk}\int{d^3x}\,x^kT^{0j}\label{qftlorFIN}\end{equation}$$ where ${T^\mu}_\nu=\frac{\partial\mathcal{L}}{\partial(\partial_\mu\phi)}\partial_\nu\phi-{\delta^\mu}_\nu\mathcal{L}$ with the Lagrangian $\mathcal{L}=\frac{1}{2}\partial_\mu\phi\partial^\mu\phi-\frac{1}{2}m^2\phi^2$ on the real scalar field $\phi=\phi(x)$. Basically a Klein-Gordon real scalar. This constitutes problem 7 on Example sheet 2 of the lecture notes by David Tong. I originally posted the question on Physics SE and got no answer before I deleted my profile.

The ultimate issue wasn't really about a mistake, nor about normal ordering. The final expression of $Q_i$ in terms of ladder operators should be
$$\begin{equation}Q_i=-i\epsilon_{ijk}\int\frac{d^3p}{(2\pi)^3} a^\dagger_\vec{p}\left(p_j\frac{\partial}{\partial{p}^k}-p_k\frac{\partial}{\partial{p}^j}\right) a_\vec{p}\label{qftlorfin}\end{equation}$$ So next I write the whole nasty calculation; if you don't feel like following it all, just skip to eq. ($\ref{qftlor5}$).

As $T^{0j}=\dot\phi\partial^j\phi$ and
$$\begin{align}\phi=\int\frac{d^3p}{(2\pi)^3}\frac{1}{\sqrt{2E_\vec{p}}}\left(a_\vec{p} e^{i\vec{p}\cdot\vec{x}}+a^\dagger_\vec{p} e^{-i\vec{p}\cdot\vec{x}}\right)\label{qftlor3}\\\dot\phi=-i\int\frac{d^3p}{(2\pi)^3}\sqrt{\frac{E_\vec{p}}{2}}\left(a_\vec{p} e^{i\vec{p}\cdot\vec{x}}-a^\dagger_\vec{p} e^{-i\vec{p}\cdot\vec{x}}\right)\label{qftlor4}\\\partial^j\phi=-i\int\frac{d^3p}{(2\pi)^3}\frac{p^j}{\sqrt{2E_\vec{p}}}\left(a_\vec{p} e^{i\vec{p}\cdot\vec{x}}-a^\dagger_\vec{p} e^{-i\vec{p}\cdot\vec{x}}\right)\end{align}$$ then
$$\begin{align}&Q_i=-2\epsilon_{ijk}\int{d^3x}\,x^k\dot\phi_p(x)\partial^j\phi_q(x)\nonumber\\ &=\epsilon_{ijk}\sqrt{\frac{E_\vec{p}}{E_\vec{q}}}\int\frac{d^3pd^3qd^3x}{(2\pi)^6}x^kq^j\left(a_\vec{p}e^{i\vec{p}\cdot\vec{x}}-a^\dagger_\vec{p}e^{-i\vec{p}\cdot\vec{x}}\right)\left(a_\vec{q}e^{i\vec{q}\cdot\vec{x}}-a^\dagger_\vec{q}e^{-i\vec{q}\cdot\vec{x}}\right)\nonumber\\ &=\cdots\left(a_\vec{p}a_\vec{q}e^{i(\vec{p}+\vec{q})\cdot\vec{x}}+a_\vec{p}^{\dagger}a_\vec{q}^{\dagger}e^{-i(\vec{p}+\vec{q})\cdot\vec{x}}-a_\vec{p}a^\dagger_\vec{q}e^{i(\vec{p}-\vec{q})\cdot\vec{x}}-a^\dagger_\vec{p}a_\vec{q}e^{-i(\vec{p}-\vec{q})\cdot\vec{x}}\right)\end{align}$$ Now,
$$\begin{align}\int{d^3x}\,x^ke^{i(\vec{p}\pm\vec{q})\cdot\vec{x}}=\mp{i}\frac{\partial}{\partial{p}^k}\int{d^3x}\,e^{i(\vec{p}\pm\vec{q})\cdot\vec{x}}=\mp{i}\,(2\pi)^3\frac{\partial}{\partial{p}^k}\delta(\vec{p}\pm\vec{q})\end{align}$$ and so
$$\begin{align}Q_i=-i\epsilon_{ijk}&\sqrt{\frac{E_\vec{p}}{E_\vec{q}}}\int\frac{d^3pd^3q}{(2\pi)^3}q^j\left[(a_\vec{p}a_\vec{q}-a^\dagger_\vec{p}a^\dagger_\vec{q})\frac{\partial}{\partial{q}^k}\delta(\vec{p}+\vec{q})+(a_\vec{p}a^\dagger_\vec{q}-a^\dagger_\vec{p}a_\vec{q})\frac{\partial}{\partial{q}^k}\delta(\vec{p}-\vec{q})\right]\end{align}$$ Now, integrating by parts on $q$, for example, for the first term
$$\begin{align}\epsilon_{ijk}a_\vec{p}\int{d^3q}\,q^ja_\vec{q}\frac{\partial}{\partial{q}^k}\delta(\vec{p}+\vec{q})&=-\epsilon_{ijk}a_\vec{p}\int{d^3q}\,q^j\left[\frac{\partial}{\partial{q}^k}a_\vec{q}\right]\delta(\vec{p}+\vec{q})\nonumber\\ &=\epsilon_{ijk}a_\vec{p}p^j\frac{\partial}{\partial(-p)^k}a_{-\vec{p}}\end{align}$$ Then, integrating each term in $q$,
$$\begin{align}Q_i=-i\epsilon_{ijk}&\int\frac{d^3p}{(2\pi)^3}p^j\left[a_\vec{p}\frac{\partial}{\partial(-p)^k}a_{-\vec{p}}-a^\dagger_\vec{p}\frac{\partial}{\partial(-p)^k}a^\dagger_{-\vec{p}}-a_\vec{p}\frac{\partial}{\partial{p}^k}a^\dagger_\vec{p}+a^\dagger_\vec{p}\frac{\partial}{\partial{p}^k}a_\vec{p}\right]\end{align}$$ The first two terms vanish upon integration because they are odd in $p$, leaving
$$\begin{equation}Q_i=-i\epsilon_{ijk}\int\frac{d^3p}{(2\pi)^3}p^j\left[a^\dagger_\vec{p}\frac{\partial}{\partial{p}^k}a_\vec{p}-a_\vec{p}\frac{\partial}{\partial{p}^k}a^\dagger_\vec{p}\right]\label{qftlor5}\end{equation}$$ So, finally comes normal ordering.

I was skeptical at first but the calculation goes well step-by-step and the pieces seem to fit together well. The normal ordering is
$$\begin{align}:a^\dagger_\vec{p}(\partial_ka_\vec{p})-a_\vec{p}(\partial_ka^\dagger_\vec{p}):&=a^\dagger_\vec{p}(\partial_ka_\vec{p})-:a_\vec{p}(\partial_ka^\dagger_\vec{p}):\nonumber\\ &=a^\dagger_\vec{p}(\partial_ka_\vec{p})-(\partial_ka^\dagger_\vec{p})a_\vec{p}\nonumber\\ &=2a^\dagger_\vec{p}(\partial_ka_\vec{p})-\partial_k(a^\dagger_\vec{p}a_\vec{p})\label{qftlorf2}\end{align}$$ (where of course I'm using $\partial_k=\frac{\partial}{\partial{p}^k}$) which yields the correct answer if $\partial_k(a^\dagger_\vec{p}a_\vec{p})=0$. I also noticed that $\partial_k(a^\dagger_\vec{p}a_\vec{p})=\partial_k(a_\vec{p}a^\dagger_\vec{p})$ since
$$\begin{equation}[a_\vec{p},a^\dagger_\vec{p}]=(2\pi)^3\delta(0)\label{qftL1}\end{equation}$$
After that long, burdensome and nasty calculation, I was unable to show that indeed $\partial_k(a^\dagger_\vec{p}a_\vec{p})=0$. I recently managed to do so and it goes as follows.

Knowing that $\partial_k(a^\dagger_\vec{p}a_\vec{p})=a_\vec{p}^\dagger(\partial_ka_\vec{p})+(\partial_ka^\dagger_\vec{p})a_\vec{p}$, one can proceed by inverting the Fourier transforms ($\ref{qftlor3}$) and ($\ref{qftlor4}$), getting a sum and a substraction of $a_\vec{p}$ and $a_\vec{p}^\dagger$, which can be combined to yield
$$\begin{align}a_\vec{p}&=\int{d^3x}\,f(x)\,e^{-i\vec{p}\cdot\vec{x}}\label{qftlorf8}\\ a_\vec{p}^\dagger&=\int{d^3y}\,g(y)\,e^{i\vec{p}\cdot\vec{y}}\label{qftlorf9}\end{align}$$ with $f=\alpha\phi+\beta\dot\phi$ and $g=\alpha\phi-\beta\dot\phi$ (keep in mind that these are the operators), and thus,
$$\begin{align}a_\vec{p}^\dagger(\p_ka_\vec{p})&=-i\int{d^3x\,d^3y}\,x^kg(y)f(x)e^{-i\vec{p}\cdot(\vec{x}-\vec{y})}\\ (\p_ka_\vec{p}^\dagger)a_\vec{p}&=i\int{d^3x\,d^3y}\,y^kg(y)f(x)e^{-i\vec{p}\cdot(\vec{x}-\vec{y})}\end{align}$$ so that, being $x$, $y$ integration variables, indeed,
$$\begin{equation}\partial_k(a^\dagger_\vec{p}a_\vec{p})=a_\vec{p}^\dagger(\partial_ka_\vec{p})+(\partial_ka^\dagger_\vec{p})a_\vec{p}=0\end{equation}$$ Also, however, knowing the final result from the start, one could have expect from ($\ref{qftlor5}$), and then show in a similar fashion, that $a_\vec{p}^\dagger(\partial_ka_\vec{p})=-a_\vec{p}(\partial_ka^\dagger_\vec{p})+\delta(0)$, just to finally do the normal ordering and remove the $\delta(0)$ infinity, which arises from $[f(x),g(y)]\propto\delta(x-y)$. It all just falls into place, so normal ordering does not seem to have been the problem either: I guess I was just being plain lazy because I didn't expected to need ($\ref{qftlorf8}$) and ($\ref{qftlorf9}$).

So finally, ($\ref{qftlor5}$) turns into
$$\begin{equation}Q_i=-2i\epsilon_{ijk}\int\frac{d^3p}{(2\pi)^3}\,a^\dagger_\vec{p}\left(p^j\frac{\partial}{\partial{p}^k}\right)a_\vec{p}\end{equation}$$ which is equivalent to ($\ref{qftlorFIN}$); however, this is the antisymmetric $\epsilon_{ijk}$ times something symmetric, so the whole thing is antisymmetric and can be written as $-\epsilon_{ijk}\int{d^3x}\left(x^kT^{0j}-x^jT^{0k}\right)$. This way the result written as ($\ref{qftlorfin}$) follows. As said before, this is actually twice the actual angular momentum operator, so I'll just drop the $2$ factor.

So let's try the new toy: being $|\vec{p}\rangle=a_\vec{p}^\dagger|0\rangle$ the one-particle state with momentum $\vec{p}$, using the relation ($\ref{qftL1}$) and integrating by parts, we find
$$\begin{align}Q_i|\vec{p}\rangle&=-i\epsilon_{ijk}\int\frac{d^3q}{(2\pi)^3}\,a^\dagger_\vec{q}\left(q^j\frac{\partial}{\partial{q}^k}\right)a_\vec{q}a_\vec{p}^\dagger|0\rangle\nonumber\\ &=-i\epsilon_{ijk}\int\frac{d^3q}{(2\pi)^3}\,a^\dagger_\vec{q}\left(q^j\frac{\partial}{\partial{q}^k}\right)\delta(\vec{p}-\vec{q})|0\rangle\nonumber\\ &=i\epsilon_{ijk}\int{d^3q}\,q^j\delta(\vec{p}-\vec{q})\frac{\partial}{\partial{q}^k}a_\vec{q}^\dagger|0\rangle\nonumber\\ &=i\epsilon_{ijk}p^j\frac{\partial}{\partial{p}^k}|\vec{p}\rangle\end{align}$$ And one can readily see with ($\ref{qftlorf9}$) that a one particle state with no momentum, $|\vec{p}=\vec{0}\rangle=a_\vec{0}^\dagger|0\rangle$ carries no internal angular momentum or spin, $Q_i|0\rangle=0$, as expected.

Finally, the previous Abstruse Goose cartoon, and the next one, reminded me of this old slogan that a lot of people I've come across preaches:

Sure, I know you can do the math, but do you really understand it?

In physics there's always this thing about understanding results physically, and that's fine, but a lot of people seem to read this as ultimately reduce all mathematical results to apples and oranges. A discussion about the first cartoon can be found here, and as can be seen, there's no problem whatsoever for people to understand things physically without reducing them to potatoes (which, when done, happens at a price, as science communicators know). To develop an intuition about physical phenomena is, however, a different matter, but then again, one that's mainly driven mathematically.

Our experience up to date justifies us in feeling sure that in Nature is actualized the ideal of mathematical simplicity. It is my conviction that pure mathematical construction enables us to discover the concepts and the laws connecting them which give us the key to the understanding of the phenomena of Nature. Experience can of course guide us in our choice of serviceable mathematical concepts; it cannot possibly be the source from which they are derived; experience of course remains the sole criterion of the serviceability of a mathematical construction for physics, but the truly creative principle resides in mathematics. In a certain sense, therefore, I hold it to be true that pure thought is competent to comprehend the real, as the ancients dreamed.

—A. Einstein, “On the Method of Theoretical Physics”

Of course there is still a great deal to learn for us all, and wandering moments remain essential to accomplish that ;-)

Abstruse Goose - Moment of Clarity 1