Annihilator of $(\mathfrak{g},K)$-modules of $\mathrm{O}(p,q)$ associated to the finite-dimensional representation of $\mathfrak{sl}_{2}$

For a complex simple Lie algebra $\mathfrak{g}$, Joseph constructed the so-called Joseph ideal in [8], which is a completely prime two-sided primitive ideal in the universal enveloping algebra $U(\mathfrak{g})$ of $\mathfrak{g}$ and has the associated variety of minimal dimension. He also proved that it is unique if $\mathfrak{g}$ is not of type $A$ (see also [2]). An irreducible unitary representation of a simple Lie group is called a minimal representation if its annihilator in $U(\mathfrak{g})$ is equal to the Joseph ideal.

The best known example of the minimal representation is the oscillator (or Segal-Shale-Weil) representation of the metaplectic group, the double cover of the symplectic group $\mathrm{Sp}(n,\mathbb{R})$. It was shown in [5] that the canonical quantization of the moment map on real symplectic vector spaces gives rise to the underlying $(\mathfrak{g},K)$-module of the oscillator representation of real reductive Lie groups $G$, where $\mathfrak{g}$ denotes the complexified Lie algebra of $G$ and $K$ a maximal compact subgroup of $G$.

In the case of $G=\mathrm{O}(p,q)$, the indefinite orthogonal group, applying this method to the symplectic vector space $((\mathbb{C}^{p+q})_{\mathbb{R}},\omega)$ with $\omega(z,w)=\operatorname{Im}(z^{*}I_{p,q}w)$, on which $G$ acts via matrix multiplication in a Hamiltonian way with moment map (see below for details), and making use of the fact that $(\mathrm{O}(p,q),\mathrm{SL}_{2}(\mathbb{R}))$ is a dual pair in $\mathrm{Sp}(p+q,\mathbb{R})$, the author constructed $(\mathfrak{g},K)$-modules $\mathit{M}^{+}(m)$ and $\mathit{M}^{-}(m)$ of $\mathrm{O}(p,q)$ associated to the finite-dimensional representation of $\mathfrak{sl}_{2}$ of dimension $m+1$ for any non-negative integer $m$ in [6]. Note that $\mathit{M}^{+}(0)=\mathit{M}^{-}(0)$ by definition and $\mathit{M}^{+}(m)$ is naturally isomorphic to $\mathit{M}^{-}(m)$ in fact. Moreover, under the condition

he obtained the $K$-type formula of $\mathit{M}^{\pm}(m)$, thereby showed that they are irreducible and that $\mathit{M}^{\pm}(0)$ is the underlying $(\mathfrak{g},K)$-module of the minimal representation of $\mathrm{O}(p,q)$. Although it is well known that the Gelfand-Kirillov dimension of the minimal representation of $\mathrm{O}(p,q)$ is equal to $p+q-3$ (see, e.g., [9, 12]), we remark that the $K$-type formula of $\mathit{M}^{\pm}(m)$ mentioned above implies that they all have the Gelfand-Kirillov dimension equal to $p+q-3$, whereas they have the Bernstein degree equal to $4(m+1)\binom{p+q-4}{p-2}$.

The original object of this project is to clarify the relation between our $(\mathfrak{g},K)$-modules $\mathit{M}^{\pm}(m)$ and the minimal representation of $\mathrm{O}(p,q)$ more directly. In fact, we shall see that the annihilator in the universal enveloping algebra $U(\mathfrak{g})$ of $\mathit{M}^{\pm}(m)$, denoted by $\mathscr{I}^{\pm}(m)$ in this paper, is the Joseph ideal if and only if $m=0$, which is the main result of this paper (Theorem 3.6). We use a criterion due to Garfinkle [3] (see also [2]) to determine whether $\mathscr{I}^{\pm}(m)$ is the Joseph ideal or not. We will see that an element of the symmetric square $S^{2}(\mathfrak{g})$, which we denote by $\Xi$ below, plays a critical role in the proof of our main result. We remark that $\Xi$ is given in terms of the Casimir elements of $\mathfrak{g}$ and $\mathfrak{k}$, where $\mathfrak{k}$ is the complexified Lie algebra of the maximal compact subgroup $K$.

The rest of the paper is organized as follows. In Section 2 we briefly review the construction of $(\mathfrak{g},K)$-modules $\mathit{M}^{\pm}(m)$ of $\mathrm{O}(p,q)$ mentioned above as well as their properties needed to prove our main results. In Section 3 we prove our main result using the criterion due to Garfinkle. In order to keep the present paper self-contained, we include some standard facts about irreducible decomposition of $S^{2}({\mathfrak{o}}_{n})$ in the Appendix.

Throughout this paper, we denote by $G$ the indefinite orthogonal group $\mathrm{O}(p,q)$ realized by

where $n=p+q$ and $I_{p,q}=\left[\begin{smallmatrix}1_{p}&\\ &-1_{q}\end{smallmatrix}\right]$. Let $\mathfrak{g}={\mathfrak{o}}(p,q)\otimes\mathbb{C}$, the complexified Lie algebra of $G$. Thus,

Needless to say, our Lie algebra $\mathfrak{g}$ is $\mathbb{C}$-isomorphic to the Lie algebra ${\mathfrak{o}}_{n}$ consisting of all complex $n\times n$ alternating matrices:

Let $K$ be a maximal compact subgroup of $G$ given by

Denote the Lie algebra of $K$ and its complexification by $\mathfrak{k}_{0}$ and $\mathfrak{k}$ respectively. Then $\mathfrak{k}\simeq\mathfrak{o}_{p}\oplus\mathfrak{o}_{q}$.

Let us fix a nondegenerate invariant bilinear form $B$ defined both on $\mathfrak{g}$ and on ${\mathfrak{o}}_{n}$ as follows:

Let $\{X_{i,j}\}_{i,j\in[n]}$ be the generators of $\mathfrak{g}$ given by

for $i,j=1,2,\dots,n$, where $E_{i,j}$ stands for the matrix unit and

Then one sees $X_{j,i}=-X_{i,j}$ and

In addition, if one sets

then $\mathfrak{g}=\mathfrak{k}\oplus\mathfrak{p}$ is the complexified Cartan decomposition. Finally, the dual generators $\{X_{i,j}^{\vee}\}_{i,j}$ to $\{X_{i,j}\}_{i,j}$ with respect to $B$, i.e., those satisfying $B(X_{i,j},X_{k,l}^{\vee})=\delta_{i,k}\delta_{j,l}$, are given by

Let $V$ denote the vector space $\mathbb{R}^{p+q}$ equipped with a quadratic form $\eta$ defined by

Then $G$ acts on $V$ by matrix multiplication, leaving $\eta$ invariant. It is well known that the space $\mathbb{C}[V]$ of polynomial functions on $V$ decomposes as follows (see e.g. [4]): for $v=\hskip 1.8pt\vphantom{1}^{t}(x_{1},\dots,x_{p},y_{1},\dots,y_{q})\in V$, one sees that

where $r_{x}^{2}=x_{1}^{2}+\cdots+x_{p}^{2},\,r_{y}^{2}=y_{1}^{2}+\cdots+y_{q}^{2}$ and $\mathscr{H}^{k}(\mathbb{R}^{p})$ denotes the space of homogeneous harmonic polynomials on $\mathbb{R}^{p}$ of degree $k$ and so on.

Taking (2.5) into account, let us introduce our space $\mathscr{E}$ of functions on $V$ by

where $\mathbb{C}\llbracket r_{x}^{2},r_{y}^{2}\rrbracket$ denotes the ring of formal power series in $r_{x}^{2}$ and $r_{y}^{2}$, and the direct sum is algebraic; we assume that the radial part should be a formal power series rather than a polynomial so that we can find a highest weight vector or a lowest weight vector with respect to the $\mathfrak{sl}_{2}$-action (2.10) given below.

Denoting the ring of polynomial coefficient differential operators on $V$ by $\mathscr{P}\!\mathscr{D}(V)$, let us define a representation $\pi:U(\mathfrak{g})\to\mathscr{P}\!\mathscr{D}(V)$ by

where we set $i^{\prime}:=i-p,\;i\in p+[q]$, for short. It is easy to verify that $\pi$ indeed defines a representation of $U(\mathfrak{g})$. It is also clear that the action of $\mathfrak{k}_{0}$ lifts to the one of $K$, i.e., if $X\in\mathfrak{k}_{0}$ then

Define elements $H,X^{+},X^{-}$ of $\mathscr{P}\!\mathscr{D}(V)$ by

where

Then they satisfy that

Namely, $\mathfrak{g}^{\prime}:=\mbox{$\mathbb{C}$-}\!\operatorname{span}\left\{{H,X^{+},X^{-}}\right\}$ forms a Lie subalgebra of $\mathscr{P}\!\mathscr{D}(V)$ isomorphic to $\mathfrak{sl}_{2}$. Moreover, $\mathfrak{g}^{\prime}$ forms the commutant of $\mathfrak{g}$ in $\mathscr{P}\!\mathscr{D}(V)$:

In view of the fact that $(\mathrm{O}(p,q),\mathrm{SL}(2,\mathbb{R}))$ is a dual pair in $\mathrm{Sp}(n,\mathbb{R})$, we introduce $(\mathfrak{g},K)$-modules of $\mathrm{O}(p,q)$ associated to the finite-dimensional representation of $\mathfrak{sl}_{2}$ as follows.

In order to describe elements of $\mathit{M}^{\pm}(m)$ explicitly, let us introduce a convergent power series $\Psi_{\alpha}$ on $\mathbb{C}$ for $\alpha\in\mathbb{C}\smallsetminus(-\mathbb{N})$ by

with $(\alpha)_{j}=\Gamma(\alpha+j)/\Gamma(\alpha)=\alpha(\alpha+1)\cdots(\alpha+j-1)$. Note that it is related to the Bessel function $J_{\nu}$ of the first kind. In fact, one has

Now we set

for brevity. We will find it convenient to write $\rho_{x}:=r_{x}^{2}/2,\,\rho_{y}:=r_{y}^{2}/2$. Then a typical element $f\in\mathit{M}^{+}(m)$ is given by

with $h_{1}\in\mathscr{H}^{k}(\mathbb{R}^{p}),\;h_{2}\in\mathscr{H}^{l}(\mathbb{R}^{q})$ and $\mu_{-}=\frac{1}{2}(m+\kappa_{+}-\kappa_{-})\in\mathbb{N}$, while a typical one $f\in\mathit{M}^{-}(m)$ is given by

with $h_{1}\in\mathscr{H}^{k}(\mathbb{R}^{p})\;h_{2}\in\mathscr{H}^{l}(\mathbb{R}^{q})$ and $\mu_{+}=\frac{1}{2}(m-\kappa_{+}+\kappa_{-})\in\mathbb{N}$ (see [6, Proposition 3.2]). A general element of $\mathit{M}^{+}(m)$ (resp. $\mathit{M}^{-}(m)$) is a linear combination of these elements given in (2.14) (resp. (2.15)).

Now we collect a few facts from [6] that we need in order to prove our main result.

First, we recall the action of $\mathfrak{p}$ on $\mathit{M}^{\pm}(m)$ ([6, Lemma 4.1]). Let $X_{i,p+j}$, $i\in[p],\,j\in[q]$, be a basis of $\mathfrak{p}$ given in (2.2). Then, for $f=h_{1}h_{2}\rho_{y}^{\mu_{-}}\psi_{\kappa_{+}}\in\mathit{M}^{+}(m)$ with $\kappa(f)=(\kappa_{+},\kappa_{-})$ and $\mu_{-}=\frac{1}{2}(m+\kappa_{+}-\kappa_{-})$, one sees that

Here, in general, for a homogeneous polynomial $P=P(x_{1},\dots,x_{\mathsf{n}})$ on $\mathbb{R}^{\mathsf{n}}$ of degree $d$, we set

with $\Delta=\sum_{i=1}^{\mathsf{n}}\partial_{x_{i}}^{2}$ and $\rho=(1/2)\sum_{i=1}^{\mathsf{n}}x_{i}^{2}$. Note that if $\Delta^{2}P=0$ then $P^{\dagger}$ is harmonic and that if $h=h(x_{1},\dots,x_{\mathsf{n}})$ is harmonic then $\Delta(x_{i}h)=2\partial_{x_{i}}h$ and $\Delta^{2}(x_{i}h)=0$.

Next, the following facts describe the $K$-types of $\mathit{M}^{\pm}(m)$ and when they are irreducibile ([6, Theorem 4.6]).

Note in particular that if the $K$-type of $f\in\mathit{M}^{\pm}(m)$ is $\kappa(f)=(\kappa_{+},\kappa_{-})$, then it follows from (2.17) that

Henceforth in the rest of the paper, we assume that

so that $\mathit{M}^{\pm}(m)$ is irreducible.

In order to translate objects in terms of ${\mathfrak{o}}_{n}$ into ones in terms of $\mathfrak{g}$, and vice versa, let us fix an isomorphism $\Phi$ between $\mathfrak{g}$ and ${\mathfrak{o}}_{n}$ given by

where $I_{p,q}^{1/2}=\operatorname{diag}(1,\dots,1,\sqrt{-1}\,,\dots,\sqrt{-1}\,)$ and $I_{p,q}^{-1/2}$ is its inverse.

Let $\{M_{i,j}\}$ be generators of ${\mathfrak{o}}_{n}$ given by

for $i,j=1,2,\dots,n$. Then, one has

We extend $\Phi$ to the isomorphism from $\bigoplus_{j=0}^{\infty}T^{j}(\mathfrak{g})$ onto $\bigoplus_{j=0}^{\infty}T^{j}({\mathfrak{o}}_{n})$, where $T^{j}(\mathfrak{g}):=\mathfrak{g}\otimes\cdots\otimes\mathfrak{g}$ ($j$ factors) etc..

Let $U(\mathfrak{g})$ denote the universal enveloping algebra of $\mathfrak{g}$, and $U_{j}(\mathfrak{g})$ its subspace spanned by products of at most $j$ elements of $\mathfrak{g}$, with $U_{-1}(\mathfrak{g})=0$. It is well known that the associated graded algebra $\operatorname{gr}U(\mathfrak{g})=\bigoplus_{j=0}^{\infty}U_{j}(\mathfrak{g})/U_{j-1}(\mathfrak{g})$ is isomorphic to the symmetric algebra $S(\mathfrak{g})=\bigoplus_{j=0}^{\infty}S^{j}(\mathfrak{g})$. Let $\sigma:\operatorname{gr}U(\mathfrak{g})\to S(\mathfrak{g})$ be the algebra isomorphism, and $\sigma_{j}$ its $j$-th piece:

Let $\gamma:S(\mathfrak{g})\to U(\mathfrak{g})$ be the linear isomorphism called symmetrization, and $\gamma_{j}$ its $j$-th piece

which is an injective linear map onto a vector-space complement to $U_{j-1}(\mathfrak{g})$ in $U_{j}(\mathfrak{g})$. Then the following diagram is commutative:

where $p_{j}$ is the canonical projection. In particular, $(\sigma_{j}\circ p_{j}\circ\gamma_{j})(v)=v$ for all $v\in S^{j}(\mathfrak{g})$.

In what follows, given a finite-dimensinal vector space $U$ over $\mathbb{C}$, we will identify $S^{j}(U)$ (resp. $\mbox{$\bigwedge^{\!j}$}U$) with the subspace of symmetric (resp. alternating) tensors in $U^{\otimes j}$.

For $u\in U(\mathfrak{g})$, we sometimes denote $\pi(u)$ by $\hat{u}$ for brevity. Let $\Omega_{\mathfrak{g}},\,\Omega_{\mathfrak{o}_{p}}$ and $\Omega_{\mathfrak{o}_{q}}$ be the Casimir elements of $\mathfrak{g},\,\mathfrak{o}_{p}$ and $\mathfrak{o}_{q}$ respectively¹¹1We regard $\mathfrak{o}_{p}$ and $\mathfrak{o}_{q}$ as subalgebras of $\mathfrak{k}\simeq\mathfrak{o}_{p}\oplus\mathfrak{o}_{q}$ canonically., which are given by

Then the corresponding operators represented in $\mathscr{P}\!\mathscr{D}(V)$ are given by

It is well known that the Casimir operator $\hat{\Omega}_{\mathfrak{g}^{\prime}}$ of $\mathfrak{g}^{\prime}=\mathfrak{sl}_{2}$ given by

satisfies the relation

(see [7, 11]), which implies that