Baklouti, A.; Ludwig, J. Invariant differential operators on certain nilpotent homogeneous spaces. (English) Zbl 0997.22007 Monatsh. Math. 134, No. 1, 19-37 (2001). Let \(G=\exp {\mathfrak g}\) be a connected, simply connected real nilpotent Lie group with Lie algebra \({\mathfrak g}\), \(H=\exp {\mathfrak h}\) be an analytic subgroup of \(G\) with Lie algebra \({\mathfrak h}\). Given a unitary character \(\chi= \chi_f\), \(\chi(\exp X)=e^{-\sqrt {-1}f(X)}\) \((\forall X\in {\mathfrak h})\) with \(f\in {\mathfrak g}^*\) satisfying \(f([{\mathfrak h}, {\mathfrak h}])=\{0\}\), we construct the induced representation \(\tau=\text{Ind}^G_H\chi_f\) of \(G\). In this paper, the authors study the algebra \(D_\tau (G/H)\) of \(G\)-invariant differential operators on the line bundle with basis \(G/H\) associated to these data.Let \(\widehat G\) be the unitary dual of \(G\), then we have the canonical central decomposition of \(\tau:\tau\cong\int^\oplus_{\widehat G}m (\pi)\pi d\mu(\pi)\) with multiplicity function \(m(\pi)\) and a certain positive Borel measure \(\mu\) on \(\widehat G\). We have the following alternative: either \(m(\pi)= \infty\) \(\mu\)-a.e. or \(m(\pi) <c\) \(\mu\)-a.e. for some constant \(c\) [cf. L. Corwin, F. P. Greenleaf and G. Grélaud: Trans. Am. Math. Soc. 304, 549-583 (1987; Zbl 0629.22005)]. We distinguish these two situations by saying that \(\tau\) is of infinite (resp. finite) multiplicities. There are two well-known conjectures from L. Corwin and F. P. Greenleaf [Comm. Pure Appl. Math. 45, 681-748 (1992; Zbl 0812.43004)]. Polynomial conjecture: when \(\tau\) is of finite multiplicities, the algebra \(D_\tau (G/H)\) is isomorphic to the algebra \(\mathbb{C} [\Gamma_\tau]^H\) of \(H\)-invariant polynomial functions on \(\Gamma_\tau= f+{\mathfrak h}^\perp\subset {\mathfrak g}^*\). Commutativity conjecture: \(\tau\) is of finite multiplicities if and only if \(D_\tau (G/H)\) is commutative. The authors prove here the polynomial conjecture under the condition that there exists a subalgebra which polarizes all generic elements in \(\Gamma_\tau\), and also the commutativity conjecture when \({\mathfrak h}\) is an ideal of \({\mathfrak g}\). Reviewer: Hidenori Fujiwara (Iizuka) Cited in 1 ReviewCited in 6 Documents MSC: 22E27 Representations of nilpotent and solvable Lie groups (special orbital integrals, non-type I representations, etc.) 22E30 Analysis on real and complex Lie groups Keywords:invariant differential operators; nilpotent Lie group Citations:Zbl 0629.22005; Zbl 0812.43004 PDFBibTeX XMLCite \textit{A. Baklouti} and \textit{J. Ludwig}, Monatsh. Math. 134, No. 1, 19--37 (2001; Zbl 0997.22007) Full Text: DOI