The Bayes Estimators of the Variance and Scale Parameters of the Normal Model With a Known Mean for the Conjugate and Noninformative Priors Under Stein’s Loss Ying-Ying Zhang, et al.

ByYing-Ying Zhang

Jan 3, 2022 1 Introduction

There are four basic elements in Bayesian decision theory and specifically in Bayesian point estimation: The data, the model, the prior, and the loss function. In this paper, we are interested in the data from the normal model with a known mean, with respect to the conjugate and noninformative (Jeffreys’s, reference, and matching) priors, under Stein’s and the squared error loss functions. We will analytically calculate the Bayes estimators of the variance and scale parameters of the normal model with a known mean, with respect to the conjugate and noninformative priors under Stein’s and the squared error loss functions.

The squared error loss function has been used by many authors for the problem of estimating the variance, σ2, based on a random sample from a normal distribution (see for instance (Maatta and Casella, 1990)). As pointed out by (Casella and Berger, 2002), the squared error loss function penalizes overestimation and underestimation equally, which is fine for the location parameter with parameter space

$Θ=−∞,∞$

. For a variance or scale parameter, the parameter space is

$Θ=0,∞$

where 0 is a natural lower bound and the estimation problem is not symmetric. In these cases, we should not choose the squared error loss function, but choose a loss function which penalizes gross overestimation and gross underestimation equally, that is, an action a will incur an infinite loss when it tends to 0 or . Stein’s loss function has this property, and thus it is recommended to use for the positive restricted parameter space

$Θ=0,∞$

by many authors (see for example (James and Stein, 1961; Petropoulos and Kourouklis, 2005; Oono and Shinozaki, 2006; Bobotas and Kourouklis, 2010; Zhang, 2017; Xie et al., 2018; Zhang et al., 2019; Sun et al., 2021)). In the normal model with a known mean μ, our parameters of interest are θ = σ2 (a variance parameter) and θ = σ (a scale parameter). Therefore, we will select Stein’s loss function.

The motivation and contributions of our paper are summarized as follows. For the normal model with a known mean μ, the Bayes estimation of the variance parameter θ = σ2 under the conjugate prior which is an Inverse Gamma distribution is studied in Example 4.2.5 (p.236) of (Lehmann and Casella, 1998) and Example 1.3.5 (p.15) of (Mao and Tang, 2012). However, they only calculate the Bayes estimator with respect to a conjugate prior under the squared error loss. (Zhang, 2017) calculates the Bayes estimator of the variance parameter θ = σ2 of the normal model with a known mean with respect to the conjugate prior under Stein’s loss function which penalizes gross overestimation and gross underestimation equally, and the corresponding Posterior Expected Stein’s Loss (PESL). Motivated by the works of (Lehmann and Casella, 1998; Mao and Tang, 2012; Zhang, 2017), we want to calculate the Bayes estimators of the variance and scale parameters of the normal model with a known mean for the conjugate and noninformative priors under Stein’s loss function. The contributions of our paper are summarized as follows. In this paper, we have calculated the Bayes estimators of the variance parameter θ = σ2 with respect to the noninformative (Jeffreys’s, reference, and matching) priors under Stein’s loss function, and the corresponding Posterior Expected Stein’s Losses (PESLs). Moreover, we have calculated the Bayes estimators of the scale parameter θ = σ with respect to the conjugate and noninformative priors under Stein’s loss function, and the corresponding PESLs. For more literature on Bayesian estimation and inference, we refer readers to (Sindhu and Aslam, 2013a; Sindhu and Aslam, 2013b; Sindhu et al., 2013; Sindhu et al., 2016a; Sindhu et al., 2016b; Sindhu et al., 2016c; Sindhu et al., 2017; Sindhu et al., 2018; Sindhu and Hussain, 2018)

The rest of the paper is organized as follows. In the next Section 2, we analytically calculate the Bayes estimators of the variance and scale parameters of the normal model with a known mean, with respect to the conjugate and noninformative priors under Stein’s loss function, and the corresponding PESLs. We also analytically calculate the Bayes estimators under the squared error loss function, and the corresponding PESLs. The quantities (prior, posterior, three posterior expectations, two Bayes estimators, and two PESLs) and expressions of the variance and scale parameters for the conjugate and noninformative priors are summarized in two tables. Section 3 reports vast amount of numerical simulation results of the combination of the noninformative prior and the scale parameter to support the theoretical studies of two inequalities of the Bayes estimators and the PESLs, and that the PESLs depend only on the number of observations, but do not depend on the mean and the sample. In Section 4, we calculate the Bayes estimators and the PESLs of the variance and scale parameters of the S&P 500 monthly simple returns for the conjugate and noninformative priors. Some conclusions and discussions are provided in Section 5.

2 Bayes Estimator, PESL, IRSL, and BRSL

In this section, we will analytically calculate the Bayes estimator

$δsπ,θx$

of the variance parameter

$θ=σ2∈Θ=0,∞$

under Stein’s loss function, the PESL at

$δsπ,θx$

,

$PESLsπ,θx$

, and the Integrated Risk under Stein’s Loss (IRSL) at

$δsπ,θ$

,

$IRSLsπ,θ=BRSLπ,θ$

, which is also the Bayes Risk under Stein’s Loss (BRSL) for π, θ. See (Robert, 2007) for the definitions of the posterior expected loss, the integrated risk, and the Bayes risk. We will also analytically calculate the Bayes estimator

$δsπ,σx$

of the scale parameter

$σ∈Θ=0,∞$

under Stein’s loss function, the PESL at

$δsπ,σx$

,

$PESLsπ,σx$

, and the IRSL at

$δsπ,σ$

,

$IRSLsπ,σ=BRSLπ,σ$

, which is also the BRSL for π, σ.

Suppose that we observe X1, X2, …, Xn from the hierarchical normal model with a mixing variance parameter θ = σ2:

$Xi|θ∼iidNμ,θ,i=1,2,…,n,θ∼πθ,(1)$

where − < μ < is a known constant,

$Nμ,θ$

is the normal distribution with a known mean μ and an unknown variance θ, and

$πθ$

is the prior distribution of θ. For the normal model with a known mean μ, the Bayes estimation of the variance parameter θ = σ2 under the conjugate prior which is an Inverse Gamma distribution is studied in Example 4.2.5 (p.236) of (Lehmann and Casella, 1998) and Example 1.3.5 (p.15) of (Mao and Tang, 2012). However, they only calculate the Bayes estimator with respect to a conjugate prior under the squared error loss. (Zhang, 2017) calculates the Bayes estimator of the variance parameter θ = σ2 with respect to the conjugate prior under Stein’s loss function, and the corresponding PESL. Motivated by the works of (Lehmann and Casella, 1998; Mao and Tang, 2012; Zhang, 2017), we want to calculate the Bayes estimators of the variance parameter of the normal model with a known mean for the noninformative (Jeffreys’s, reference, and matching) priors under Stein’s loss function. The usual Bayes estimator with respect to a prior

$πθ$

is to calculate

$δ2π,θx=Eθ|x$

under the squared error loss function. As pointed out in the introduction, we should calculate and use the Bayes estimator of the variance parameter θ with respect to a prior

$πθ$

under Stein’s loss function, that is,

$δsπ,θx$

.

Alternatively, we may be interested in the scale parameter θ = σ. Motivated by the works of (Lehmann and Casella, 1998; Mao and Tang, 2012; Zhang, 2017), we also want to calculate the Bayes estimators of the scale parameter θ = σ with respect to the conjugate and noninformative priors under Stein’s loss function, and the corresponding PESLs. Suppose that we observe X1, X2, …, Xn from the hierarchical normal model with a mixing scale parameter θ = σ:

$Xi|σ∼iidNμ,σ2,i=1,2,…,n,σ∼πσ,(2)$

where − < μ < is a known constant,

$Nμ,σ2$

is the normal distribution with a known mean μ and an unknown variance σ2, and

$πσ$

is the prior distribution of σ. The usual Bayes estimator with respect to a prior

$πσ$

is to calculate

$δ2π,σx=Eσ|x$

under the squared error loss function. As pointed out in the introduction, we should calculate and use the Bayes estimator of the scale parameter σ with respect to a prior

$πσ$

under Stein’s loss function, that is,

$δsπ,σx$

.

Now let us explain why we choose Stein’s loss function on

$Θ=0,∞$

. Stein’s loss function is given by

$Lsθ,a=aθ−logaθ−1,(3)$

where θ > 0 is the unknown parameter of interest and a is an action or estimator. The squared error loss function is given by

The asymmetric Linear Exponential (LINEX) loss function ((Varian et al., 1975; Zellner, 1986; Robert, 2007)) is given by

$LLθ,a=eca−θ−ca−θ−1,(5)$

where c ≠ 0 serving to determine its shape. In particular, when c > 0, the LINEX loss function tends to exponentially, while when c < 0, the LINEX loss function tends to linearly. Note that on the positive restricted parameter space

$Θ=0,∞$

, Stein’s loss function penalizes gross overestimation and gross underestimation equally, that is, an action a will incur an infinite loss when it tends to 0 or . Whereas, the squared error loss function does not penalize gross overestimation and gross underestimation equally, as an action a will incur a finite loss (in fact θ2) when it tends to 0 and incur an infinite loss when it tends to . Similarly, the LINEX loss functions also do not penalize gross overestimation and gross underestimation equally, as an action a will incur a finite loss (in fact e + − 1) when it tends to 0 and incur an infinite loss when it tends to . Figure 1 shows the four loss functions on

$Θ=0,∞$

when θ = 2.

As pointed out by (Zhang, 2017), the Bayes estimator

minimizes the PESL, that is,

$δsπ,θx=argmina∈AELsθ,a|x,$

where

$A=ax:ax>0$

is an action space,

$a=ax>0$

is an action (estimator), which is a function only of x,

$Lsθ,a$

given by (Eq. 3) is Stein’s loss function, and θ > 0 is the unknown parameter of interest. Note that Stein’s loss function has a nice property that it penalizes gross overestimation and gross underestimation equally, that is, an action a will incur an infinite loss when it tends to 0 or . Moreover, note that θ may be the variance parameter σ2 or the scale parameter σ.

The usual Bayes estimator of θ is

$δ2π,θx=Eθ|x$

which minimizes the Posterior Expected Squared Error Loss. It is interesting to note that

$δsπ,θx≤δ2π,θx,(6)$

whose proof exploits Jensen’s inequality and the proof can be found in (Zhang, 2017). Note that the inequality (Eq. 6) is a special inequality in (Zhang et al., 2018). As calculated in (Zhang, 2017), the PESL at

$δsπ,θx=Eθ−1|x−1$

is

$PESLsπ,θx=ELsθ,a|xa=1E1θ|x=logE1θ|x+Elog⁡θ|x,$

and the PESL at

$δ2π,θx=Eθ|x$

is

$PESL2π,θx=ELsθ,a|xa=Eθ|x=Eθ|xE1θ|x−logEθ|x+Elog⁡θ|x−1.$

As observed in (Zhang, 2017),

$PESLsπ,θx≤PESL2π,θx,(7)$

which is a direct consequence of the general methodology for finding a Bayes estimator or due to

$δsπ,θx$

minimizes the PESL. The numerical simulations will exemplify (Eqs 6, 7) later. Note that the calculations of

$δsπ,θx$

,

$δ2π,θx$

,

$PESLsπ,θx$

, and

$PESL2π,θx$

depend only on the three expectations

$Eθ|x$

,

$Eθ−1|x$

, and

$Elog⁡θ|x$

.

2.1 Conjugate Prior

The problem of finding the Bayes estimator under a conjugate prior is a standard problem that is treated in almost every text on Mathematical Statistics.

The quantities and expressions of the variance and scale parameters of the normal models (Eqs 1, 2) with a known mean μ for the conjugate prior are summarized in Table 1. In the table, α > 0 and β > 0 are known constants,

$α∗=α+n2,β∗=1β+12∑i=1nxi−μ2−1,$
$ψz=Γ′zΓz=ddzlogΓz=digammaz$

is the digamma function, and

$Γz$

is the gamma function. In R software (R Core Team. R, 2021), the function digamma(z) calculates

$ψz$

. The quantities and expressions of the variance parameter θ = σ2 for the conjugate prior are calculated in and quoted from (Zhang, 2017). The calculations of the quantities and expressions of the scale parameter θ = σ for the conjugate prior can be found in the Supplementary Material. We remark that the calculations of the quantities and expressions in Table 1 are not trivial, especially

$Eπclog⁡θ|x$

.

2.2 Noninformative Priors

Famous noninformative priors include the Jeffreys’s ( (Jeffreys, 1961)), reference ( (Bernardo, 1979; Berger and Bernardo, 1992)), and matching ( (Tibshirani, 1989; Datta and Mukerjee, 2004)) priors. See also (Berger, 2006; Berger et al., 2015) and the references therein.

The Jeffreys’s noninformative prior for θ = σ2 is

$πJθ∝1θ or πJσ2∝1σ2.$

See Part I (p.66) of (Chen, 2014), where μ is assumed known in the normal model

$Nμ,θ$

. The Jeffreys’s noninformative prior for θ = σ is

See Example 3.5.6 (p.131) of (Robert, 2007), where μ is assumed known in the normal model

$Nμ,σ2$

.

Since μ is assumed known in the normal models, there is only one unknown parameter. Therefore, the reference prior is equal to the Jeffreys’s prior, and the matching prior is also equal to the Jeffreys’s prior (see pp.130–131 of (Ghosh et al., 2006)). In summary, when μ is assumed known in the normal models, the three noninformative priors equal, that is,

$πnθ=πJθ=πRθ=πMθ∝1θ$

and

$πnσ=πJσ=πRσ=πMσ∝1σ,$

where

$πn⋅$

stands for the noninformative prior.

Note that as in many statistics textbooks, the probability density function (pdf) of

$θ∼IGα,β$

is given by

$fθθ|α,β=1Γαβα1θα+1⁡exp−1βθ,θ>0,α>0,β>0.$

The conjugate prior of the scale parameter θ = σ is a Square Root of the Inverse Gamma (SRIG) distribution that we define below.

DEFINITION 1 Let

$θ=σ2∼IGα,β$

with α > 0 and β > 0. Then

$σ=θ∼SRIGα,β$

and the pdf of σ is given by

$fσσ|α,β=2Γαβα1σ2α+1⁡exp−1βσ2,σ>0,α>0,β>0.$

Definition 1 gives the definition of the SRIG distribution, which is the conjugate prior of the scale parameter θ = σ of the normal distribution. Because the SRIG distribution can not be found in standard textbooks, so we give its definition here. Moreover, Definition 1 is reasonable, since

$fσσ|α,β=fθθ|α,βθ′σ=1Γαβα1σ2α+1⁡exp−1βσ2⋅2σ=2Γαβα1σ2α+1⁡exp−1βσ2.$

We have the following proposition which gives the three expectations of the

$SRIGα,β$

distribution. The calculations needed in the proposition can be found in the Supplementary Material. We remark that the calculations of

$Eσ$

and

$Eσ−1$

are straightforward by utilizing a simple transformation of θ = σ2 and the integration of an

$IGα,β$

distribution. However, the calculations of

$Elog⁡σ$

is skillful by first a transformation of

$y=1/βσ2$

and then a change of the order of integration and differentiation.

PROPOSITION 1 Let

$σ=θ∼SRIGα,β$

with α > 0 and β > 0. Then

$Eσ=Γα−12Γαβ12,forα>12andβ>0,E1σ=Γα+12β12Γα,forα>0andβ>0,Elog⁡σ=−12log⁡β−12ψα,forα>0andβ>0.$

The relationship between the two distributions

$IGα,β$

and

$SRIGα,β$

are given in the following proposition whose proof can be found in the Supplementary Material. We remark that the proof of the proposition is straightforward by utilizing monotone transformations θ = σ2 and

$σ=θ$

.

PROPOSITION 2

$θ=σ2∼IGα,β$

if and only if

$σ=θ∼SRIGα,β$

, where α > 0 and β > 0.The posterior distributions of θ and σ for the noninformative priors are given in the following theorem whose proof can be found in the Supplementary Material.

THEOREM 1 Let

$X|θ∼Nμ,θ$

and

$X|σ∼Nμ,σ2$

where μ is known and θ = σ2 is unknown,

$πθ∝1θ$

, and

$πσ∝1σ$

. Then

$πθ|x∼IGα̃,β̃ and πσ|x∼SRIGα̃,β̃,$

where

$α̃=n2 and β̃=2∑i=1nxi−μ2.(8)$

We have the following two remarks for Theorem 1.

Remark 1 Let θ = σ2. In the derivation of

$πσ|x$

, if we derive it in this way,

$fσσ=πσ|x∝1σn+1⁡exp−12σ2∑i=1nxi−μ2=1σ2n+12⁡exp−12σ2∑i=1nxi−μ2=1θn+12⁡exp−12θ∑i=1nxi−μ2=fθθ∼IGα̃1,β̃,$

where

$α̃1=n−12 and β̃=2∑i=1nxi−μ2,$

then by Proposition 2,

$fσσ=πσ|x∼SRIGα̃1,β̃$

, which is different from

$SRIGα̃,β̃$

. In fact, the above practice is equivalent to the derivation of the pdf of θ in terms of the pdf of σ by

$fθθ=fσσ$

, ignoring the

$σ′θ$

term, which is obviously wrong. Therefore, the above derivation which is a pitfall for incautious users is wrong. ‖

Remark 2 The two posterior distributions in Theorem 1,

$πθ|x∼IGα̃,β̃$

and

$πσ|x∼SRIGα̃,β̃$

, follow Proposition 2 by accident. We have

$fθθ=πθ|x∝fx|θπθ∝fx|θ1θ∼IGα̃,β̃$

and

$fσσ=πσ|x∝fx|σπσ∝fx|θ1σ∼SRIGα̃,β̃.$

Note that

$σ=θ$

, and thus

$fσσσ′θ∝fx|θ1σ12θ=fx|θ1θ12θ=fx|θ12θ∝fθθ,(9)$

which is the reason why

$πθ|x=fθθ$

and

$πσ|x=fσσ$

follow Proposition 2. Note that the posterior distributions depend on the prior distributions. If the prior distributions

$πθ$

and

$πσ$

are selected different from

$1θ$

and

$1σ$

, then the relationship (Eq. 9) may not be satisfied, and thus

$πθ|x$

and

$πσ|x$

may not follow Proposition 2. ‖

2.2.1 The Quantities and Expressions of the Variance Parameter

In this subsubsection, we will calculate the expressions of the quantities (three posterior expectations, two Bayes estimators, and two PESLs) of the variance parameter θ = σ2.

Now we calculate the three expectations

$Eθ|x$

,

$Eθ−1|x$

, and

$Elog⁡θ|x$

for the variance parameter θ = σ2. By Theorem 1,

$πθ|x∼IGα̃,β̃$

, and thus

$Eθ|x=1α̃−1β̃,α̃>1 and E1θ|x=α̃β̃.$

From (Zhang, 2017), we know that

$Elog⁡θ|x=−logβ̃−ψα̃.$

It is easy to see that, for

$α̃>1$

,

$δsπ,θx=1E1θ|x=1α̃β̃<1α̃−1β̃=Eθ|x=δ2π,θx,$

which exemplifies (Eq. 6). From (Zhang, 2017), we find that

$PESLsπ,θx=logα̃−ψα̃, for α̃>0,$

and

$PESL2π,θx=1α̃−1+logα̃−1−ψα̃, for α̃>1.$

It can be directly proved that

$PESLsπ,θx≤PESL2π,θx$

for

$α̃>1$

, which exemplifies (Eq. 7), and its proof which exploits the Taylor series expansion for ex can be found in the Supplementary Material. Note that

$PESLsπ,θx$

and

$PESL2π,θx$

depend only on

$α̃=n/2$

. Therefore, they depend only on n, but do not depend on μ and x. Numerical simulations will exemplify this result.

The IRSL at

$δsπ,θ$

or the BRSL for θ = σ2 is (similar to (Robert, 2007))

$IRSLsπ,θ=BRSLπ,θ=rπ,δsπ,θ=EπRθ,δsπ,θ=∫ΘRθ,δsπ,θπθdθ=∫Θ∫XLθ,δsπ,θxfx|θdxπθdθ=∫X∫ΘLθ,δsπ,θxfx|θπθdθdx=∫X∫ΘLθ,δsπ,θxπθ|xdθmπ,θxdx=∫XPESLπ,θax|xa=δsπ,θmπ,θxdx=∫XPESLsπ,θxmπ,θxdx=∫Xlogα̃−ψα̃mπ,θxdx=logα̃−ψα̃=PESLsπ,θx,$

since

$α̃$

does not depend on x, where

$mπ,θx=∫0∞fx|θπθdθ$

is the marginal density of x with prior

$πθ$

.

2.2.2 The Quantities and Expressions of the Scale Parameter

In this subsubsection, we will calculate the expressions of the quantities (three posterior expectations, two Bayes estimators, and two PESLs) of the scale parameter θ = σ.

Now let us calculate

$δsπ,σx$

,

$δ2π,σx$

,

$PESLsπ,σx$

, and

$PESL2π,σx$

for the scale parameter σ. To calculate these quantities, we need to calculate the three expectations

$Eσ|x$

,

$Eσ−1|x$

, and

$Elog⁡σ|x$

. Since

$πσ|x∼SRIGα̃,β̃$

by Theorem 1, from Proposition 1, we have

$Eσ|x=Γα̃−12Γα̃β̃12, for α̃>12 and β̃>0,(10)$
$E1σ|x=Γα̃+12β̃12Γα̃, for α̃>0 and β̃>0,(11)$
$Elog⁡σ|x=−12logβ̃−12ψα̃, for α̃>0 and β̃>0.(12)$

It can be proved that, for

$α̃>12$

,

$δsπ,σx=1E1σ|x=Γα̃Γα̃+12β̃12<Γα̃−12Γα̃β̃12=Eσ|x=δ2π,σx,$

which exemplifies (Eq. 6), and the proof which exploits the positivity of

$ψ′x$

can be found in the Supplementary Material.

Now we calculate

$PESLsπ,σx$

and

$PESL2π,σx$

for the scale parameter σ. From (Zhang, 2017), we know that the PESL at

$δsπ,σx=Eσ−1|x−1$

is

$PESLsπ,σx=ELsθ,a|xa=1E1σ|x=logE1σ|x+Elog⁡σ|x,$

and the PESL at

$δ2π,σx=Eσ|x$

is

$PESL2π,σx=ELsθ,a|xa=Eσ|x=Eσ|xE1σ|x−1−logEσ|x+Elog⁡σ|x.$

Substituting (Eqs 10, 11, 12), into the above expressions, we obtain

$PESLsπ,σx=logΓα̃+12β̃12Γα̃−12logβ̃−12ψα̃=logΓα̃+12−logΓα̃−12ψα̃,$

for

$α̃>0$

and

$β̃>0$

, and

$PESL2π,σx=Γα̃−12Γα̃β̃12Γα̃+12β̃12Γα̃−1−logΓα̃−12Γα̃β̃12−12logβ̃−12ψα̃=Γα̃−12Γα̃+12Γ2α̃−1−logΓα̃−12+logΓα̃−12ψα̃,$

for

$α̃>12$

and

$β̃>0$

. It can be directly proved that

$PESLsπ,σx≤PESL2π,σx$

for

$α̃>12$

and

$β̃>0$

, which exemplifies (Eq. 7), and its proof which exploits the Taylor series expansion for log  u with u near 1 can be found in the Supplementary Material. Note that

$PESLsπ,σx$

and

$PESL2π,σx$

depend only on

$α̃=n/2$

. Therefore, they depend only on n, but do not depend on μ and x. Numerical simulations will exemplify this result.

The IRSL at

$δsπ,σ$

or the BRSL for θ = σ is (similar to (Robert, 2007))

$IRSLsπ,σ=BRSLπ,σ=rπ,δsπ,σ=EπRσ,δsπ,σ=∫ΣRσ,δsπ,σπσdσ=∫Σ∫XLσ,δsπ,σxfx|σdxπσdσ=∫X∫ΣLσ,δsπ,σxfx|σπσdσdx=∫X∫ΣLσ,δsπ,σxπσ|xdσmπ,σxdx=∫XPESLπ,σax|xa=δsπ,σmπ,σxdx=∫XPESLsπ,σxmπ,σxdx=∫XlogΓα̃+12−logΓα̃−12ψα̃mπ,σxdx=logΓα̃+12−logΓα̃−12ψα̃=PESLsπ,σx,$

since

$α̃$

does not depend on x, where

$mπ,σx=∫0∞fx|σπσdσ$

is the marginal density of x with prior

$πσ$

.

The quantities and expressions of the variance and scale parameters for the noninformative priors are summarized in Table 2. In the table,

$α̃$

and

$β̃$

are given by (Eq. 8).

From Tables 1, 2, we find that there are four combinations of the expressions of the quantities: conjugate prior and variance parameter, conjugate prior and scale parameter, noninformative prior and variance parameter, and noninformative prior and scale parameter. The forms of the expressions of the quantities are the same for the variance parameter under the conjugate and noninformative priors, since they have the same Inverse Gamma posterior distributions. Similarly, the forms of the expressions of the quantities are the same for the scale parameter under the conjugate and noninformative priors, since they have the same Square Root of the Inverse Gamma posterior distributions.

The inequalities (Eqs 6, 7) exist in Tables 1, 2. In fact, there are 8 inequalities in Tables 1, 2 and 4 inequalities in each table. Since the forms of the expressions of the quantities are the same in Tables 1, 2, with the only difference of the parameters, there are actually 4 different inequalities which are in Table 2. One inequality of the four inequalities about the Bayes estimators is obvious, and the proofs of the other three inequalities can be found in the Supplementary Material.

3 Numerical Simulations

In this section, we will numerically exemplify the theoretical studies of (Eqs 6, 7), and that the PESLs depend only on n, but do not depend on μ and x. The numerical simulation results are similar for the four combinations of the expressions of the quantities, and thus we only present the results for the combination of the noninformative prior and the scale parameter.

First, we fix μ = 0 and n = 10, and assume that σ = 1 is drawn from the improper prior distribution. After that, we draw a random sample

$x=rnorm(n = n, mean = μ, sd = σ)$

from N(μ, σ2).

To generate a random sample

$σ=σ1,…,σk$

with k = 1000 from

we will adopt the following algorithm. First, compute

$α̃$

and

$β̃$

from (Eq. 8). Second, generate a random sample

$G=rgamma(n = k, shape = α̃, scale = β̃)∼Gα̃,β̃.$

Third, compute

Fourth, compute

Hence, σ is a random sample from the

$SRIGα̃,β̃$

distribution. Figure 2 shows the histogram of σ|x and the density estimation curve of πn(σ|x). It is πn(σ|x) that we find

$δsπn,σx$

to minimize the PESL. From the figure, we see that the

$SRIGα̃,β̃$

distribution is left peaked, right skewed, and continuous.

The Bayes estimators (

$δsπn,σx$

and

$δ2πn,σx$

) and the PESLs (

$PESLsπn,σx$

and

$PESL2πn,σx$

) are computed by the following algorithm. First, compute

$α̃$

and

$β̃$

from (Eq. 8). Second, compute

$E1=Eσ|x=Γα̃−12Γα̃β̃12,E2=E1σ|x=Γα̃+12β̃12Γα̃,E3=Elog⁡σ|x=−12logβ̃−12ψα̃.$

Third, compute

$δsπn,σx=1E2,δ2πn,σx=E1,PESLsπn,σx=logE2+E3,PESL2πn,σx=E1×E2−logE1+E3−1.$

Numerical results show that

$δsπn,σx=0.7712483<0.8152161=δ2πn,σx$

and

$PESLsπn,σx=0.0267013<0.02826706=PESL2πn,σx,$

which exemplify the theoretical studies of (6) and (7).

In Figure 3, we fix μ = 0 and n = 10, but allow the seed number to change from 1 to 10 (i.e., we change x). From the figure we see that the estimators and PESLs are functions of x. We see from the left plot of the figure that the estimators depend on x in an unpredictable manner, and

$δsπn,σx$

are unanimously smaller than

$δ2πn,σx$

, and thus (Eq. 6) is exemplified. The two Bayes estimators are distinguishable since we fix n = 10 to be a small number. The right plot of the figure exhibits that the PESLs do not depend on x, and

$PESLsπn,σx$

are unanimously smaller than

$PESL2πn,σx$

, and thus (Eq. 7) is exemplified.

Now we allow one of the two parameters μ and n to change, holding other parameters fixed. Moreover, we also assume that the sample x is fixed, as it is the case for the real data. Figure 4 shows the estimators and PESLs as functions of μ and n. We see from the left plots of the figure that the estimators depend on μ and n, and (Eq. 6) is exemplified. More specifically, the estimators are first decreasing and then increasing functions of μ, and the estimators attain the minimum when μ = 0. However, the estimators fluctuate around some value when n increases. The right plots of the figure exhibit that the PESLs depend only on n, but do not depend on μ , and (Eq. 7) is exemplified. More specifically, the PESLs are decreasing functions of n. Furthermore, the two PESLs as functions of n are indistinguishable, as the two PESLs are very close. In summary, the results of the figure exemplify the theoretical studies of (Eqs 6, 7).

Since the estimators

$δsπn,σx$

and

$δ2πn,σx$

and the PESLs

$PESLsπn,σx$

and

$PESL2πn,σx$

depend on

$α̃$

and

$β̃$

, where

$α̃>1/2$

and

$β̃>0$

, we can plot the surfaces of the estimators and the PESLs on the domain

$α̃,β̃∈(0.5,10]×(0,10]=D$

via the R function persp3d() in the R package rgl (see (Adler and Murdoch, 2017; Zhang et al., 2017; Zhang et al., 2019; Sun et al., 2021)). We remark that the R function persp() in the R package graphics can not add another surface to the existing surface, but persp3d() can. Moreover, persp3d() allows one to rotate the perspective plots of the surface according to one’s wishes. Figure 5 plots the surfaces of the estimators and the PESLs, and the surfaces of the difference of the estimators and the difference of the PESLs. From the left two plots of the figure, we see that

$δsπn,σx<δ2πn,σx$

for all

$α̃,β̃$

on D, which exemplifies (Eq. 6). From the right two plots of the figure, we see that

$PESLsπn,σx

for all

$α̃,β̃$

on D, which exemplifies (Eq. 7). In summary, the results of the figure exemplify the theoretical studies of (Eqs 6, 7). FIGURE 5. The domain for

$α̃,β̃$

is D = (0.5, 10] × (0, 10] for all the plots. a is for

$α̃$

and b is for

$β̃$

in the axes of all the plots. The red surface is for

$δ2πn,σx$

and the blue surface is for

$δsπn,σx$

in the upper two plots. (upper left) The estimators as functions of

$α̃$

and

$β̃$

.

$δsπn,σx<δ2πn,σx$

for all

$α̃,β̃$

on D. (upper right) The PESLs as functions of

$α̃$

and

$β̃$

.

$PESLsπn,σx

for all

$α̃,β̃$

on D. (lower left) The surface of

$δ2πn,σx−δsπn,σx$

which is positive for all

$α̃,β̃$

on D. (lower right) The surface of

$PESL2πn,σx−PESLsπn,σx$

which is also positive for all

$α̃,β̃$

on D.

4 A Real Data Example

In this section, we exploit the data from finance. The R package quantmod ( (Ryan and Ulrich, 2017)) is exploited to download the data ˆGSPC (the S&P 500) during 2020-04-24 and 2021-07-02 from “finance.yahoo.com.” It is commonly believed that the monthly simple returns of the index data or the stock data are normally distributed. It is simple to check that the S&P 500 monthly simple returns follow the normal model. Usually, the data from real examples can be regarded as iid from the normal model with an unknown mean μ. However, the mean μ could be estimated by prior information or historical information. Alternatively, the mean μ could be estimated by the sample mean. Therefore, for simplicity, we assume that the mean μ is known. Assume that

for the S&P 500 monthly simple returns.

The Bayes estimators and the PESLs of the variance and scale parameters of the S&P 500 monthly simple returns for the conjugate and noninformative priors are summarized in Table 3. From the table, we observe the following facts.

• The two inequalities (Eqs 6, 7) are exemplified.

• Given the prior (conjugate or noninformative), the Bayes estimators are similar across different loss functions (Stein’s or squared error).

• Given the loss function, the Bayes estimators are quite different across different priors. Therefore, the prior has a larger influence than the loss function in calculating the Bayes estimators.

More results (the data of the S&P 500 monthly simple returns, the plot of the S&P 500 monthly close prices, the plot of the S&P 500 monthly simple returns, the histogram of the S&P 500 monthly simple returns) for the real data example can be found in the Supplementary Material due to space limitations.

5 Conclusions and Discussions

For the variance (θ = σ2) and scale (θ = σ) parameters of the normal model with a known mean μ, we recommend and analytically calculate the Bayes estimators,

$δsπ,θx$

, with respect to the conjugate and noninformative (Jeffreys’s, reference, and matching) priors under Stein’s loss function which penalizes gross overestimation and gross underestimation equally. These estimators minimize the PESLs. We also analytically calculate the Bayes estimators,

$δ2π,θx=Eθ|x$

, with respect to the conjugate and noninformative priors under the squared error loss function, and the corresponding PESLs. The quantities (

$πθ$

,

$πθ|x$

,

$Eπθ|x$

,

$Eπθ−1|x$

,

$Eπlog⁡θ|x$

,

$δsπ,θx$

,

$δ2π,θx$

,

$PESLsπ,θx$

,

$PESL2π,θx$
$Eπlog⁡θ|x$

, which is essential for the calculation of

$PESLsπ,θx$

and

$PESL2π,θx$

, depends on the digamma function.

Proposition 1 gives the three expectations of the

$SRIGα,β$

distribution. Moreover, Proposition 2 gives the relationship between the two distributions

$IGα,β$

and

$SRIGα,β$

.

For the conjugate and noninformative priors, the posterior distribution of θ = σ2,

$πθ|x$

, follows an Inverse Gamma distribution, and the posterior distribution of σ,

$πσ|x$

, follows an SRIG distribution which is defined in Definition 1.

We find that the IRSL at

$δsπ,θ$

or the BRSL for θ = σ2 is

$PESLsπ,θx=logα̃−ψα̃.$

$δsπ,σ$

or the BRSL for θ = σ is

$PESLsπ,σx=logΓα̃+12−logΓα̃−12ψα̃.$

The numerical simulations of the combination of the noninformative prior and the scale parameter exemplify the theoretical studies of (Eqs 6, 7), and that the PESLs depend only on n, but do not depend on μ and x. Moreover, in the real data example, we have calculated the Bayes estimators and the PESLs of the variance and scale parameters of the S&P 500 monthly simple returns for the conjugate and noninformative priors.

$σ̂$

is the Maximum Likelihood Estimator (MLE) of σ, then

$σ̂2$

is the MLE of σ2. In Bayesian paradigm, we usually should estimate the variance parameter σ2 and the scale parameter σ separately. In Table 2, we find that

$δsπn,σ2x=1α̃β̃ and δsπn,σx=Γα̃Γα̃+12β̃12.$

It is easy to see that

$δsπn,σ2x≠δsπn,σx2.$

Similarly,

$δ2πn,σ2x≠δ2πn,σx2.$

When there is no prior information about the unknown parameter of interest, we prefer the noninformative prior, as the hyperparameters α and β are somewhat arbitrary for the conjugate prior.

We remark that the Bayes estimator under Stein’s loss function is more appropriate than that under the squared error loss function, not because the former is smaller, but because Stein’s loss function which penalizes gross overestimation and gross underestimation equally is more appropriate for the positive restricted parameter.

Data Availability Statement

The original contributions presented in the study are included in the article/Supplementary Material, further inquiries can be directed to the corresponding author.

Author Contributions

This work was carried out in collaboration among all authors. Author YYZ wrote the first draft of the article. Author TZR did literature searches and revised the article. Author MML revised the article. All authors read and approved the final article.

Funding

The research was supported by the Ministry of Education (MOE) project of Humanities and Social Sciences on the west and the border area (20XJC910001), the National Social Science Fund of China (21XTJ001), the National Natural Science Foundation of China (12001068; 72071019), and the Fundamental Research Funds for the Central Universities (2020CDJQY-Z001; 2021CDJQY-047).

Conflict of Interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher’s Note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors, and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Acknowledgments

The authors are extremely grateful to the editor, the guest associate editor, and the reviewers for their insightful comments that led to significant improvement of the article.