HW 2

This HW is a combination of questions from Chapter 3 Robert, Casella, and Casella (1999) and Chapter 9 of Owen (2013). I’ve rewritten some of the questions to make them clearer where needed, but if anything is still unclear, please let me know!

Question 1

Suppose we want an estimate of $P(X > 2.5)$ when $X$ is normally distributed with mean $0$ and variance $1$ and we use a simple Monte Carlo estimator to estimate this probability. This estimator will use $h(x) = \ind{x > 2.5}$: \[ \begin{aligned} \hat{\mu} & = \frac{1}{S} \sum_{i=1}^S h(X_i) \\ X_i & \overset{\text{iid}}{\sim} \text{N}(0, 1),\, i = 1, \dots, S \end{aligned} \] Our Monte Carlo standard error is then: \[ \text{Var}(\hat{\mu}) = \frac{\Var{h(X_1)}}{S} \]

How many samples would we need to get the Monte Carlo standard error down to $O(10^{-4})$?
Show this using simulation in R.
How many digits should we report for our point estimate for $\hat{\mu}$ if we want a 0.95 probability that our reported point estimate wouldn’t change with another sample? Put another way, when we create a 95% confidence interval for $\mu$ using our estimate $\hat{\mu}$ and $\Var{\hat{\mu}}$, how many digits are common between our lower bound and our upper bound?

Question 2

Let $X \sim \text{Gamma}(1, 1)$

Show that $P(X > 5.3) \approx 0.005$ using simulations.
Using the following fact about the asymptotic sampling distribution for the $\alpha^{\mathrm{th}}$ quantile out of $S$ samples, denoted as $a$ below: \[ \hat{F}^{-1}(\alpha) \overset{\text{asymp}}{\sim} N\lp a, \frac{\alpha (1 - \alpha)}{S f^2(a)}\rp \] Determine the $0.995^{\mathrm{th}}$ quantile of the $\text{Gamma}(1,1)$ to three digits of accuracy with $99\%$ probability (i.e. to report $5.xx \pm z_{0.995}\text{MCSE}$ where $z_{0.995}$ is the upper 0.995 quantile of a standard normal RV.

Question 3

The density function for the truncated exponential distribution (denoted as $X \sim \text{TE}(\lambda, a)$) is \[ g(x;\lambda, a) = \ind{x > a}\lambda e^{-\lambda(x - a)} \] Show that $X \sim \text{TE}(\lambda,a)$ is equal in distribution to $Y \sim \text{Exp}(\lambda)$ and $X = Y + a$
Using $g$ from part a. as \[ g(x) = \ind{x > 2.5}e^{-(x - 2.5)} \] as an importance density, construct an importance sampling estimator $\hat{\mu}_{\text{IS}}$ and determine the number of samples required to estimate $P(Z > 2.5)$ where $Z \sim \text{N}(0,1)$ to the same accuracy as you did in question 1a. This will require that you use the variance formula for an importance sampling estimator that we derived in class (see here: Variance formula and to do some calculus. When computing your variance estimate, you may use the pnorm function.
Again, using Variance formula and the logic we followed in class to minimize variance of importance sampling estimators, and the fact that $P(Z > x) \approx \frac{1}{x\sqrt{2 \pi}} e^{-x^2/2}$ when $Z \sim \text{N}(0,1)$ find $\lambda$ that minimizes the variance of the importance sampling estimator.

Question 4

Owen Chapter 9, problem 9.1

Question 5

This is adapted from Gelman et al. (2014).

The target density is a standard normal density, while the importance density is a t distribution with 3 degrees of freedom, with a mean of zero and a scale parameter such that the second derivative of the log-density evaluated at the mode is equal to -1 (note that this matches the normal distribution’s curvature at 0).

What is the scale parameter for the t distribution?
Draw 100 samples from the importance distribution and make a histogram of the log of the importance weights
Estimate $\Exp{X}$ and $\Var{X}$ using the importance sampling estimator and compare to the true values
Repeat b and c for $10{,}000$ samples
Repeat b, c, and d where we use the normal distribution as the importance density and the target distribution is the same t distribution you used above.
Comment on why the estimates of the variance in part e. are too small.

Question 6

Grad students only

Owen Chapter 9, problem 9.7. We worked through part of this in class, see normal example

For the second part of the question, give the ratio of the variances.

References

Gelman, Andrew, John Carlin, Hal S Stern, David B Dunson, Aki Vehtari, and Donald B Rubin. 2014. Bayesian Data Analysis. 3rd Edition. Chapman & Hall/CRC.

Owen, Art B. 2013. Monte Carlo Theory, Methods and Examples. https://artowen.su.domains/mc/.

Robert, Christian P, George Casella, and George Casella. 1999. Monte Carlo Statistical Methods. Vol. 2. Springer.

--- title: "HW 2" bibliography: refs.bib format: html: html-math-method: method: mathjax url: "https://cdn.jsdelivr.net/npm/mathjax@4.1/tex-mml-chtml.js" include-in-header: text: | <script> window.MathJax = { startup: { ready() { MathJax.startup.defaultReady(); const {STATE} = MathJax._.core.MathItem; MathJax.tex2mml(String.raw` \newcommand{\lp}{\left(} \newcommand{\rp}{\right)} \newcommand{\R}{\mathbb{R}} \newcommand{\Z}{\mathbb{Z}} \newcommand{\lb}{\left[} \newcommand{\rb}{\right]} \newcommand{\ind}[1]{\mathbbm{1}\lp#1\rp} \newcommand{\Exp}[1]{\mathbb{E} \lb #1 \rb} \newcommand{\ExpA}[2]{\mathbb{E}_{#2} \lb #1 \rb} \newcommand{\Prob}[2]{\mathbb{P}_{#2} \lp #1 \rp} \newcommand{\indy}{\mathrel{\unicode{x2AEB}}} \newcommand{\comp}{^\mathsf{c}} \newcommand{\widebar}[1]{\overline{\! #1}} \newcommand{\Var}[1]{\mathrm{Var}\lp #1 \rp} \newcommand{\VarA}[2]{\mathrm{Var}_{#2}\lp #1 \rp} \newcommand{\norm}[1]{\left\lVert #1 \right\rVert} \newcommand{\ddtA}[1]{\frac{\partial^2}{\partial #1^2}} \newcommand{\ddtB}[2]{\frac{\partial^2}{\partial #1 \partial #2}} \newcommand{\diff}[1]{{\mathrm{d}}\!\lp #1 \rp} \newcommand{\veco}[1]{{\mathrm{vec}}\lp #1 \rp} \newcommand{\tr}[1]{{\mathrm{tr}}\lp #1 \rp} `); } }, loader: {load: ['[tex]/bbm','[tex]/upgreek']}, tex: {packages: {'[+]': ['bbm','upgreek']}} }; </script> code-tools: source: true toggle: false caption: "Download Source" --- This HW is a combination of questions from Chapter 3 @robert1999monte and Chapter 9 of @mcbook. I've rewritten some of the questions to make them clearer where needed, but if anything is still unclear, please let me know! ## Question 1 Suppose we want an estimate of $P(X > 2.5)$ when $X$ is normally distributed with mean $0$ and variance $1$ and we use a simple Monte Carlo estimator to estimate this probability. This estimator will use $h(x) = \ind{x > 2.5}$: $$ \begin{aligned} \hat{\mu} & = \frac{1}{S} \sum_{i=1}^S h(X_i) \\ X_i & \overset{\text{iid}}{\sim} \text{N}(0, 1),\, i = 1, \dots, S \end{aligned} $$ Our Monte Carlo standard error is then: $$ \text{Var}(\hat{\mu}) = \frac{\Var{h(X_1)}}{S} $$ a. How many samples would we need to get the Monte Carlo standard error down to $O(10^{-4})$? b. Show this using simulation in R. c. How many digits should we report for our point estimate for $\hat{\mu}$ if we want a 0.95 probability that our reported point estimate wouldn't change with another sample? Put another way, when we create a 95% confidence interval for $\mu$ using our estimate $\hat{\mu}$ and $\Var{\hat{\mu}}$, how many digits are common between our lower bound and our upper bound? ## Question 2 Let $X \sim \text{Gamma}(1, 1)$ a. Show that $P(X > 5.3) \approx 0.005$ using simulations. b. Using the following fact about the asymptotic sampling distribution for the $\alpha^{\mathrm{th}}$ quantile out of $S$ samples, denoted as $a$ below: $$ \hat{F}^{-1}(\alpha) \overset{\text{asymp}}{\sim} N\lp a, \frac{\alpha (1 - \alpha)}{S f^2(a)}\rp $$ Determine the $0.995^{\mathrm{th}}$ quantile of the $\text{Gamma}(1,1)$ to three digits of accuracy with $99\%$ probability (i.e. to report $5.xx \pm z_{0.995}\text{MCSE}$ where $z_{0.995}$ is the upper 0.995 quantile of a standard normal RV. ## Question 3 a. The density function for the truncated exponential distribution (denoted as $X \sim \text{TE}(\lambda, a)$) is $$ g(x;\lambda, a) = \ind{x > a}\lambda e^{-\lambda(x - a)} $$ Show that $X \sim \text{TE}(\lambda,a)$ is equal in distribution to $Y \sim \text{Exp}(\lambda)$ and $X = Y + a$ b. Using $g$ from part a. as $$ g(x) = \ind{x > 2.5}e^{-(x - 2.5)} $$ as an importance density, construct an importance sampling estimator $\hat{\mu}_{\text{IS}}$ and determine the number of samples required to estimate $P(Z > 2.5)$ where $Z \sim \text{N}(0,1)$ to the same accuracy as you did in question 1a. This will require that you use the variance formula for an importance sampling estimator that we derived in class (see here: [Variance formula](../lectures/Lecture-4-Importance-Sampling.qmd#eq-var-is) and to do some calculus. When computing your variance estimate, you may use the `pnorm` function. c. Again, using [Variance formula](../lectures/Lecture-4-Importance-Sampling.qmd#eq-var-is) and the logic we followed in class to minimize variance of importance sampling estimators, and the fact that $P(Z > x) \approx \frac{1}{x\sqrt{2 \pi}} e^{-x^2/2}$ when $Z \sim \text{N}(0,1)$ find $\lambda$ that minimizes the variance of the importance sampling estimator. ## Question 4 Owen Chapter 9, problem 9.1 ## Question 5 This is adapted from @gelmanbayesian. The target density is a standard normal density, while the importance density is a t distribution with 3 degrees of freedom, with a mean of zero and a scale parameter such that the second derivative of the log-density evaluated at the mode is equal to -1 (note that this matches the normal distribution's curvature at 0). a. What is the scale parameter for the t distribution? b. Draw 100 samples from the importance distribution and make a histogram of the log of the importance weights c. Estimate $\Exp{X}$ and $\Var{X}$ using the importance sampling estimator and compare to the true values d. Repeat b and c for $10{,}000$ samples e. Repeat b, c, and d where we use the normal distribution as the importance density and the target distribution is the same t distribution you used above. f. Comment on why the estimates of the variance in part e. are too small. ## Question 6 Grad students only Owen Chapter 9, problem 9.7. We worked through part of this in class, see [normal example](../lectures/Lecture-4-Importance-Sampling.qmd#exm-normal-is) For the second part of the question, give the ratio of the variances.