---
title: "High Dimensional Example of MultiEFM"
author: "Ruihan Zhang and Wei Liu"
date: "`r Sys.Date()`"
output: rmarkdown::html_vignette
vignette: >
  %\VignetteIndexEntry{High Dimensional Example of MultiEFM}
  %\VignetteEngine{knitr::rmarkdown}
  %\VignetteEncoding{UTF-8}
---

```{r, include = FALSE}
knitr::opts_chunk$set(
  collapse = TRUE,
  comment = "#>"
)
```

This vignette introduces the usage of MultiEFM for the analysis of high-dimensional multi-study multivariate data with heavy tail. In high-dimensional settings, computational efficiency and robustness are particularly crucial.

The package can be loaded with the command, and define some metric functions:

```{r prep, message=FALSE, warning=FALSE,eval = TRUE}
library(MultiEFM)
library(irlba)   


trace_statistic_fun <- function(H, H0){
  tr_fun <- function(x) sum(diag(x))
  mat1 <- t(H0) %*% H %*% qr.solve(t(H) %*% H) %*% t(H) %*% H0
  return(tr_fun(mat1) / tr_fun(t(H0) %*% H0))
}
trace_list_fun <- function(Hlist, H0list){
  trvec <- sapply(seq_along(Hlist), function(i) trace_statistic_fun(Hlist[[i]], H0list[[i]]))
  return(mean(trvec, na.rm = TRUE))
}
```


## Generate High-Dimensional Simulated Data
First, we generate the simulated data with heavy tail, where the error term follows from a multivariate t-distribution with degree of freedom 2. To simulate a high-dimensional scenario while ensuring computational feasibility, we set the feature dimension to p = 300 and sample sizes to 80 and 100.
```{r gene, message=FALSE, warning=FALSE,eval = TRUE}
set.seed(1)
nu <- 2 # nu is set to 2 for heavier tails
p <- 300 # High-dimensional setting
nvec <- c(80, 100);  q <- 3; qs <- c(2,2); S <- length(nvec)
sigma2_eps <- 1
datList <- gendata_simu_robust(seed=1, nvec=nvec, p=p, q=q, qs=qs, rho=c(5,5), err.type='mvt', nu=nu)
XList <- datList$Xlist
```

## Fit the MultiEFM model
Fit the MultiEFM model using the function MultiEFM() in the R package MultiEFM. Users can use ?MultiEFM to see the details about this function. For two matrices $\widehat D$ and $D$, we use trace statistic to measure their similarity. The trace statistic ranges from 0 to 1, with higher values indicating better performance.  

```{r fit, message=FALSE, warning=FALSE,eval = TRUE}
tic <- proc.time()
res <- MultiEFM(XList, q=q, qs_vec=qs, verbose = FALSE)
toc <- proc.time()
time_use <- toc[3] - tic[3]
```

## Performance Evaluation
We summarized the estimation accuracy metrics for MultiEFM by calculating the trace statistics for loading and factor score matrices against the true parameters.

```{r perf, message=FALSE, warning=FALSE,eval =TRUE}
results <- data.frame(
  Metric = c('Shared Loading (A)', 'Specific Loading (B)', 'Shared Factor (F)', 'Specific Factor (H)', 'Time (s)'),
  Value = c(trace_statistic_fun(res$A, datList$A0),
            trace_list_fun(res$B, datList$Blist0),
            trace_list_fun(res$F, datList$Flist),
            trace_list_fun(res$H, datList$Hlist),
            time_use)
)
print(results)
```


## Select the parameters
We applied the proposed TSP method to select the number of factors. The results showed that the method has the potential to identify the true values.

```{r auto, message=FALSE, warning=FALSE,eval = TRUE}
hq_res <- selectFac.MultiEFM(XList, q_max=10, qs_max=5, verbose = FALSE)
message("Estimated shared q = ", hq_res$hq, " VS true q = ", q)
```

<details>
<summary>Session Info</summary>

```{r setdown, message=FALSE, warning=FALSE,eval = TRUE}
sessionInfo()
```

</details>