# Jackknife acceleration (BCa) · Permutation tests · Confidence guidance

## 1. Jackknife acceleration `a`

Used inside BCa bootstrap CIs.

\[
a = \frac{\sum_i (\bar\theta_{(\cdot)} - \theta_{(i)})^3}{6\bigl(\sum_i (\bar\theta_{(\cdot)} - \theta_{(i)})^2\bigr)^{3/2}}
\]

- \(\theta_{(i)}\) = statistic with observation \(i\) removed  
- \(\bar\theta_{(\cdot)}\) = mean of leave-one-out values  
- Small \(|a|\) → almost percentile CI; large \(|a|\) → skew correction matters  

```python
from correlation_metrics import jackknife_acceleration, spearman, bootstrap_report

jk = jackknife_acceleration(da, qe, stat_fn=spearman)
print(jk["acceleration"], jk["jackknife_bias"])

# BCa already uses this internally:
print(bootstrap_report(da, qe, stat="spearman")["ci_bca"])
print(bootstrap_report(da, qe, stat="spearman")["acceleration"])
```

Jackknife **bias** (separate from bootstrap bias):

\[
\mathrm{Bias}_{\mathrm{jack}} \approx (n-1)(\bar\theta_{(\cdot)} - \hat\theta)
\]

---

## 2. Permutation test for significance

**H0:** DA and QE are independent (any pairing equally likely).

1. Compute observed \(\hat\rho\) (or τ, γ, …).  
2. For \(b = 1..B\): shuffle QE relative to DA; recompute statistic.  
3. Two-sided p ≈ \((1 + \#\{|\rho^{*}| \ge |\hat\rho|\})/(B+1)\).

```python
from correlation_metrics import permutation_test

pt = permutation_test(da, qe, stat="spearman", n_perm=5000, alternative="two-sided")
print(pt["observed"], pt["p_value"], pt["null_mean"], pt["null_std"])
```

| Result | Meaning |
|--------|---------|
| p small (e.g. < 0.05) | Reject independence; association present |
| p large | Compatible with noise / no rank association |

Does **not** by itself give a CI for the size of ρ.

---

## 3. Confidence: permutation vs bootstrap

| Tool | Answers |
|------|---------|
| **Permutation p-value** | Is there association? (significance) |
| **Null reference band** | Where does the statistic fall under H0? |
| **Bootstrap percentile / BCa CI** | What is a plausible range for the **parameter** ρ? |
| **BC point** | Bias-adjusted estimate of ρ |

**Confidence permutation “tests”** sometimes refer to inverting permutation tests to form CIs. That requires a model for dependence under H1 (e.g. restricted permutations). For DA↔QE calibration, **BCa bootstrap CI** is the practical parameter interval; **permutation p** is the significance statement.

```python
from correlation_metrics import permutation_ci

r = permutation_ci(da, qe, stat="spearman", n_perm=5000)
# r["permutation_p_value"]      — significance
# r["null_reference_band"]      — H0 band (not parameter CI)
# r["bootstrap_ci_bca"]         — parameter CI (preferred)
# r["bootstrap_point_bc"]       — bias-corrected point
```

---

## 4. Recommended reporting line

```
Spearman ρ = 0.71 (BC 0.70), BCa 95% CI [0.61, 0.79],
permutation p < 0.001 (B=5000), n=180
Jackknife acceleration a = 0.012
```

---

## 5. Pitfalls

- Permuting **pairs** breaks association; do not reshuffle within-language blocks if you only want within-lang tests — stratify.  
- Heavy ties → discrete null; still valid with τ-b / Spearman midranks.  
- Tiny n → p-values granular (multiples of 1/(B+1)); collect more DA.  
- Do not treat null reference band as a CI for ρ.