5
$\begingroup$

Original question: ~~I don't know how to intrepet the result I got from my OLS analysis. The issue is that I can fit a horizontal line within the 95% CI even though the overall p-value for the model was less than 0.05. I find this somewhat confusing since fitting a horizontal line within the 95% CI indicate that it satisfies the null hypothesis.~~

Revised question: Based on the linear regression output below, the overall p-value of the model is close to 0.05 (p-value < 0.05246). With such a low p-value, I expected the 95% CI band to be much smaller/ narrower. Am I right to have such assumption?

Below is the R script and data that I used for this analysis. The black line in the figure shows the fitted line; red-dashed shows the 95% CI of the linear model; blue-dashed line shows a reference horizontal line. Please correct me if I am doing something wrong.

# Data
x <- c(693.72, 723.44, 678.94, 385.28, 395.72, 665.89, 659.67, 680.28, 476.28, 657.56)
y <- c(14.220000, 12.073333, 22.106667,  9.746667, 10.553333, 20.256667, 17.270000, 13.246667, 8.346667, 21.760000)
df <- cbind(x,y) |>
  as.data.frame()

# Linear model
lmm <- lm(formula = y~x, data = df)
summary(lmm)

# 95% confidence intervals of the linear model
ci <- predict(lmm, newdata = df, interval = "confidence") # default is 95% CI
ci_df <- cbind(ci, x = df$x) |> 
  as.data.frame()
ci_df <- ci_df[order(ci_df$x), ]

# Plot x vs y and its 95% CI
plot(df$x, df$y)
abline(lmm) # linear model
lines(ci_df$x, ci_df$lwr, lty = 2, col = "red") # lower CI
lines(ci_df$x, ci_df$upr, lty = 2, col = "red") # upper CI
abline(a = 14, b = 0, lty = 2, col = "blue")    # reference line

Edit: As suggested by Geln_b and whuber, I expanded the 95% CI lines beyond the observed data points.The blue horizontal line is y = 14.75.

OLS output from R:

Improve this question
New contributor
Melon is a new contributor to this site. Take care in asking for clarification, commenting, and answering. Check out our Code of Conduct.
$\endgroup$
7
  • 1
    $\begingroup$ Never mind, the p-value was 0.052, but can someone tell me if the 95% CI is too wide despite being so close to pvalue <= 0.05? $\endgroup$ Commented Nov 8 at 4:59
  • 4
    $\begingroup$ It looks to me like you did a pointwise interval rather than a simultaneous one (so I'm not 100% sure what you originally expected to happen will always happen. ... maybe it does?), but in any case it's not clear to me that the interval doesn't cross the line you drew, since you only evaluated it at the data points. You should evaluate it at more points if you want to be sure (and to see how narrow the interval is in tbe middle), and gake those points out far enough that all four of the ends of the curves start to 'turn away' from the horizontal $\endgroup$ Commented Nov 8 at 5:07
  • 1
    $\begingroup$ @Melon Nothing I suggested requires additional data. You are not required to only find the CI at the observed values $\endgroup$ Commented Nov 8 at 11:01
  • 2
    $\begingroup$ The plot does not show the full extent of the confidence bands and it uses a limited approximation to them. I suspect that if you were to extend the plot further to the right and display the lower band more precisely, it might very well turn around without crossing above the x-axis. $\endgroup$ Commented Nov 8 at 14:32
  • 2
    $\begingroup$ Here is a relevant question: stats.stackexchange.com/questions/635217/… $\endgroup$ Commented Nov 8 at 14:49

1 Answer 1

Reset to default
11
$\begingroup$

I think your basic understanding is correct.

Linear regression reports a p-value testing the null hypothesis that the slope of the population or distribution is zero. That null hypothesis is represented by a line with a slope of 0.0 (horizontal) and a Y intercept equal to the mean of all Y values. Linear regression can also plot 95% confidence bands. If the p-value is greater than 0.05, you don't reject the null hypothesis, so the null hypothesis line must be entirely within the 95% bands. If p<0.05, then the null hypothesis line must not be entirely within the 95% confidence bands. [The same rule applies to p<0.01 and 99% confidence bands etc.]

In your case, the p-value is just slightly larger than 0.05, so you expect the horizontal line to be entirely within the confidence bands, but because it is so closoe to 0.05, you expect the null hypothesis line to be close to the confidence bands.

You can see this in the graphs below, which shows your data points, the fitted regression line, the 95% confidence bands (in blue), and the null hypothesis line n red. The two graphs are identical except the second one extends the range of the x axis (as @whuber suggested in a comment).

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.