Celebration Thread! Biogen is going to the FDA with Aducan.

Alzheimer's, cardiovascular, and other chronic diseases; biomarkers, lifestyle, supplements, drugs, and health care.
J11
Contributor
Contributor
Posts: 3351
Joined: Sat May 17, 2014 4:04 pm

Re: Celebration Thread! Biogen is going to the FDA with Aducan.

Post by J11 »

Blue Random 1.GIF
The fit in the above post is not really that great.
The red curve fit never actually reaches up to the top of the black curve.
What I think that we could do is remove the left hand points that are hugging the x-axis.
If we censor the data until about 2.5 then we have what looks like a real gamma curve.
That can be for later.

What I have also done is created 100 random samples from a gamma distribution with shape 11.265 and rate 1.70.
This is the gamma distribution that we fit to the 302 placebo.

What is quite exciting here is that the mean of this sample is 6.767.
Subtract 5 and we have 1.767. The 302 placebo had a mean of 1.74.
This is what we are trying to model.


x <- rgamma(100, shape = 11.265, rate = 1.70)
x
den <- density(x)
den
mean(x)

dat <- data.frame(x = den$x, y = den$y)

# Plot density as points

ggplot(data = dat, aes(x = x, y = y)) +
geom_point(size = 3) +
theme_classic()
You do not have the required permissions to view the files attached to this post.
Last edited by J11 on Sat Nov 28, 2020 1:23 am, edited 2 times in total.
J11
Contributor
Contributor
Posts: 3351
Joined: Sat May 17, 2014 4:04 pm

Re: Celebration Thread! Biogen is going to the FDA with Aducan.

Post by J11 »

OK, here is the R update.

for (i in 1:100){
x <- rgamma(299, shape = 11.265, rate = 1.70)
print(mean(x)-5)}

[1] 1.501982
[1] 1.668911
[1] 1.43815
[1] 1.672636
[1] 1.789249
[1] 1.513188
[1] 1.504964
[1] 1.548712
[1] 1.49335
[1] 1.585495
[1] 1.622047
[1] 1.588354
[1] 1.549784
[1] 1.514559
[1] 1.377794
[1] 1.671135
[1] 1.936958
[1] 1.759398
[1] 1.75866
[1] 1.616827
[1] 1.446674
[1] 1.862424
[1] 1.544658
[1] 1.623395
[1] 1.599571
[1] 1.765547
[1] 1.625131
[1] 1.434725
[1] 1.541069
[1] 1.657216
[1] 1.648692
[1] 1.312358
[1] 1.543404
[1] 1.800886
[1] 1.567482
[1] 1.606802
[1] 1.447847
[1] 1.554459
[1] 1.504879
[1] 1.707855
[1] 1.538123
[1] 1.593575
[1] 1.559158
[1] 1.601904
[1] 1.471353
[1] 1.465511
[1] 1.798857
[1] 1.710416
[1] 1.550932
[1] 1.670279
[1] 1.510871
[1] 1.663748
[1] 1.469294
[1] 1.455663
[1] 1.408895
[1] 1.561606
[1] 1.389093
[1] 1.563184
[1] 1.881156
[1] 1.560776
[1] 1.705032
[1] 1.689452
[1] 1.792148
[1] 1.676438
[1] 1.815733
[1] 1.598904
[1] 1.691945
[1] 1.60774
[1] 1.687211
[1] 1.685919
[1] 1.752654
[1] 1.849882
[1] 1.602297
[1] 1.809721
[1] 1.764932
[1] 1.628757
[1] 1.45924
[1] 1.741954
[1] 1.72022
[1] 1.643208
[1] 1.772078
[1] 1.811953
[1] 1.766738
[1] 1.78612
[1] 1.545758
[1] 1.678209
[1] 1.654998
[1] 1.397922
[1] 1.747751
[1] 1.58392
[1] 1.515337
[1] 1.553879
[1] 1.720032
[1] 1.438558
[1] 1.857357
[1] 1.553428
[1] 1.776146
[1] 1.543341
[1] 1.615389
[1] 1.558987
J11
Contributor
Contributor
Posts: 3351
Joined: Sat May 17, 2014 4:04 pm

Re: Celebration Thread! Biogen is going to the FDA with Aducan.

Post by J11 »

Right about now I am sure some might be wondering. J11 what's this all about?
What I think is going on here is that we are simulating the type 1 error.
alpha is when you take a sample from the null distribution and you get something that you find unexpected. For example, a 1% extreme from the distribution in a treatment arm.

Here the distribution (the Null) is the placebo.
What we want to know is how often could you take a sample of 299 (the number from the high dose in 302) and have a
sample mean <= 1.74 - 0.39 =1.35? This is what alpha is! We have the placebo distribution so we can just run the simulation.
We do not even need the treatment distribution.

What you might notice is that in this random sampling from the Null distribution 1 of the random samples actually did hit below
1.35 (i.e,. 1.312358). That's the alpha that was calculated for the high dose! This is very exciting!

(perhaps a parallel logic could apply to the treatment distribution. We could sample from this distribution and see how often the mean of the placebo distribution occurred. This would be our beta. How often would the placebo result occur if the treatment distribution applied?)
Last edited by J11 on Sat Nov 28, 2020 1:28 am, edited 2 times in total.
J11
Contributor
Contributor
Posts: 3351
Joined: Sat May 17, 2014 4:04 pm

Re: Celebration Thread! Biogen is going to the FDA with Aducan.

Post by J11 »

New R code. I am moving towards doing a large scale simulation.
What is a good approximation of alpha with a large number of runs?

trials<-10
for (i in 1:trials){
x <- rgamma(299, shape = 11.265, rate = 1.70)
if ( mean(x)-5 <=1.35) {count<- count+ 1}
print(mean(x)-5)
print(count)
}


OK, I ran the above code with various numbers of trials.
What would happen if there were 100 aducan phase 3 Emerge trials with a -0.39 treatment effect with 299 high dose patients.
I sample from the placebo distribution and see how often I have a mean CDRsb of 1.35 or less in a sample.
With a run of 1 million there were about .7% of trials that fulfilled this property.
Alpha is roughly .7%.

I could make the simulated gamma distribution ( red curve in earlier post) closer to the placebo distribution
to be more accurate.

runs
10 0,0,0,0,0
20 0,0,1,0,0
100 0,1,0,0,1
1000 10,8,7,5,12
10000 82,80,71,76,63
100000 724,705,762,728,745
1000000 7080
J11
Contributor
Contributor
Posts: 3351
Joined: Sat May 17, 2014 4:04 pm

Re: Celebration Thread! Biogen is going to the FDA with Aducan.

Post by J11 »

I included the post PV4 placebo to the pre-PV4 placebo.
There are now 111 points. I shifted it over by 5 again because some of the patients on placebo did have negative scores and negative scores are not allowed with the gamma distribution.

The file name is now BlGr
R code

x<-BlGr$V2 +5
den <- density(x)
den


dat <- data.frame(x = den$x, y = den$y)

# Plot density as points

ggplot(data = dat, aes(x = x, y = y)) +
geom_point(size = 3) +
theme_classic()
BlGr Distribution.GIF
#Here are the fit parameters

fit.params <- fitdistr(x, "gamma", lower = c(0, 0))
fit.params

fit.params
shape rate
11.2031294 1.6453690
( 1.4819457) ( 0.2225936)

(These were the shape and rate parameters when we only used the Blue placebo patients to fit the gamma)
shape rate
11.265466 1.700323
( 1.962672) ( 0.302923)
BlGr Distribution 2.GIF
You do not have the required permissions to view the files attached to this post.
J11
Contributor
Contributor
Posts: 3351
Joined: Sat May 17, 2014 4:04 pm

Re: Celebration Thread! Biogen is going to the FDA with Aducan.

Post by J11 »

The plan now is to take it off road.
Instead of reading the Briefing Document and trying to understand their analysis, it might be best to take the dataset and do our own analysis. Self-evidently this could be considered IQ inappropriate behavior-- might hit a few potholes-- will hit many potholes. It'll be an adventure and fun.

The first section of the Document did not allow for datascrapping as there was no data available. The later parts of the document do provide this information. It is startling how powerful a learning tool the internet is. With statistics and programming not having exact understanding can lead to a great deal of frustration. Yet, internet browsers allow rapid access to answers.
Double Distribution1.GIF

This is a centrally important finding from the dataset from Figure 7.
This shows the CDRsb distributions of treatment (left red) versus Placebo (right blue).
The dots include all the patients from Figure 3 for Pre-PV4 and Post-PV4 in Appendix 3.
There is not that much of a leftward shift in the treatment group, though there does clearly appear to be some.

One of the problems with this figure is that the high dose treatment group includes a fair number of patients who did not receive many (sometimes any) high doses. It is very unclear how these patients should be included in the figure. Simply saying that they did not receive high dose and should be removed is perhaps simplistic. What if they had received high dose? Would these patients really have done much much better? Yet, at the same time there are not a great many rapid responders who received large numbers of high doses. There are other figures from Appendix 2 that will us also to have a larger dataset to work with. The basic idea and code can be developed on Figure 7 and then applied to these expanded datasets.
You do not have the required permissions to view the files attached to this post.
J11
Contributor
Contributor
Posts: 3351
Joined: Sat May 17, 2014 4:04 pm

Re: Celebration Thread! Biogen is going to the FDA with Aducan.

Post by J11 »

Start off with some descriptive statistics.

This is quite interesting.
In this subset of the data the difference in the means was 0.329; this is close to the top line result of 0.39
Yet, the difference of the medians is 0.510. This is the typical benefit that a patient at the median might be expected to achieve on aducan treatment.



Placebo
mean 1.808886
sd 2.135428
median 1.443128
mode 1.44

Treatment
mean 1.479872
sd 2.442099
median 0.9330472
mode 0.6

The blue and red vertical lines are the modes for the treatment and placebo respectively.
The green line is the median for the treatment and the red line is also the placebo's median.
As we can see above the actual mean treatment effect for this not complete 302 sample is ~0.329,
the median treatment effect is 0.51 and the mode treatment effect is ~0.84.

This is the off road part of the adventure and also the first pot hole: Why? Why is there such a large
difference between the mode and mean treatment effects? The difference is big perhaps more accurate
massive 0.329 --> 0.84. The focus in the Briefing Document is so much on the top line mean difference
0.39 as reported. Yet, here we are looking at a much more substantial difference of 0.84. No word on
this in the report.

Answer 1 : It is possible that this is a statistical artifact that is more mirage than real. This can be investigated with
the larger dataset.

Answer 2: Possibly there is an error. The blue distribution is quite non-symmetric and yet its median is almost exactly on its
mode. This seems strange. The red distribution appears almost perfectly symmetric and yet its median is displaced by
0.33 to the right of its mode. Hmm, that is probably it. It is for the very reason that the red distribution is so symmetric that
the long tail to the right has this influence on the median. With the blue distribution, the non-symmetry on the left is counter-balanced by the long tail on the right.

Answer 3: This is a real aspect of the data, though it is related to the nature of the distortions in the data. It is possibly giving us a clue that there is something amiss with the data. Remember we are using the data as presented in Figure 7 without data cleaning.
The blue distribution is what it is. Realistically, there is not much we data cleaning that we should do to the blue distribution. The red distribution also is what it is, though there is an opportunity to see how different dosing subgroupings change the results. The high dose patients who received no high doses and were rapid progressors are all in the high dose group. There are also some patients in the high dose who received no high doses and were slow progressors or slight regressors. The dataset is confusing.


While the blue distribution is what it is, it really should not be what it is. As seen in the figure a substantial fraction of the placebo were non progressors. The left hand blue vertical line is set at 0 CDRsb change. I can illustrate this with the cumulative distribution later.This makes it so difficult for an Alzheimer treatment to actually show benefit. To show benefit there needs to be progression in the placebo because it is not easy to actually heal the brain with an anti-amyloid treatment especially in the short term.
Double Distribution Median-Placebo.GIF
You do not have the required permissions to view the files attached to this post.
J11
Contributor
Contributor
Posts: 3351
Joined: Sat May 17, 2014 4:04 pm

Re: Celebration Thread! Biogen is going to the FDA with Aducan.

Post by J11 »

This is the treatment cumulative density function.
Treatment Cumulative.GIF
This is the placebo cumulative density function.

I am not sure yet how to put these two figures into one. One figure would let us easily see how and where divergence occurred.
Due to the steppy nature of the density function, each set of dots in each vertical line are 0.5 CDRsb higher than the set of dots to the left. I am also not clear on how to call up the points of the cumulative density function.

These figures are of considerable relevance when thinking about the aducan treatment effect. The caveat still applies that this is only a provisional look at the numbers after we crunch through more of the dataset things might look differently.

These figure are important because they can give us a feel for how patients would respond at different percentiles.
For example, roughly 30% of aducan treated patients on high dose in 302 had a stable CDRsb at 78 weeks. That is fairly impressive.
With placebo, the number is ~22%. Roughly 50% more treated than placebo had no decline.

Roughly 45% of aducan treated patients on high dose had a 0.5 point worsening on CDRsb against ~32% on placebo. The topline
primary analysis did not give this feel of the numbers. The entire conversation was fixated on the 1.74 mean placebo versus the 1.35 mean treatment (difference 0.39). This is what happens when you go off road-- you have a different view.

I am surprised by these numbers. It almost seems too good to be true. Perhaps the numbers are wrong. ~45% of the aducan treated patients had a 0.5 point worsening or less on CDRsb? Whoa time! That is not too bad. The right skew of the data with patients having rapid progression ( of up to ~10 CDRsb points) or even the more moderate-rapid declines from 3-10 highly distort the data.

Future anti-amyloid trials that have a more selective patient recruitment process will probably show better results. The focus on the mean response seems to introduce a great amount of distortion into the analysis. The simple answer of having virtual pre-trial recruitment makes an overwhelming amount of sense. It is possible though that this early patient population simply do not consistently decline and there is also the problem that even if they did decline current testing instruments are not always able to detect it.


Placebo Cumulative.GIF
You do not have the required permissions to view the files attached to this post.
Last edited by J11 on Sat Nov 28, 2020 4:57 pm, edited 1 time in total.
J11
Contributor
Contributor
Posts: 3351
Joined: Sat May 17, 2014 4:04 pm

Re: Celebration Thread! Biogen is going to the FDA with Aducan.

Post by J11 »

This is the summary for the placebo.
Empirical CDF: 110 unique values with summary
Min. 1st Qu. Median Mean 3rd Qu. Max.
-3.0805 0.4361 1.4404 1.7939 2.4498 9.9383

This is the summary for treatment.
Empirical CDF: 113 unique values with summary
Min. 1st Qu. Median Mean 3rd Qu. Max.
-3.06796 -0.06378 0.93305 1.47987 2.03117 10.96656

Note: I used the data point extractor which did not give exact CDRsb scores, so all of these points should be rounded to the nearest 0.5.
This introduced a small amount of data noise though, not much. I can post the dataset for transparency

What we see above is that 25% of high dose treated patients had no decline. For the placebo group the most favored 25% had a decline of 0.5 CDRsb.
J11
Contributor
Contributor
Posts: 3351
Joined: Sat May 17, 2014 4:04 pm

Re: Celebration Thread! Biogen is going to the FDA with Aducan.

Post by J11 »

This is a good one. Cumulative density -- x (blue) is placebo, xx(pink) is treatment.
Bluish dots show units of the CDRsb score.
A data clean that removed 10% of the questionable moderate to rapid progressors on high dose would only increase the 0 CDRsb
cumulative value from ~30% to ~33%.


Dual ECDF.GIF
You do not have the required permissions to view the files attached to this post.
Post Reply