MWJB Yes, extraction time was not the target, it was a variable that was kept constant. EY was the only dependent variable. My point is that the variability in EY cannot be due to variable extraction times (as Lance points out too). So if there is variability, there must be other variables at play affecting EY that you cannot control.

How do you control for this variability? By taking multiple measurements. N=8 is not enough to make any statistical judgments. Especially if you are doing multiple comparisons, the chances of finding a difference purely due to chance increase exponentially.

Lance claims that the Blind Shaker is superior overall as the mean difference in EY is 0.7%. This is based on the difference in mean values between different groups. I do not think this is a valid statement. For example, If you do a t-test comparing Blind Shaker vs Moonraker, you do not get a statistically significant difference. ( p is .10768)

Post hoc testing shows a statistically significant difference only for Blind Shaker vs Autocomb.

Also, what is the minimum difference in EY that can be detected by taste? Ultimately that is what matters.

The inference you could make is that all methods are equal. But this completely ignores the probability of type 2 error- not finding a difference because the sample size was too small.

The method that I think makes more sense, is to do a double blind study. Someone else plays with the parameters, another person brings it over the Lance and his job would be to just judge the flavours. Use a different coffee for each extraction and try out the different methods. Not entirely sure what the appropriate sample size would be- this will need power analysis

  • MWJB replied to this.

      Emc2 Also, what is the minimum difference in EY that can be detected by taste? Ultimately that is what matters.

      Not really, again, the devices are marketed on their effect on EY. Cups at the same EY (within reasonable expectation) will still differ in taste, +/-0.3%EY has been mooted as the detectable level, but I don’t personally think it is that clear cut. A coffee with 10 brews at the same parameters might have a stdev of 0.4EY and this be considered good/acceptable, but we’re brewing tiny amounts, in short times and, yes as you say there will always be some variability that we cannot control. Context and reasonable expectation is important in making sense of this.

      EY is an objective measure of brew efficiency and useful for gauging objective consistency. It has been found to have a correlation to regions of interest where people have a preference for taste too, but it is not a direct indicator of taste/quality (as stated for the last 70yrs), hence why I think it important to also have taste scores - EY without taste is too vague to be meaningful, taste scores without an objective value makes troubleshooting & sharing more difficult. (You can’t fix a coffee you will never like by knowing EY, you might be missing out on a better result with one you do if you are consistently off in EY).

      If you are going to carry out tests yourself for your channel, it would be informative if you did some EY measurements yourself (follow VST protocols whichever refractometer you buy).

      My apologies, I didn’t notice at first look that Lance only did 8 measurements per condition (he does state this in the video), I agree a larger sample size would have been more robust (He stated that he pulled 100 shots…I wrongly assumed that would be enough to get at least 10 per condition).

      Yes, double blind testing would be ideal, with sensory testing amongst a random sample who are unaware of what the test is…but now you are talking thousands (or tens of thousands) of pounds, teams of people & finding people in the street who drink espresso…good luck with that 😀

      Look around your coffee gear, which device/accessory that you have, or even that you know of, has been tested to the degree that you suggest? It’s the lay of the land that we are reliant on sole agents & garden shed protocols to ‘test’ this stuff. ☺️

      • Emc2 replied to this.

          MWJB I agree that most of the gear we use does not have any valid experimental data to support their use. Such robust testing can only be done by full-time academics. But what I would like to see is an acknowledgment of the limitations of the study and to refrain from making absolute statements such as the one about a 0.7% increase in EY.

          I review a fair few research articles- this is perhaps why my expectations are a bit high.

            MWJB my mistake. I’ve corrected it in the post above

              Meh. I took the study as more “ You can remove a lot of faff in your process without much detrimental effects if you do this.”

              It’s not necessarily about what is best. What is good enough.

                Personally taste tests would always be well personal to the tester other than broadly saying good/bad imho. He would never win going that route instead using repeatably recordable numbers.

                However my simplistic view on the vid was overall how little there was between the various methods - so if i find a way to simplify further my workflow - or if I get some benefit in prepping my coffee its a win 😂

                And I have ordered a cheap blind shaker from amazon just to try it out by using that rather direct into my portafilter.

                I still need to try making a really under then over extracted shot just to see how it tastes for my taste buds… dont think I nailed that yet 👍

                Loonster The point is that the data doesn’t show that! You can’t make any conclusions based on it. I guess the only way is to try it yourself. I’ve ordered a shaker as well.

                  Suddenly a rush on shakers 😂

                  Though not the weber - 65 quid and then 30 odd shipping - plus you would get hit with VAT etc

                    I got one for 15 from amazon but its coming from china.. i did sort of look at aliexpress and got alot of we dont ship to the UK

                    • Emc2 replied to this.

                      MattH it’s often a problem with the settings. If you haven’t selected a country in your profile, you will get this message.

                          Emc2 aha - thanks for that - I will try that out 👍

                            Tried shaking the coffee in the grind cup this morning for about 20 seconds or so. Surprisingly, the espresso took longer to extract and tasted good. Not sure about the EY, but this method does seem to take away the need for deep WDT. ( I did puck rake though).

                            it is getting more and more ridiculous. instead of having a simple, smooth, relatively foolproof workflow - people are trying to make pulling a shot a ritual instead. As if only them know and can enjoy a good cup

                              delta76 I I think the video was about making the workflow easier? Not having to do WDT will massively improve the workflow and make things easier for beginners.

                                  Emc2 Agreed I am hoping to reduce what (little) I already do 😂

                                    Emc2 no I was saying in general, not specifically to the video. I tried to watch it but could not finish.

                                    I am biased - because I like James a lot - but I wish we have more of this

                                    in the end, his workflow is simple and easy. and it creates just as good espresso as with $200 moonraker whatever things. We (as the espresso world in general) are spending way too much time and money to optimize the last 1% where it could be gained much cheaper and easier else where.