A couple of vids from Lance about WDT and Tampers.. personally I enjoyed these and took away a couple of easy tips, doing a second tamp - or using a lidded shaker for distribution (amazon have a few for little pennies..)
WDT and Tamping…
Me making coffee when I see the lengths some people go to:
La Marzocco Linea Mini - Mazzer Philos
Ikawa Roaster
I really want to see how a Niche cup with chopsticks rates.
OK. Having looked at the data, I am not entirely certain that I can be as confident as Lance Hedrick.
Not a statistician, but this is what I think.
He has used the same coffee and measured the data multiple times. Not entirely sure if repeated measures ANOVA testing can be used, but it is the same coffee being tested in different conditions. Repeated measures ANOVA shows significant difference but post hoc testing shows a difference between only the Blind Shaker and Autocomb.
If you consider each extraction to be independent (despite the coffee being the same), I suppose you could use ANOVA. The results are statistically significant which means at least 2 groups are statistically different. However, you do not see any statistically significant differences in post hoc testing.
I think this means that based on the data, we cannot say for certain that the blind shaker produces the most extraction.
The sample size is too small to draw any definite conclusions. Unfortunately, the question can only be answered by a large dataset.
I have put this comment on his video. Let’s see if he responds.
Gotta chase that 0.0000015% of ey
Not that a high ey even matters, high ey on some coffees tastes shite. Not to mention there are to many variables just to pin down to puck prep.
Decent De1pro v1.45 - Niche Duo - Niche Zero - Decent is the best machine ever made -
- Edited
Emc2 He has used the same coffee and measured the data multiple times. Not entirely sure if repeated measures ANOVA testing can be used, but it is the same coffee being tested in different conditions. Repeated measures ANOVA shows significant difference but post hoc testing shows a difference between only the Blind Shaker and Autocomb.
If you consider each extraction to be independent (despite the coffee being the same), I suppose you could use ANOVA.
If he used different coffees with the same set up & grind setting, then you would have a lot more scatter & noise, because different coffees hit different extractions at the same parameters and that is normal & even desirable. Using the same coffee is a more practical & feasible test of the specific effect of the tool.
Wouldn’t ANOVA testing favour the conditions that had means closest to the overall mean, and not necessarily pick up on a more consistent outlier?
Cuprajake Gotta chase that 0.0000015% of ey
It’s more like an increase of up to quarter quarter over the span of samples at a glance, but I agree with the notion that the increase in extraction doesn’t mean a lot unless liking scores are also included. Bear in mind these gizmos are usually marketed as specifically increasing extraction and/or making extractions more consistent…this doesn’t seem to be born out, even if not conclusively disproved.
Under-extracted espresso is the most common failure I experience in cafes…apart from the ones that fill a demitasse!
Samo Smrke is a coffee scientist working at Zurich in the coffee excellence team, of all the purported coffee science that gets articles in the press and sparks forum fads, I would have more faith in him and the rest of the team there (Wellenger & Yeretzian).
MWJB ANOVA is not robust against outliers as they will cause the estimate to deviate significantly and increase the chances of type-1 errors. Can’t see many outliers in this data.
It would be useful to know what test was used. For such a small data, where the number of observations is more than the sample size, I suspect you need specialised statistical tests.
Another issue is that even with similar extraction times within a group, the EY was variable. There are so many variables that affect EY, so this isn’t surprising. Which begs the question- how reliable is it to use EY as the dependent variable. In my view, a more practical and pragmatic approach for such small samples would be to use flavour to evaluate differences.
Emc2 You have to pick one aspect to be the focus, extraction time was not the target, EY was as this is the claim most often made by the gizmo makers. The standard deviation of 0.32%EY is certainly small enough to consider the blind shaker condition ‘the same extraction’. Sure taste preference scores would be great, but then you would still want EY so that you could present like for like (or as close as can reasonably be expected) samples.
- Edited
MWJB Yes, extraction time was not the target, it was a variable that was kept constant. EY was the only dependent variable. My point is that the variability in EY cannot be due to variable extraction times (as Lance points out too). So if there is variability, there must be other variables at play affecting EY that you cannot control.
How do you control for this variability? By taking multiple measurements. N=8 is not enough to make any statistical judgments. Especially if you are doing multiple comparisons, the chances of finding a difference purely due to chance increase exponentially.
Lance claims that the Blind Shaker is superior overall as the mean difference in EY is 0.7%. This is based on the difference in mean values between different groups. I do not think this is a valid statement. For example, If you do a t-test comparing Blind Shaker vs Moonraker, you do not get a statistically significant difference. ( p is .10768)
Post hoc testing shows a statistically significant difference only for Blind Shaker vs Autocomb.
Also, what is the minimum difference in EY that can be detected by taste? Ultimately that is what matters.
The inference you could make is that all methods are equal. But this completely ignores the probability of type 2 error- not finding a difference because the sample size was too small.
The method that I think makes more sense, is to do a double blind study. Someone else plays with the parameters, another person brings it over the Lance and his job would be to just judge the flavours. Use a different coffee for each extraction and try out the different methods. Not entirely sure what the appropriate sample size would be- this will need power analysis
Emc2 Also, what is the minimum difference in EY that can be detected by taste? Ultimately that is what matters.
Not really, again, the devices are marketed on their effect on EY. Cups at the same EY (within reasonable expectation) will still differ in taste, +/-0.3%EY has been mooted as the detectable level, but I don’t personally think it is that clear cut. A coffee with 10 brews at the same parameters might have a stdev of 0.4EY and this be considered good/acceptable, but we’re brewing tiny amounts, in short times and, yes as you say there will always be some variability that we cannot control. Context and reasonable expectation is important in making sense of this.
EY is an objective measure of brew efficiency and useful for gauging objective consistency. It has been found to have a correlation to regions of interest where people have a preference for taste too, but it is not a direct indicator of taste/quality (as stated for the last 70yrs), hence why I think it important to also have taste scores - EY without taste is too vague to be meaningful, taste scores without an objective value makes troubleshooting & sharing more difficult. (You can’t fix a coffee you will never like by knowing EY, you might be missing out on a better result with one you do if you are consistently off in EY).
If you are going to carry out tests yourself for your channel, it would be informative if you did some EY measurements yourself (follow VST protocols whichever refractometer you buy).
My apologies, I didn’t notice at first look that Lance only did 8 measurements per condition (he does state this in the video), I agree a larger sample size would have been more robust (He stated that he pulled 100 shots…I wrongly assumed that would be enough to get at least 10 per condition).
Yes, double blind testing would be ideal, with sensory testing amongst a random sample who are unaware of what the test is…but now you are talking thousands (or tens of thousands) of pounds, teams of people & finding people in the street who drink espresso…good luck with that 😀
Look around your coffee gear, which device/accessory that you have, or even that you know of, has been tested to the degree that you suggest? It’s the lay of the land that we are reliant on sole agents & garden shed protocols to ‘test’ this stuff. ☺️
- Edited
MWJB I agree that most of the gear we use does not have any valid experimental data to support their use. Such robust testing can only be done by full-time academics. But what I would like to see is an acknowledgment of the limitations of the study and to refrain from making absolute statements such as the one about a 0.7% increase in EY.
I review a fair few research articles- this is perhaps why my expectations are a bit high.
- Edited
Deleted
Meh. I took the study as more “ You can remove a lot of faff in your process without much detrimental effects if you do this.”
It’s not necessarily about what is best. What is good enough.
- Edited
Personally taste tests would always be well personal to the tester other than broadly saying good/bad imho. He would never win going that route instead using repeatably recordable numbers.
However my simplistic view on the vid was overall how little there was between the various methods - so if i find a way to simplify further my workflow - or if I get some benefit in prepping my coffee its a win 😂
And I have ordered a cheap blind shaker from amazon just to try it out by using that rather direct into my portafilter.
I still need to try making a really under then over extracted shot just to see how it tastes for my taste buds… dont think I nailed that yet 👍
Suddenly a rush on shakers 😂
Though not the weber - 65 quid and then 30 odd shipping - plus you would get hit with VAT etc
- Edited
MattH Ive ordered one from Aliexpress for £10
The price seems to have gone up a bit!
I got one for 15 from amazon but its coming from china.. i did sort of look at aliexpress and got alot of we dont ship to the UK
- Edited
Tried shaking the coffee in the grind cup this morning for about 20 seconds or so. Surprisingly, the espresso took longer to extract and tasted good. Not sure about the EY, but this method does seem to take away the need for deep WDT. ( I did puck rake though).
it is getting more and more ridiculous. instead of having a simple, smooth, relatively foolproof workflow - people are trying to make pulling a shot a ritual instead. As if only them know and can enjoy a good cup
delta76 - Well… You don’t need to do it. :-)
MediumRoastSteam I don’t. I just think it’s ridiculous, it is creating a barrier for people want to get into espresso IMO
Emc2 no I was saying in general, not specifically to the video. I tried to watch it but could not finish.
I am biased - because I like James a lot - but I wish we have more of this
in the end, his workflow is simple and easy. and it creates just as good espresso as with $200 moonraker whatever things. We (as the espresso world in general) are spending way too much time and money to optimize the last 1% where it could be gained much cheaper and easier else where.
delta76 - It’s all relative. You could argue that for some, when they see us with a machine on our couter top that weights 30kg and takes 35 minutes or so to warm up, weighing in and out, WDT (or stockflesh, or nswe or whatever we had back in the day) they must think we are just crazy insane.
You have to be happy with what works for you. If grind, tamp, pull works… Happy days. If you feel the need that things taste better adding further steps, happy days too.
- Edited
MediumRoastSteam if they make a quantifiable difference in cup quality (i.e. in blind test you have 8 out of 10 tasters agree that cup with this technique is better than cup with that technique), sure. but for now it seems like all added extra steps without any real benefits, just people think it does
delta76 - I do not disagree. Don’t forget that there’s a lot of people in this “hobby” with the main focus being the hobby itself - parameters, ratios, graphs, data in general - rather than a consistent and tasty cup of coffee every day - and that’s up to them. If that’s what floats their boat, then so be it. My goal is to:
- Get a nice cup of coffee twice a day, consistently.
- Without making a mess (both from grinders (static) and from the machine (channeling, spray);
- easy to clean my portafilter and basket after every shot;
- Easy to keep the machine clean.
So my routine is:
- dump pre-weighted dose into Niche hopper;
- Grind into Niche cup;
- Flip into basket;
- Use spyrographic tool (10 turns) to distribute;
- Tamp with Bravo tamper;
- Puck screen on top;
- Pull until desired weight.
For me, this works, and it seems consistent. For others, it might be too much, and no doubt, for some, too little.
Happy coffee! :-)
delta76 You would test amongst more than 10 tasters, these tasters would also need to be unaware of what the test was. Tests on this scale are rare in specialty coffee.
However, if you’re making coffee for yourself, friends and family you’re going to primarily be concerned with what you like. You don’t need to convince anyone else and you can do this easily. You can share what you do in good faith and others can try it, adopt it, or bin it. You won’t accidentally create a black hole that swallows the Earth 😄
delta76 We (as the espresso world in general) are spending way too much time and money to optimize the last 1% where it could be gained much cheaper and easier else where.
Nobody is doing this.
would shaking in the cup no move the fines to the bottom then inverting put the fines on top to clog?
i only wdt to distribute the grinds to be flatter as i cant palm tap for shit,
suppose its a kin the the poeple who wash cars then those whose who detail using a 100 step method. it appeals to different people,
Decent De1pro v1.45 - Niche Duo - Niche Zero - Decent is the best machine ever made -
good to know, ill be honest its not something ive taken too much notice in, can appreciate it but to me its over complicating things, next ill push back on to grinders and we will be finding a new king grinder, they miss the biggest key, everyone’s tastes are different.
id love them to re do this test now with all the info available
Decent De1pro v1.45 - Niche Duo - Niche Zero - Decent is the best machine ever made -
- Edited
I grind into the basket, which sits (with a dosing collar) on a block of wood on my Niche Zero. I WDT, don’t tap, I then tamp and pour. It works. I used to (when I used the wrong needles) have channeling issues but now don’t.
I serve myself and occasionally friends or family. I’m not a barista in a cafe. I don’t care whether I save or don’t save a minute in work flow. I do care about how my coffee tastes.
WDT works for me. IF a blind shaker significantly reduced the extent or risk of channeling than WDT and was significantly less faff than WDT, I might try it. But all I’ve seen is one sensational video by one pundit looking for continued fame and fortune. When and if more studies are done that prove that a blind shaker or other distribution method is more effective, I will try it.
I personally have found that WDT is more effective (for me) than hand distribution or tapping. I never got into using the Niche cup, though it works for many. I learned that wedge type distribution tools move too much coffee around (less grounds movement is in my experience generally better) and I do like a solid tamp.
But that’s what works for me. There are many ways to effectively distribute coffee into the PF basket, once you get good at it. What I do object to is folks getting too hot under the collar about their way. Or using pejorative words to criticize what I do (WDT or warming my basket/top screen) if I spend an extra minute or two making my coffee the way I like to make it.
And if I could get my buddy to actually put a measured dose of coffee into his PF and do better than a quick smoosh with one finger and pay attention to his brew ratio, in order to make a decent espresso with his Breville, that would be an achievement. But he just dumps coffee into his PF, tamps with the plastic tamper that came with the machine and pours until the shot cup looks full enough. Using stale coffee from Costco. But he’s happy with his coffee and otherwise enjoys life, so after my couple of polite suggestions were ignored, I just accept the coffee cup he gives me with thanks and compliments. Each to their own.