Seriously?

Walt Disney World
1

Back in November, my wife and I decide to take some first-timers to WDW. I consult touringplans to find a good time to go and find that March 8-15 looks good. It’s close to Spring Break. But, hey the “data scientists” must know what they are talking about. Maybe everyone has a late break this year. With a handful of exceptions, all of the days are 5’s and 6’s. Looks promising. $10,000 later, we’re booked.

And then, yesterday (literally yesterday) happened. those 5’s and 6’s turned into 9’s and 10’s. Wow, I sure am glad I spent money to subscribe to this thing.

So, is this radical change the norm? If so, why does TP.com bother to publish crowd levels so far in advance. I could write a random number generator with that level of reliability. What’s even more absurd, is that the predictions included hour by hour heat maps of what the crowd levels would be.

So, seriously. Is the radical change 60 days out normal?

2

This is what Len wrote on another thread asking similar questions (find this topic thread to read others’ thoughts January 2019 WDW Crowd and Hours Update - are you sure?!):

"We’ll have a blog post out on this shortly.

We think the estimates from the last crowd cal update were incorrectly low, primarily due to a default setting change in the software we use to produce the models. (The setting was changed in a new release of the software, and it changed how the models were produced.)

Fred alluded to this in our mid-December blog post.

We knew November’s estimates came in lower than actuals. We spent most of December trying to figure out why, and how to fix it.

We expect to update the calendar on a regular basis going forward. The other things that we’re trying to model now are:

  • How many people will delay WDW trips until Galaxy’s Edge opens
  • The effect of the discounts Disney’s releasing to combat that
  • The potential for the US economy to slow a bit"
3

A default setting in the software you use to produce the models? Aren’t those models the bread and butter of what TP does? Astonishing.

I can’t even imagine what would happen if I went to my boss and said, “Do you know those estimates that I gave you last month? Whoopsy! I was off by 100% But, don’t worry. The problem was a default setting in the software I use to do what you pay me to do. So, it’s not really a problem” Grrrr. Bad answer.

What you are saying is that the models don’t work. Right?

4

Here’s another thread if you want others’ opinions and nuance: Everyone take a breath about Crowd Levels. It’s not the end of the world!

1 Like
5

There was an element of a software bug. There was also the element of Disney getting higher crowds in the fall than they had ever gotten before at that time of year. Both of these factors among others contributed to the change.

The most valuable parts of the site are the knowledge and the touring plans. Make a good custom touring plan and a 10 will feel like a 5.

1 Like
6

Your abrasive approach here is not fitting for this community. It is perfectly fine to ask questions - many members have this week - but it is not acceptable here to be so disrespectful. You might want to reconsider your tone.

Best.

36 Likes
7

You make me smile!!!

4 Likes
8

Lauren, my angst is not so much with the fact that I am going to have to wait an additional 30 minutes for Small World. The problem is that TP is selling a faulty product - plain and simple. There is a certain audacity in providing hour by hour predictions by park five months (or more) in advance when the accuracy of those predictions is +/- 100%. In my business (writing software), we call that a WAG. And, a WAG is not science - data or otherwise.

In the end, those days could be anything between 4 and 12. Since we don’t have reliable estimates, we have no way of knowing. So, we’ll roll with it. (Needless to say, I won’t be renewing my subscription.)

Mike, If the factors that go into the models are changing and there is no way to validate them, then don’t publish the estimates since they don’t provide value. And, I appreciate the encouragement. But, if you honestly believe that a schedule will turn a 10 into a 5 then I’m guessing that you have never experienced a 10. I’ve been in WDW during Christmas when it was hard to breath in those parks. Making a plan for those days is futile at best.

9

Thanks for the advice! I’ll file it accordingly.

10

I’m sorry the CLs went up for your trip. Do you have TP for your park days? Have you re-evaluated them since the update? It seems to me watching this unfold through conversations on here that folks have posted with what seemed to me to be initial shock, panic, frustration over the last few days only then to post updates saying their actual wait times did not change that much when they updated their TPs. Until you, I have not seen anyone report any wait change as high as 30 mins for one ride but I am not sure if that is an actual change in your predicted wait time or what you fear will happen on a 10 day. I have seen several report total wait time increases over the course of full days less than that. Then there have been conversations that followed explaining why and how that could be. I for one have learned a lot more about the process of coming up with the CLs this week. I have always felt that my TP subscription was about the best deal out there and still do. I have gotten far more out of this community and their tools than I could have ever asked for and certainly more than my subscription fee.

4 Likes
11

MrsWeb, Thanks for the kind note. We are in the process of making tweaks to our plans. But, I’m not using the CL’s from TP since I simply don’t trust them anymore. That said, I’m certain that it will be a fun trip regardless of whether it is a 2 or a 20.

BTW: I meant the 30 minutes to just be a figure of speech. I don’t know that the adjusted ride times are supposed to go up that much.

12

I am sure you will have fun and I hope you find the waits (predicted and actual) aren’t as bad as the jump in crowd level made it seem. I will add as I looked back over your comments and saw the mention of Christmas crowds that the TP staff have said several times since the update some version of what I would boil down to “there are 10s and then there are Christmas level 10s” and my read on what they have been saying is that the 10s they are now predicting in the coming months are not on the Christmas end of that range unless they also include a major holiday like Easter.

1 Like
13

I’ve been a customer of Touring Plans for several trips over the past 9 years. I’m also a Software Engineer who has some experience in data science and have, for fun, studied a lot of the data TP provides (they publish an awful lot of their data, including historical data and how they did). And their success rate is very high. It’s been spot on for every one of my trips, and you can look back and see actual crowd levels vs predicted for every day since 2011.

Lauren, my angst is not so much with the fact that I am going to have to wait an additional 30 minutes for Small World. The problem is that TP is selling a faulty product - plain and simple. There is a certain audacity in providing hour by hour predictions by park five months (or more) in advance when the accuracy of those predictions is +/- 100%. In my business (writing software), we call that a WAG. And, a WAG is not science - data or otherwise.

The +/- 100% isn’t really accurate. Going from a 5 to a 10 doesn’t mean that it’s double the crowds. It is closer to (although not exactly) percentiles, where a 10 is in the 90th percentile of days, a 5 would be the 40th percentile. Again, it’s not quite that but it is closer to that than it is to double the crowds. So saying it’s a +/- 100% isn’t accurate.

It’s also not right to call it a WAG. In data science, a change in inputs can have a high effect on the model. It’s important to remember that what TP does is a forecast. It’s based on past history, numerous other factors (e.g. state of the economy, school schedule across the US, etc.). But when one of those inputs unexpectedly changes, it throws off the model and they have to recalibrate.

I’ll give you an example. Last winter, Disney tried an experiment and cut staffing. They effectively ran many rides at 50% capacity. This was something they had never done before. Wait times skyrocketed. There is no way anyone or any model could have predicted this. (By the way, the model is factoring in that as a risk for this winter too.).

It would be a WAG if the model was consistently wrong and consistently adjusting in this way. But my experience and history shows that this isn’t the case ordinarily. This just happens to be a bad perfect storm. I’m sorry it affected you (FWIW, it affects me too as I’m going President’s Day week - but I know enough to not let it bother me).

In the end, those days could be anything between 4 and 12. Since we don’t have reliable estimates, we have no way of knowing. So, we’ll roll with it. (Needless to say, I won’t be renewing my subscription.)

Mike, If the factors that go into the models are changing and there is no way to validate them, then don’t publish the estimates since they don’t provide value. And, I appreciate the encouragement. But, if you honestly believe that a schedule will turn a 10 into a 5 then I’m guessing that you have never experienced a 10. I’ve been in WDW during Christmas when it was hard to breath in those parks. Making a plan for those days is futile at best.

This isn’t close to a Christmas 10. You can view the actual peak wait time predictions for your days and compare them to Christmas week. They aren’t anywhere near the same. I’ve been on non-Christmas 10 days. With the Touring Plan software, I never waited more than 30 min for anything.

13 Likes
14

Thank you. Thing happen and they have corrected it. I know one thing that people sometimes forget is that a CL of 8, 9, 10 in Feb/March is not the same as one during Christmas and most importantly, a good touring plan for the day is much more important than the CL ever will be.

2 Likes
15

We have the same week, just one day off 3/9 - 3/16. Crowd levels change and I still trust the Touringplans.com The crowd levels did go up and it doesn’t matter to me as we will still enjoy ourselves. The touring plans that we created and the fastpasses that we got and added work fine for us.

Here is something to consider - What would you have done if there was no touring plans out here? Yes this is not perfect but you must remember that they are still predictions. Also if you look on the website about what they look and the data that is used and how it is used, you will find out that it ins’t anything simple. Disney has changes so many things and there were some issues but they have fixed it.

When you go, just remember to enjoy yourself and have fun, relax and take in all the sights, sounds and smells. IT will be worth it if you go there with an open mind and a good touring plan for each day but leave room for some changes.

3 Likes
16

I will not blame anyone for being super irritated if they paid for a product and that product is essentially buggy, for whatever reason. Venting frustrations in that regard is not out of line, and the fact is, the developers of software themselves are often just as irritated when they discover they have such bugs. (I know from first hand experience, as a Principal Software Engineer!)

It is okay to want to defend the successes of TouringPlans as well, as it helps paint a well-rounded picture. But, for example, I don’t really care how well a piece of software works for everyone else if it isn’t working for me! Particularly when it is product I pay for. (By comparison, I can’t really complain about the CLs reported by UT since they are providing the information free of charge, not as a selling point for anything.)

Having said that, my personal experience with TP is that the CL calendars aren’t as accurate as those from other sites. I’ve also found from my observations here that the use of TP software for planning individual DAYS in the parks is absolute invaluable, regardless of crowd levels.

9 Likes
17

This this this.

18

Agree with your last paragraph completely.

19

Great breakdown of the issue!

1 Like
20

So I agree with you on most of your points. Buggy software is no fun for a developer or a user (and I’m not sure how much that had to do with this vs changing trends.). I got called at 5:30 this morning because software my team is responsible for hit a really weird edge case that was affecting customers. Not fun - we want our customers to be happy (and to avoid 5AM troubleshooting). And I get the OPs frustration. I just wanted to add some balance and point out that TP has a history, at least for me, of being more than a WAG.

I do think the question is interesting though. I looked at most of the other CCs this morning out there for the week the OP is referencing and the consensus is pretty much moderate crowds that week, much like TouringPlans had before the change. So the question - and what will be interesting to see - which prediction ends up being more accurate? Did TP overreact to last year’s increased crowds, or overcompensate for their software bug? Or did they spot a new trend before anyone else did? Had the OP not used TP and used another site, he’d still be likely thinking moderate crowds, and if TP ends up being right, he’d be unprepared.

Totally agree on the daily planning being the killer feature of TP. It’s how I’ve navigated high CL days with little stress and lots of enjoyment. And rope drop is your friend.

2 Likes