Accuracy of TouringPlans wait predictions for Christmas 2017?

For the first time in many visits, our family will be visiting WDW at Christmas time this year, from 12/23 - 12/29.

We’ve been at other relatively busy times (February President’s Day and April spring break / Easter), but never during Christmas to New Years, so used this as a good reason to resubscribe to TouringPlans. We’ve already made our 3 FP+ selections for every day (window opened ~3 days ago!) and our dining ressies.

Can anyone comment on a general sense of the accuracy of TouringPlans’ wait estimates during the crazy Christmas time period?

Like in this thread (4-16 minute wait times on Christmas day/boxing day?!?), I’m a little skeptical about some of the short predicted wait times. And it’s fine if the estimates are optimistic – we know WDW well and can adjust on-the-fly, and have already front-loaded the headliners for just after rope drop or with FP, but it’d be nice to know what to expect.

Perhaps TouringPlans doesn’t differentiate between a “normal” 10 crowd level and a Christmas time TEN crowd level?

Hi there :blush:

Firstly, the thread you linked above was mine, and I was thoroughly reassured that as the software uses historical data, the days for ‘Christmas crowd level ten’ is indeed accounted for.

While I still haven’t been-that touring plan was part of my research for our trip THIS Christmas; I have been watching testimonials from others praising the touring plans (scarily accurate is a phrase used often) and have come to trust that the software may not be minute by minute accurate, but it’s certainly going to be fairly close for the most part.

Unfortunately, Pandora and Irma are two unknowns that could potentially mess it up a bit, but overall I trust the touring plans to give me a solid plan for everything except the avatar rides. Sticking with your current plan, it should give you a pretty good idea of what to expect, with the already noted exception of the avatar rides.

I hope you have a great trip :blush:

Yeah, they do. The important thing to remember is that the CL ratings are determined based on the wait times that the model has predicted; these wait times are what is used when working out TPs, and there is no upper limit on them.

I recall a discussion with @len about having a higher number just for Christmas, but then you get into the Spinal Tap Dilemma These go to 11.mpg - YouTube

The crowd level doesn’t go above 10, because it’s just different shades of bad.

The wait time estimates for the individual attractions will definitely reflect the crowds. I wouldn’t be surprised to see wait-time estimates in excess of 200 minutes at many attractions.

The first thing I’d suggest is to use the app. A park like the MK has ~40 attractions. A couple of them will start the day in a way that isn’t typical, just because that’s the way it goes with 40 attractions.

Within 10 minutes of the park opening, once we have a couple of wait times from every ride, we’ll start adjusting that day’s forecast and your touring plan. The plan will be able to take advantage of any lines that are lower than predicted, and work around anything that is higher than predicted.

The other thing you’ll see during Christmas is secondary and tertiary attractions developing lines, as guests look for anything that doesn’t have a 60-minute wait. The lines at Haunted Mansion (a secondary attraction) will build up much faster than normal. Once that happens, you’ll start seeing lines for Tom Sawyer Island and Country Bear Jamboree.

It’s still doable, of course, especially if you get there early. But really, use the app too.

2 Likes

Len,

Ladythomas mentioned that the Avatar rides are still a bit of an unknown in terms of ride time accuracy. Out of curiosity, how many months of data is required is required for a new ride / land before you have the same level of confidence in the accuracy as you would with other rides?

Also, How frequently are crowd calendars updated? Wouldn’t the group of individuals that rescheduled thier trips due to Irma be reflected in an updated crowd calender?

Thanks

Fiel

The calendars are updated at least every 30 days. We’ve already done one post-Irma update, and another is scheduled for next week. We went back and added data around every hurricane to impact Florida since 2010, so that helps in terms of assessing delayed travel.

I think we’re reasonably comfortable with Pandora now that we’ve had a summer and fall. Ideally it’d be great to have an entire year’s worth, so we have one of every holiday. But the purpose of the machine learning algorithms is that they’re able to reason under some of this uncertainty.

1 Like

Thanks to everyone for their responses.

In particular:

… sounds like a really good idea – “No plan survives first contact with [the WDW crowds]”, and this sounds like the best way to adjust and adapt, especially in extreme conditions.


@len – I’m very curious about the geeky aspects of how TouringPlans predictions are done. I tried searching the blog. Googling on TouringPlans and words like ‘regression’ brought up the interview you did with Carl Trent. It sounds like you went from ‘standard’ regression models, to tree-based, and at the time of the interview, were looking into ensemble methods?

I consider myself a bit of a hack at data science (use it a bit at work, mostly in R, but also for anything fun, like analyzing 17 million Pokemon spawns by k-means clustering to quantitatively define ‘biomes’: https://www.reddit.com/r/TheSilphRoad/comments/5etwz9/analysis_identification_of_potential_biomes_by/).

Do you know of any publicly available large datasets of WDW wait times, just to have fun playing with the data? Obviously, TouringPlans would have a proprietary interest in keeping its current data sources private, but, for example, would you make available obsolete data that are no longer useful, from prior to new attractions or other major changes, but could still be fun to play with?

The current models use stochastic gradient boosted trees for long-term predictions. The day-of predictions are regression with some hacks.

I spent most of 2016 trying to beat the existing models. I failed. Here’s a presentation we did to UCF’s Data Science department about what was tried (mostly Python’s SciKit Learn, some TensorFlow). That also contains some examples of problems we see with the data.

Yeah, we have data available. How large do you want large to be?

So cool!!! Can’t wait to check out that presentation, just need to find some time.

“Large” – I’m kind of sure that your definition of large will be some orders of magnitude bigger than mine… I do my work on just a laptop rather than on a cluster, and found that TENS of millions of rows was too much.

Since this would just be “for fun”, I think an ideal playground dataset would be one park only (Magic Kingdom would be my favorite, and lots more attractions!) with some millions of rows and whatever set of columns convenient for you?

Maybe I could use this to get my daughters interested in data science! Or, I’ll use this as an excuse to branch out into getting more proficient in Python, instead of my usual go-to R.

Thanks much for considering!

OK. Send me an email and we can send you links to a data file.