Mine and DH’s first DL trip with our own kids was a combined family trip with extended family to both DL and San Diego. Our own 2 kids were both under 2 years old. The other two families had preschool and elementary aged kids & had been on a trip together 2 years before (when I was home 39 weeks pregnant with my oldest). Those two families elected for 3 days. We felt 4 days was essential with having such young kids and needing midday naps (we also stayed right across the street on Harbor so it was easy to go back and forth). It was most definitely essential and allowed us to really feel like we could take the time in the middle of the day to nap/rest with them.
Ever since then we’ve become DLR-obsessed junkies who 13 months after that first trip (on our 4th trip over those 13 months), realized that we would save a whole lot by getting APs & upgraded on the last day (and haven’t yet figured out how to quit).
Most of our trips are 3-4 day weekends when we can squeeze them but my favorite trips are the 5-6 days once we manage once a year or so.
I always recommend 4 days for a first trip if you want to not feel like you have to cram everything in and prefer to be more relaxed in your touring, and especially if you have small kids who will be cranky midday anyway unless you get them some sleep/downtime. I also add that 3 days is good if your whole group is mobile and would enjoy 3 full days without more than a leisurely lunch break. Then in true enabler fashion, I always follow that advice by saying (as has been mentioned above) the difference between 4 and 5 days is so small that if you have even partial time it’s so worth it to get and go for even just 2-3 hours on that 5th day. The math is less favorable for the jump from 3 to 4 but still really worth considering for partial days. I think that having park time on your arrival day and departure day with 3 full days in between is a great baseline for the makings of a fabulous first timer plan.