There are two types of philosophies when building a non-conference schedule for college football: scheduling for interest and scheduling for wins.
When you’ve been recruiting and winning at a high enough level for a long enough time and are looking to compete for Playoff spots then I would absolutely insist on scheduling for interest! Alabama plays other blue bloods from other conferences, Georgia has played Clemson, Ohio State has played Texas and Oregon, etc. Elite teams play higher caliber teams because they feel confident they can win them and - a few, noticeable games aside - they tend to win those games.