The Rays (51-49), who are currently in third place in the American League East in front of the fourth-place Orioles (48-51), now have the Orioles creeping up from behind.
Gausman allowed five hits and three walks over six innings to pick up his seventh win of the season. He has made two starts against the Rays this season and has not allowed a run in 13 innings.
Manny Machado did his part to help Gausman by starting two critical inning-ending double plays.
Jones gave the Orioles a 1-0 lead when he hit a 1-2 fastball from Blake Snell into the left-field stands, giving the Orioles slugger 19 for the season.