The India opener hit 10 fours and six sixes in a sparkling knock but his side fell 34 runs short.
Peter Handscomb, Usman Khawaja and Shaun Marsh all scored half-centuries in Australia's 288-5.
India's reply started badly when they slipped to 4-3, but Rohit's knock gave them some hope of victory.
He put on 137 for the fourth wicket with MS Dhoni (51), who became only the fifth Indian batsman to pass 10,000 one-day international runs during the course of his innings.
The partnership was ended when Dhoni was given out lbw off Jason Behrendorff in a decision that would have been overturned by a review had India not already used their only referral.
The tourists, who won the Test series for the first time down under, were always behind the run-rate but reached the last six overs needing 76 to win.
However, both Ravindra Jadeja and Rohit fell in quick succession to end any hopes of a late push for victory, Jhye Richardson taking 4-26.
India were without all-rounder Hardik Pandya and batsman KL Rahul after they were suspended following "inappropriate" comments about women on a television show.