Dan Parks converted the winning penalty with nine minutes remaining in a game where he and Shaun Connor were the only scorers with a series of penalties.
Two quick Connor scores in succession before the interval looked to have Ospreys on course for victory.
But Parks, who spurned a couple of chances to increase his points tally, kept his nerve to seal the win.
Glasgow looked the hungrier side in the early exchanges and their early effort was rewarded after just three minutes.
The Ospreys were guilty of handling on the deck - leaving Parks to stroke over a penalty from 20 metres.
The hosts continued to dominate without possessing the cutting edge to create try-scoring opportunities.