That... sounds like they may have kept the leveling system from the original.
What's your class set to? When you level in Oblivion based on your class, it increases your stats a little, and ranks up the monsters as well: your class is tied to ONLY the skills you picked at chargen.
If it is like the original, you can focus on your non-class skills to bolster those for a while... and you only level when you sleep, as well.
Oblivion's leveling system is largely regarded as a bad idea and angered a great many people.