A certain math word has 4 letters.
Take two letters away and you have four left.
Take 1 more letter away and you have five left.
What is the word?

The word is FIVE.
Take away F and E, and you get IV (the Roman Numeral for 4).
Take away I and you get V (the Roman Numeral for 5).