Over the past few months I have started to describe ISO to my photography students like a pair of sunglasses. You have sunlight hitting your eyes. When wearing sunglasses your eyes feel all nice and comfortable even with all that sunlight. However, if you take off your sunglasses, the same amount of light is hitting your eyes, except now it hurts and you squint. There is a cost to taking off your sunglasses.
It is the same thing with ISO. There is a certain amout of light hitting your DSLR camera's sensor. If you increase the ISO setting, the same amount of light will hit the sensor, except now the camera's sensor with a higher ISO is more sensitive to that same amount of light. This is just like your eyes getting more sensitive without sunglasses. The tradeoff with no sunglasses is squinting. The tradeoff with high ISO is introducing noise, grain and artifacting into the photograph.
This is why I usually only increase ISO last when trying to get the correct exposure. Most of the time I will use the largest aperture possible with the slowest shutter speed possible. Once I hit those limits, only then do I start to increase ISO because I want to maximize image quality as much as possible.