Rolling a Spell Check
Spells may fail if the target actively resists. The victim may make a opposed check vs. the spellcaster to resist the spell, unless it says otherwise. The spellcaster and the victim both use the attribute as prescribed by spell's type of magic for the roll. This is called a spell check. Success means the spell fizzles out and does not affect the target. In this case, the MP spent to cast the spell are lost.
Concentration & Ending Spells
Some spells specifically require you to concentrate for the entire duration of the spell, or else the spell ends and your victims are released from its effects. Such spells will say so explicitly in their text. Other spells may imply that concentration is necessary, but from a mechanical point of view these are directed spells. For example, a spell like Ensorcell specifies concentration explicitly, whereas a spell like Control (Gravity) only implies it.
More on Concentration
See the rules on concentration in spell logic for more details.
Ending Spells Prematurely
A spellcaster may end a spell he casts at any time by his own volition as a reaction. If a spellcaster is unconscious and he's cast a directed or concentration spell, the spell ends. Note that spells which are not directed (meaning, they do not require the spellcaster to make decisions about them from round to round) persist for their full duration, even if the spellcaster is unconscious. For example, if a spellcaster cast the spell Enchant Weapon for 6 rounds and is knocked unconscious before those 6 rounds are up, the enchantment persists until the end of the 6th round.
Note that spellcasters are still conscious when paralyzed, although if they are held, they will lose concentration and their directed spell will end because they're no longer in the same time stream as their spell's effects. Similarly, if a spellcaster is subject to the mad, feebleminded, or enraged status effects, their concentration and directed spells will end because they are no longer of sound mind to maintain them. Finally, spells that require the spellcaster to continuously see, hear, or speak to the target are ended when the spellcaster is blinded, deafened, or silenced.
Resisting Ongoing Effects
When you're subject to a spell that renders you helpless or otherwise incapacitated, for each round you're subject to the effects of the spell you may make a spell check to break the spell's hold over you. This is the case for directed spells and spells maintained by concentration.
Each round after the spell is in effect, you make a new spell check to resist at the start of your turn. This check is made at an escalating +2, and the spellcaster must roll to set a new TN for you to resist. Therefore if you're subject to petrification, on the first round after you've been petrified, you may attempt to resist at a +2. On the second round, at a +4, and the third round, at a +6, and so on.