Verses 2 and 3 tell us not to speak for the sake of speaking. I'm sure we've all been in Christian circles, whether Bible studies or prayer meetings or just hanging out with a group of Christian friends where someone feels they have to say something and ends up saying something foolish or Biblically incorrect.
The verses following these two tell us not to make rash vows to God. We've all heard of a situation where someone prays "God, if you get me out of this, I'll become a priest or a nun or a preacher" or something. This is complete foolishness. I think it also makes God out to be a sucker.
The Book of James speaks quite a bit about how powerful the tongue is. We should always be careful about what we say and be mindful of those times when we shouldn't say anything at all.